Multi-scene live broadcast interaction system and method based on XR virtual production

Abstract

The invention discloses a multi-scene live broadcast interaction system and method based on XR virtual production, and relates to the field of data interaction, and the system comprises a monitoring module which is used for monitoring XR special effect animations independently selected by audiences in a live broadcast room of a live broadcast platform in real time, and storing the monitored selection result of the audiences for the XR special effect animations in the live broadcast room of the live broadcast platform; the cleaning module is used for receiving the XR special effect animation selection result stored in the monitoring module and cleaning the stored XR special effect animation selection result; through XR special effect animation management, an administrator can consult conditions at any time so as to timely know audience preferences, and meanwhile, the system can intelligently process audience selection results, recognize and clear repeated or invalid special effect selections, reasonably plan playing time according to attributes such as duration, position and size of special effects, and ensure that the special effects are played orderly.

Description

Technical Field

[0001] The present invention relates to the field of data interaction technology, and specifically to a multi-scene live broadcast interactive system and method based on XR virtual production. Background Art

[0002] Online live broadcast interaction is a model that relies on Internet technology to achieve real-time communication between the host and the audience during the live broadcast.

[0003] The invention patent application with application number 202211341211.2 discloses a multi-person interactive network live broadcast method based on a virtual scene, including: step 1: constructing a virtual scene based on a live broadcast device; step 2: determining a default viewing position for each viewing device, and obtaining a first virtual position matching the default viewing position from the virtual scene; step 3: recording the viewing position movement trajectory of each viewing device, and matching the corresponding virtual viewing trajectory to the corresponding viewing device according to the viewing position movement trajectory; step 4: when a live broadcast interaction request of the corresponding viewing user is captured at the current moment, determining the required rendering mode and the corresponding voice object and video object according to the live broadcast interaction request; step 5: performing a first rendering of the voice track of the voice object in the virtual viewing trajectory and a second rendering of the video track of the video object in the virtual trajectory based on the required rendering mode, to realize multi-person interactive network live broadcast. The application aims to solve the problem that "on common network interaction platforms, such as Douyin and Kuaishou, users generally interact with live broadcasts in the user live broadcast room through chatting, sending gifts, or voice access, and common multi-person interactions are voice or video interactions, which cannot meet the user's interactive experience."

[0004] However, in scenarios where viewers give virtual gifts to users in the live broadcast room, some virtual gifts with special effects will interact with the body language of the users in the live broadcast room. Currently, there is no technology to coordinate the interaction between virtual gifts and user body language for such scenarios.

[0005] To this end, a multi-scene live interactive system and method based on XR virtual production is proposed. Summary of the Invention

[0006] In response to the above-mentioned shortcomings of the existing technology, the present invention provides a multi-scene live interactive system and method based on XR virtual production, which can effectively solve the problems of the existing technology.

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0008] The present invention discloses a multi-scene live interactive system based on XR virtual production, comprising:

[0009] The monitoring module is used to monitor the XR special effects animations selected by the audience in the live broadcast room of the live broadcast platform in real time, and store the selection results of the audience for the XR special effects animations in the live broadcast room of the monitored live broadcast platform; the cleaning module is used to receive the selection results of the XR special effects animations stored in the monitoring module, and clean the stored XR special effects animation selection results; the pushing module is used to receive the cleaning results of the XR special effects animation selection results in the cleaning module, and push the XR special effects animations retained by the cleaning results to the live broadcast room for playback; the tracking module is used to track the dynamic target outline in the live broadcast room image during the playback of the XR special effects animation, decide the playback position of the XR special effects animation based on the dynamic target outline, and apply it to the playback instructions of the XR special effects animation in the pushing module; the refresh module is used to monitor the running status of the tracking module, and refresh the system operation after each monitoring of the end of the tracking module operation.

[0010] Furthermore, during the operation stage of the monitoring module, the XR special effects animations independently selected by the audience in the live broadcast room of the live broadcast platform come from the live broadcast room of the live broadcast platform. When the selection results of the XR special effects animations by the audience in the live broadcast room of the live broadcast platform are stored, they are sorted and stored based on the selection time of the XR special effects animations, and the sorted and stored XR special effects animation selection results are synchronously backed up, and the backed-up XR special effects animation selection results are fed back to the live broadcast room background of the live broadcast platform in real time for the live broadcast room management user to read.

[0011] Furthermore, a logic unit and a configuration unit are provided at the lower level of the cleaning module. The logic unit is used to set the cleaning logic and control the cleaning module to clean the received selection result of the XR special effect animation based on the cleaning logic. The configuration unit is used to receive the cleaning result of the selection result of the XR special effect animation and configure the XR special effect animation playback time domain based on the XR special effect animation included in the cleaning result.

[0012] The cleaning logic set in the logic unit is:

[0013] Run the received XR special effect animation based on the time sequence traversal cleaning module, identify at least three adjacent identical XR special effect animations in the selection results of the XR special effect animation, and use the identified XR special effect animations as the cleaning target group;

[0014] A time interval is determined by identifying the source selection time of each cleaning target in the cleaning target group. According to the properties of the XR special effect animation corresponding to the cleaning target, a post-positioned sub-time interval is selected in the determined time interval as the playback time interval of the XR special effect animation corresponding to the cleaning target;

[0015] The cleaning targets in the cleaning target group belonging to the time interval other than the unselected sub-time interval in the time interval are cleared.

[0016] Furthermore, the properties of the XR special effect animation include: duration, special effect position, and special effect animation size;

[0017] When selecting a post-positioned sub-time interval in a determined time interval, the duration of the selected sub-time interval is equal to the duration contained in the attributes of its corresponding XR special effect animation, and the end timestamp of the selected sub-time interval is the same as the end timestamp of the original determined time interval;

[0018] Among them, the XR special effects animation that is not identified as the cleaning target group uses the duration in its own attributes as the playback time interval, and places the playback time interval outside the originally determined time interval and in an adjacent position, and the adjacent position used for placement corresponds to the start time of the time interval, which is placed after the selection time of the XR special effects animation.

[0019] Furthermore, during the operation phase of the push module, all XR special effects animations configured with playback time intervals, that is, the XR special effects animations retained by the cleaning results, will be pushed to the live broadcast room based on their configured playback time intervals and played in the live broadcast room.

[0020] Furthermore, the tracking module is provided with a capture unit and a labeling unit at the lower level. The capture unit is used to capture the dynamic target outline in the live broadcast room image during the playback of the XR special effect animation. The labeling unit is used to receive the dynamic target outline captured by the capture unit, identify and label the playback position of the XR special effect animation in the dynamic target outline;

[0021] Among them, after receiving the dynamic target outline, the labeling unit identifies and distinguishes the outlines of each body part of the dynamic target in the dynamic target outline based on the dynamic target detection algorithm, and performs corresponding body part text information labeling, and simultaneously obtains the special effect position in the attributes of the currently playing XR special effect animation, and searches in the text information labeling of each body part based on the special effect position, and uses the dynamic target body part outline with the same body part text information labeling source retrieved as the playback position of the XR special effect animation.

[0022] Furthermore, when capturing the dynamic target contour, the capture unit obtains two adjacent frames in the live broadcast room image screen, uses all pixels in the two frames as dynamic target contour capture targets, and performs a capture operation:

[0023] No1: Convert both frames into grayscale images;

[0024] No2: Calculate the difference image of two grayscale images:

[0025] D(x,y)=|G1(x,y)-G2(x,y)|;

[0026] Where: G1(x,y) and G2(x,y) represent two frames converted into grayscale images;

[0027] No3: Calculate the gradients of the two grayscale images in the x and y directions respectively, and based on the gradient calculation results, further calculate the gradient amplitudes of the two grayscale images;

[0028] No4: Calculate the difference image of the gradient magnitude:

[0029] MD(x,y)=|M1(x,y)-M2(x,y)|;

[0030] Where: M1(x,y) and M2(x,y) represent the gradient amplitudes of the two grayscale images;

[0031] No5: Fusion difference and gradient amplitude information:

[0032] F(x,y)=α×D(x,y)+(1-α)×MD(x,y);

[0033] Where: F(x,y) represents the fused image; α is the weight, which ranges from 0 to 1;

[0034] No.6: Adaptive threshold processing: The fusion image F(x,y) is binarized using the adaptive threshold method to obtain a binary image B(x,y); the preset threshold is T, then;

[0035]

[0036] Among them, after obtaining the binary image based on the above logic, the connected region analysis algorithm is used to find the connected regions in the binary image, and the connected regions represent the contours of the dynamic target.

[0037] Furthermore, after the user of the marking unit identifies and marks the XR special effects animation playback position in the dynamic target outline, the XR special effects animation is synchronously scaled based on the size of the XR special effects animation playback position, so that the XR special effects animation is always covered in the area defined by the XR special effects animation playback position during the playback process, and the edge of the XR special effects animation coincides with the edge of the XR special effects animation playback position.

[0038] Furthermore, the monitoring module is interactively connected to a cleaning module via a wireless network, the cleaning module is interactively connected to a logic unit and a configuration unit via a wireless network, the cleaning module is interactively connected to a push module and a tracking module via a wireless network, the tracking module is interactively connected to a capture unit and a labeling unit via a wireless network, and the tracking module is interactively connected to a refresh module via a wireless network.

[0039] On the other hand, the multi-scene live interactive method based on XR virtual production includes the following steps:

[0040] Monitor the XR special effects animations selected by the audience in the live broadcast room of the live broadcast platform in real time, and store the XR special effects animations; set the XR special effects animation cleaning logic, and clean the stored XR special effects animations based on the XR special effects animation cleaning logic; pick up the dynamic target outline in the live broadcast room image screen, configure the playback time interval for the cleaned XR special effects animation, and place the XR special effects animation with the configured playback time interval into the dynamic target outline based on the properties of the XR special effects animation; control the XR special effects animation to play synchronously with the image screen in the live broadcast room; after all XR special effects animations are played, the refresh step is executed.

[0041] Compared with the prior art, the technical solution provided by the present invention has the following beneficial effects:

[0042] The present invention provides a multi-scene live interactive system and method based on XR virtual production. During operation, the system and method provide administrators with access conditions at any time through the management of XR special effects animation, so as to timely understand the audience's preferences. At the same time, the system will intelligently process the audience's selection results, identify and remove repeated or invalid special effects selections, and reasonably plan the playback time according to the duration, position, size and other attributes of the special effects to ensure the orderly playback of special effects. When the special effects are playing, the system can accurately capture the contours of dynamic targets in the live broadcast room image, identify and distinguish the body parts of dynamic targets through detection algorithms, accurately locate the playback area according to the preset position of the special effects, and automatically scale the special effects to make it completely cover the target area, thereby achieving seamless fitting of special effects and dynamic images, thereby enhancing the audience's sense of participation and interactivity, and making the special effects display more accurate and vivid, effectively improving the fun and attractiveness of live broadcasts, and providing convenient and efficient technical support for live broadcast operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort.

[0044] Figure 1 This is a structural diagram of a multi-scene live interactive system based on XR virtual production;

[0045] Figure 2 This is a flowchart of a multi-scene live interactive method based on XR virtual production. DETAILED DESCRIPTION

[0046] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0047] The present invention will be further described below with reference to the embodiments.

[0048] Example 1:

[0049] The multi-scene live interactive system based on XR virtual production in this embodiment is as follows: Figure 1 Shown, including:

[0050] A monitoring module is used to monitor in real time the XR special effects animations selected by viewers in the live broadcast room of the live broadcast platform, and store the selection results of the viewers in the live broadcast room of the live broadcast platform for the XR special effects animations;

[0051] During the operation phase of the monitoring module, the XR special effects animations selected by the audience in the live broadcast room of the live broadcast platform are derived from the live broadcast room of the live broadcast platform. When the selection results of the XR special effects animations by the audience in the live broadcast room of the monitored live broadcast platform are stored, they are sorted and stored based on the selection time of the XR special effects animations, and the sorted and stored XR special effects animation selection results are backed up synchronously. The backed-up XR special effects animation selection results are fed back to the live broadcast room backend of the live broadcast platform in real time for the live broadcast room management user to read;

[0052] A cleaning module is used to receive the selection results of the XR special effects animation stored in the monitoring module and clean the stored XR special effects animation selection results;

[0053] The cleaning module is provided with a logic unit and a configuration unit at the lower level. The logic unit is used to set the cleaning logic and control the cleaning module to clean the selected results of the received XR special effects animation based on the cleaning logic. The configuration unit is used to receive the cleaning results of the selected results of the XR special effects animation and configure the XR special effects animation playback time domain based on the XR special effects animation contained in the cleaning results.

[0054] The cleaning logic set in the logic unit is:

[0055] Run the received XR special effect animation based on the time sequence traversal cleaning module, identify at least three adjacent identical XR special effect animations in the selection results of the XR special effect animation, and use the identified XR special effect animations as the cleaning target group;

[0056] A time interval is determined by identifying the source selection time of each cleaning target in the cleaning target group. According to the properties of the XR special effect animation corresponding to the cleaning target, a post-positioned sub-time interval is selected in the determined time interval as the playback time interval of the XR special effect animation corresponding to the cleaning target;

[0057] Clearing the cleaning targets in the cleaning target group within the time interval other than the unselected sub-time interval in the time interval;

[0058] The properties of XR special effects animation include: duration, special effects position, and special effects animation size;

[0059] When selecting a post-positioned sub-time interval in a determined time interval, the duration of the selected sub-time interval is equal to the duration contained in the attributes of its corresponding XR special effect animation, and the end timestamp of the selected sub-time interval is the same as the end timestamp of the original determined time interval;

[0060] Among them, the XR special effects animation that is not identified as the cleaning target group uses the duration in its own attributes as the playback time interval, and the playback time interval is placed outside the originally determined time interval and adjacent to it. The adjacent position used for placement corresponds to the start time of the time interval, which is placed after the selection time of the XR special effects animation;

[0061] The push module is used to receive the cleaning results of the XR special effects animation selected in the cleaning module, and push the XR special effects animation retained in the cleaning results to the live broadcast room for playback;

[0062] During the push module operation phase, all XR special effects animations configured with a playback time interval, i.e., the XR special effects animations retained after the cleaning results, will be pushed to the live broadcast room based on their configured playback time interval and played in the live broadcast room;

[0063] The tracking module is used to track the dynamic target outline in the live broadcast room image during the playback of XR special effects animations, determine the playback position of the XR special effects animation based on the dynamic target outline, and apply it to the playback instructions of the XR special effects animation in the push module;

[0064] The tracking module is equipped with a capture unit and a labeling unit. The capture unit is used to capture the dynamic target outline in the live broadcast room image during the playback of the XR special effects animation. The labeling unit is used to receive the dynamic target outline captured by the capture unit, identify and label the XR special effects animation playback position in the dynamic target outline;

[0065] Among them, after receiving the dynamic target outline, the labeling unit identifies and distinguishes the contours of each body part of the dynamic target in the dynamic target outline based on the dynamic target detection algorithm, and performs corresponding body part text information labeling, and simultaneously obtains the special effect position in the attributes of the currently playing XR special effect animation, searches in the text information labels of each body part based on the special effect position, and uses the dynamic target body part outline with the same body part text information labeling source retrieved as the playback position of the XR special effect animation;

[0066] When capturing the dynamic target outline, the capture unit obtains two adjacent frames in the live broadcast room image screen, uses all pixels in the two frames as the dynamic target outline capture target, and performs the capture operation:

[0067] No1: Convert both frames into grayscale images;

[0068] No2: Calculate the difference image of two grayscale images:

[0069] D(x,y)=|G1(x,y)-G2(x,y)|;

[0070] Where: G1(x,y) and G2(x,y) represent two frames converted into grayscale images;

[0071] No3: Calculate the gradients of the two grayscale images in the x and y directions respectively, and based on the gradient calculation results, further calculate the gradient amplitudes of the two grayscale images;

[0072] No4: Calculate the difference image of the gradient magnitude:

[0073] MD(x,y)=|M1(x,y)-M2(x,y)|;

[0074] Where: M1(x,y) and M2(x,y) represent the gradient amplitudes of the two grayscale images;

[0075] No5: Fusion difference and gradient amplitude information:

[0076] F(x,y)=α×D(x,y)+(1-α)×MD(x,y);

[0077] Where: F(x,y) represents the fused image; α is the weight, which ranges from 0 to 1;

[0078] No.6: Adaptive threshold processing: The fusion image F(x,y) is binarized using the adaptive threshold method to obtain a binary image B(x,y); the preset threshold is T, then;

[0079]

[0080] After obtaining the binary image based on the above logic, the connected region analysis algorithm is used to find the connected regions in the binary image. The connected regions represent the contours of the dynamic target.

[0081] By calculating the above logic formula, the capture logic of the dynamic target contour is limited;

[0082] After the user of the marking unit identifies and marks the XR special effect animation playback position in the dynamic target outline, the XR special effect animation is synchronously scaled based on the size of the XR special effect animation playback position, so that the XR special effect animation is always covered in the area defined by the XR special effect animation playback position during playback, and the edge of the XR special effect animation coincides with the edge of the XR special effect animation playback position;

[0083] The refresh module is used to monitor the running status of the tracking module and refresh the system operation every time the tracking module is detected to have finished running;

[0084] The monitoring module is interactively connected to the cleaning module through a wireless network. The cleaning module is interactively connected to the logic unit and the configuration unit through a wireless network. The cleaning module is interactively connected to the push module and the tracking module through a wireless network. The tracking module is interactively connected to the capture unit and the labeling unit through a wireless network. The tracking module is interactively connected to the refresh module through a wireless network.

[0085] In this embodiment, the monitoring module runs in real-time monitoring of the XR special effects animations selected by the audience in the live broadcast room of the live broadcast platform, and stores the selection results of the audience for the XR special effects animations in the live broadcast room of the monitored live broadcast platform. The cleaning module runs in the post-position to receive the selection results of the XR special effects animations stored in the monitoring module, and cleans the stored XR special effects animation selection results. The logic unit synchronously sets the cleaning logic, and controls the cleaning module to clean the received XR special effects animation selection results based on the cleaning logic. The configuration unit receives the cleaning results of the XR special effects animation selection results in real time, and configures the XR special effects animation playback time domain based on the XR special effects animation contained in the cleaning results. The push module is used to receive the XR special effects animations in the cleaning module. The cleaning result of the XR special effects animation selection result is pushed to the live broadcast room for playback. The tracking module further tracks the dynamic target outline in the live broadcast room image during the playback of the XR special effects animation, and decides the playback position of the XR special effects animation based on the dynamic target outline, and applies it to the playback instruction of the XR special effects animation in the push module. The capture unit captures the dynamic target outline in the live broadcast room image during the playback of the XR special effects animation in real time. The marking unit synchronously receives the dynamic target outline captured by the capture unit, identifies and marks the playback position of the XR special effects animation in the dynamic target outline, and finally monitors the operation status of the tracking module through the refresh module. After each monitoring of the end of the tracking module operation, the system operation is refreshed.

[0086] Through the operation of the above system, the virtual gift interaction scenario in the live broadcast room is optimized, so that the special effects playback of virtual gifts given by the audience in the live broadcast room can be managed more logically, and the special effects playback of virtual gifts can be more consistent with the presentation of special effects by the anchor in the live broadcast room, thereby enhancing the audience's sense of participation and interactive effect in the live broadcast room.

[0087] Example 2:

[0088] In terms of specific implementation, based on Example 1, this example refers to Figure 2 The multi-scene live interactive system based on XR virtual production in Example 1 is further described in detail:

[0089] The multi-scene live interactive method based on XR virtual production includes the following steps:

[0090] Step 1: Real-time monitoring of the XR special effects animations selected by the audience in the live broadcast room of the live broadcast platform, and storage of the XR special effects animations;

[0091] Step 2: Set the XR special effects animation cleaning logic and clean the stored XR special effects animation based on the XR special effects animation cleaning logic;

[0092] Step 3: Pick up the dynamic target outline in the live broadcast room image, configure the playback time interval for the cleaned XR special effects animation, and place the XR special effects animation with the configured playback time interval into the dynamic target outline based on the properties of the XR special effects animation;

[0093] Step 4: Control the XR special effects animation to play synchronously with the image in the live broadcast room;

[0094] Step 5: After all XR special effects animations are played, the refresh step is executed.

[0095] In summary, during the operation of the systems and methods in the above embodiments, through the management of XR special effects animation, the administrator is provided with access conditions at any time so as to understand the audience's preferences in a timely manner. At the same time, the system will intelligently process the audience's selection results, identify and remove duplicate or invalid special effects selections, and reasonably plan the playback time according to the duration, position, size and other attributes of the special effects to ensure the orderly playback of special effects. When the special effects are playing, the system can accurately capture the contours of dynamic targets in the live broadcast room image, identify and distinguish the body parts of dynamic targets through detection algorithms, accurately locate the playback area according to the preset position of the special effects, and automatically scale the special effects size to fully cover the target area, thereby achieving seamless integration of special effects and dynamic images, thereby enhancing the audience's sense of participation and interactivity, and making the special effects display more accurate and vivid, effectively improving the fun and attractiveness of the live broadcast, and providing convenient and efficient technical support for live broadcast operations.

[0096] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A multi-scene live interactive system based on XR virtual production, characterized by: include: A monitoring module is used to monitor in real time the XR special effects animations selected by viewers in the live broadcast room of the live broadcast platform, and store the selection results of the viewers in the live broadcast room of the live broadcast platform for the XR special effects animations; A cleaning module is used to receive the selection results of the XR special effects animation stored in the monitoring module and clean the stored XR special effects animation selection results; The push module is used to receive the cleaning results of the XR special effects animation selected in the cleaning module, and push the XR special effects animation retained in the cleaning results to the live broadcast room for playback; The tracking module is used to track the dynamic target outline in the live broadcast room image during the playback of XR special effects animations, determine the playback position of the XR special effects animation based on the dynamic target outline, and apply it to the playback instructions of the XR special effects animation in the push module; The refresh module is used to monitor the running status of the tracking module and refresh the system operation every time the tracking module is monitored to be finished.

2. The multi-scene live interactive system based on XR virtual production according to claim 1 is characterized in that: During the operation stage of the monitoring module, the XR special effects animations independently selected by the audience in the live broadcast room of the live broadcast platform come from the live broadcast room of the live broadcast platform. When the selection results of the XR special effects animations by the audience in the live broadcast room of the live broadcast platform are stored, they are sorted and stored based on the selection time of the XR special effects animations, and the sorted and stored XR special effects animation selection results are synchronously backed up. The backed-up XR special effects animation selection results are fed back to the live broadcast room background of the live broadcast platform in real time for the live broadcast room management user to read.

3. The multi-scene live interactive system based on XR virtual production according to claim 1 is characterized in that: The cleaning module is provided with a logic unit and a configuration unit at a lower level. The logic unit is used to set the cleaning logic and control the cleaning module to clean the received selection result of the XR special effect animation based on the cleaning logic. The configuration unit is used to receive the cleaning result of the selection result of the XR special effect animation and configure the XR special effect animation playback time domain based on the XR special effect animation included in the cleaning result; The cleaning logic set in the logic unit is: Run the received XR special effect animation based on the time sequence traversal cleaning module, identify at least three adjacent identical XR special effect animations in the selection results of the XR special effect animation, and use the identified XR special effect animations as the cleaning target group; A time interval is determined by identifying the source selection time of each cleaning target in the cleaning target group. According to the properties of the XR special effect animation corresponding to the cleaning target, a post-positioned sub-time interval is selected in the determined time interval as the playback time interval of the XR special effect animation corresponding to the cleaning target; The cleaning targets in the cleaning target group belonging to the time interval other than the unselected sub-time interval in the time interval are cleared.

4. The multi-scene live interactive system based on XR virtual production according to claim 3 is characterized in that: The properties of the XR special effects animation include: duration, special effects position, and special effects animation size; When selecting a post-positioned sub-time interval in a determined time interval, the duration of the selected sub-time interval is equal to the duration contained in the attributes of its corresponding XR special effect animation, and the end timestamp of the selected sub-time interval is the same as the end timestamp of the original determined time interval; Among them, the XR special effects animation that is not identified as the cleaning target group uses the duration in its own attributes as the playback time interval, and places the playback time interval outside the originally determined time interval and in an adjacent position, and the adjacent position used for placement corresponds to the start time of the time interval, which is placed after the selection time of the XR special effects animation.

5. The multi-scene live interactive system based on XR virtual production according to claim 1 is characterized in that: During the operation phase of the push module, all XR special effects animations configured with playback time intervals, that is, the XR special effects animations retained by the cleaning results, are pushed to the live broadcast room based on their configured playback time intervals and played in the live broadcast room.

6. The multi-scene live interactive system based on XR virtual production according to claim 1 is characterized in that: The tracking module is provided with a capture unit and a marking unit at the lower level. The capture unit is used to capture the dynamic target outline in the live broadcast room image during the playback of the XR special effect animation. The marking unit is used to receive the dynamic target outline captured by the capture unit, identify and mark the XR special effect animation playback position in the dynamic target outline; Among them, after receiving the dynamic target outline, the labeling unit identifies and distinguishes the outlines of each body part of the dynamic target in the dynamic target outline based on the dynamic target detection algorithm, and performs corresponding body part text information labeling, and simultaneously obtains the special effect position in the attributes of the currently playing XR special effect animation, and searches in the text information labeling of each body part based on the special effect position, and uses the dynamic target body part outline with the same body part text information labeling source retrieved as the playback position of the XR special effect animation.

7. The multi-scene live interactive system based on XR virtual production according to claim 6 is characterized in that: When capturing the dynamic target outline, the capture unit obtains two adjacent frames in the live broadcast room image screen, uses all pixels in the two frames as the dynamic target outline capture target, and performs the capture operation: No1: Convert both frames into grayscale images; No2: Calculate the difference image of two grayscale images: D(x,y)=|G1(x,y)-G2(x,y)|; Where: G1(x,y) and G2(x,y) represent two frames converted into grayscale images; No3: Calculate the gradients of the two grayscale images in the x and y directions respectively, and based on the gradient calculation results, further calculate the gradient amplitudes of the two grayscale images; No4: Calculate the difference image of the gradient magnitude: MD(x,y)=|M1(x,y)-M2(x,y)|; Where: M1(x,y) and M2(x,y) represent the gradient amplitudes of the two grayscale images; No5: Fusion difference and gradient amplitude information: F(x,y)=α×D(x,y)+(1-α)×MD(x,y); Where: F(x,y) represents the fused image; α is the weight, which ranges from 0 to 1; No.6: Adaptive threshold processing: The fused image F(x,y) is binarized using the adaptive threshold method to obtain a binary image B(x,y); the preset threshold is T, then; Among them, after obtaining the binary image based on the above logic, the connected region analysis algorithm is used to find the connected regions in the binary image, and the connected regions represent the contours of the dynamic target.

8. The multi-scene live interactive system based on XR virtual production according to claim 6 is characterized in that: After the user of the marking unit identifies and marks the XR special effects animation playback position in the dynamic target outline, the XR special effects animation is synchronously scaled based on the size of the XR special effects animation playback position, so that the XR special effects animation is always covered in the area defined by the XR special effects animation playback position during the playback process, and the edge of the XR special effects animation coincides with the edge of the XR special effects animation playback position.

9. The multi-scene live interactive system based on XR virtual production according to claim 1 is characterized in that: The monitoring module is interactively connected to the cleaning module via a wireless network, the cleaning module is interactively connected to the logic unit and the configuration unit via a wireless network, the cleaning module is interactively connected to the push module and the tracking module via a wireless network, the tracking module is interactively connected to the capture unit and the labeling unit via a wireless network, and the tracking module is interactively connected to the refresh module via a wireless network.

10. A multi-scene live interactive method based on XR virtual production, the method being an implementation method of a multi-scene live interactive system based on XR virtual production according to any one of claims 1 to 9, characterized in that: The following steps are involved: Step 1: Real-time monitoring of the XR special effects animations selected by the audience in the live broadcast room of the live broadcast platform, and storage of the XR special effects animations; Step 2: Set the XR special effects animation cleaning logic and clean the stored XR special effects animation based on the XR special effects animation cleaning logic; Step 3: Pick up the dynamic target outline in the live broadcast room image, configure the playback time interval for the cleaned XR special effects animation, and place the XR special effects animation with the configured playback time interval into the dynamic target outline based on the properties of the XR special effects animation; Step 4: Control the XR special effects animation to play synchronously with the image in the live broadcast room; Step 5: After all XR special effects animations are played, the refresh step is executed.

Images