Virtual-real interaction method and device for virtual shooting and storage medium

Abstract

The present disclosure relates to a virtual-real interaction method and device for virtual shooting and a storage medium. The method comprises: determining a positional relationship between an entity object in front of a screen and a virtual scene displayed in the screen during virtual shooting; triggering a target interaction operation between the entity object and the virtual scene when the positional relationship meets a condition for triggering the target interaction operation; and changing the virtual scene in response to triggering of the target interaction operation. Thus, real-time interaction between the entity object and the virtual scene can be achieved.

Description

Technical Field

[0001] This disclosure relates to the field of virtual photography technology, and in particular to a virtual photography method, apparatus, and storage medium for virtual-real interaction. Background Technology

[0002] Virtual shooting, also known as digital background shooting, is a film and television shooting process that uses video or real-time rendered images projected onto a screen, such as an LED screen, as a digital background.

[0003] Virtual shooting solutions have significant technological advantages over traditional green screen keying. By projecting a real-time rendered 3D scene onto a screen, actors can be placed in a virtual scene, enabling the shooting of outdoor scenes or science fiction backgrounds in a studio.

[0004] While existing virtual shooting solutions can place actors in virtual scenes, actors cannot interact with objects in the virtual scene in real time, such as pushing objects, having virtual characters follow the actor's movements, or leaving footprints on the beach. This results in a gap between the shot and the real world, or the desired shooting effect cannot be achieved. Summary of the Invention

[0005] In view of this, this disclosure proposes a virtual shooting method, device and storage medium for virtual-real interaction, so as to realize real-time interaction between physical objects and virtual scenes.

[0006] According to a first aspect of this disclosure, a virtual-real interaction method for virtual shooting is provided, comprising: determining, during virtual shooting, the positional relationship between a physical object in front of a screen and a virtual scene displayed on the screen; when the positional relationship satisfies the conditions for triggering a target interaction operation, triggering a target interaction operation between the physical object and the virtual scene; and the virtual scene changing in response to the triggering of the target interaction operation.

[0007] In one possible implementation, determining the positional relationship between an entity object and a virtual scene includes: acquiring positioning data of positioning points installed on the entity object in real time; and determining the positional relationship between the entity object and the virtual scene based on the positioning data.

[0008] In one possible implementation, triggering a target interaction operation between an entity object and a virtual scene includes: generating an interaction control instruction corresponding to the target interaction operation and sending the interaction control instruction to a rendering engine, wherein the interaction control instruction is used to define the changes that the virtual scene should undergo in response to the triggering of the target interaction operation.

[0009] In one possible implementation, generating the interactive control instruction corresponding to the target interactive operation includes: determining, based on the positioning data of the positioning point, modifications to the model and / or effects of the virtual scene in response to the triggering of the target interactive operation; and generating the interactive control instruction based on the modifications to the model and / or effects of the virtual scene to be made.

[0010] In one possible implementation, the virtual-real interaction method further includes: in response to a user's editing operation on an editable interaction template, determining a target interaction template, wherein the target interaction template includes a positioning point corresponding to the target interaction operation and a change method of the virtual scene corresponding to the target interaction operation.

[0011] In one possible implementation, when the positional relationship satisfies the condition for a target interaction operation, a target interaction operation between the entity object and the virtual scene is triggered, including: when the positional relationship satisfies the condition for a target interaction operation, triggering an interaction script generated according to a target interaction template corresponding to the target interaction operation, and executing the interaction script to realize the target interaction operation.

[0012] In one possible implementation, the target interactive operation is achieved by executing the interaction script, including: determining model data and / or special effects data in the next frame of the virtual scene according to the interaction script; generating interactive control instructions according to the model data and / or special effects data, and sending the interactive control instructions to the rendering engine.

[0013] In one possible implementation, the virtual-real interaction method further includes: performing offset correction on the positioning data; correspondingly, determining the positional relationship between the entity object and the virtual scene based on the positioning data includes: determining the positional relationship between the entity object and the virtual scene based on the offset-corrected positioning data.

[0014] According to a second aspect of this disclosure, a virtual-real interaction device for virtual shooting is provided, comprising: a determining unit for determining the positional relationship between an entity object and a virtual scene; and a triggering unit for triggering a target interaction operation between the entity object and the virtual scene when the positional relationship satisfies the conditions for triggering the target interaction operation; wherein the virtual scene changes in response to the triggering of the target interaction operation.

[0015] According to a third aspect of this disclosure, a virtual-real interaction device for virtual shooting is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-described virtual-real interaction method when executing the instructions stored in the memory.

[0016] According to a fourth aspect of this disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions, when executed by a processor, implement the above-described virtual-real interaction method.

[0017] According to the virtual-real interaction method, apparatus, and storage medium disclosed herein, during virtual shooting, the positional relationship between a physical object in front of the screen and a virtual scene displayed on the screen is determined. When the positional relationship satisfies the conditions for triggering a target interaction operation, the target interaction operation between the physical object and the virtual scene is triggered, and the virtual scene changes in response to the triggering of the target interaction operation. Therefore, whether to trigger the target interaction operation can be controlled based on whether the positional relationship between the physical object and the virtual scene satisfies the conditions for triggering the target interaction operation, and the virtual scene changes in response to the triggering of the target interaction operation. This enhances the interaction control logic between the physical object and the virtual scene, enabling real-time interaction between the real character and the virtual scene, such as knocking away a wooden box in the virtual scene or leaving footprints on the beach. This improves the presentation effect of the virtual shooting solution, making the virtual shooting effect more realistic, thus meeting the needs of more shooting scenarios and increasing the application scenarios of the virtual shooting solution.

[0018] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0019] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this disclosure together with the specification and serve to explain the principles of this disclosure.

[0020] Fig. 1 A flowchart illustrating a virtual-real interaction method for virtual shooting according to an exemplary embodiment of the present disclosure is shown.

[0021] Fig. 2 A system architecture diagram illustrating a virtual-real interaction method for virtual shooting according to an exemplary embodiment of the present disclosure is shown.

[0022] Fig. 3 A block diagram of a virtual-real interaction device for virtual shooting according to an exemplary embodiment of the present disclosure is shown. Detailed Implementation

[0023] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0024] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0025] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0026] As described above, existing virtual filming solutions can place actors in virtual scenes, but actors cannot interact with objects in the virtual scene in real time. In other words, existing virtual filming solutions can achieve basic virtual filming functions, but none support real-time interaction between actors and the virtual scene. Therefore, how to achieve real-time interaction between physical objects and virtual scenes is one of the problems that this disclosure aims to solve.

[0027] Therefore, this disclosure determines the positional relationship between a physical object in front of the screen and the virtual scene displayed on the screen during virtual shooting. When this positional relationship meets the conditions for triggering a target interaction operation, the target interaction operation between the physical object and the virtual scene is triggered, and the virtual scene changes in response to the triggering of the target interaction operation. Thus, based on whether the positional relationship between the physical object (a real character, such as an actor) and the virtual scene meets the conditions for triggering the target interaction operation, the triggering of the target interaction operation can be controlled, and the virtual scene can be changed in response to the triggering of the target interaction operation. This enhances the interaction control logic between the physical object and the virtual scene, enabling real-time interaction between the physical object and the virtual scene, such as knocking away a wooden box in the virtual scene or leaving footprints on the beach, improving the presentation effect of the virtual shooting solution and expanding its application scenarios.

[0028] The following combination Figs. 1-3 This invention will elaborate on the virtual shooting method and apparatus for virtual-real interaction according to exemplary embodiments of the present disclosure.

[0029] Fig. 1 A flowchart illustrating a virtual-real interaction method for virtual shooting according to an exemplary embodiment of the present disclosure is shown. This virtual-real interaction method is executed by a virtual-real interaction system, such as... Fig. 1 As shown, this virtual-real interaction method includes:

[0030] Step S120: During the virtual shooting process, determine the positional relationship between the physical object in front of the screen and the virtual scene displayed on the screen.

[0031] Step S140: When the positional relationship satisfies the condition for triggering the target interaction operation, the target interaction operation between the entity object and the virtual scene is triggered, and the virtual scene changes in response to the triggering of the target interaction operation.

[0032] In this embodiment, during virtual shooting, there is a relative positional relationship between the physical object and the virtual scene, and there may be a need for various virtual-real interactions between the physical object and the virtual scene, such as knocking away a wooden box in the virtual scene or leaving footprints on the beach. The execution of various virtual-real interaction methods (also called target interaction operations) may be predicated on the positional relationship between the physical object and the virtual scene meeting certain conditions. That is, the target interaction operation corresponding to that condition is executed only if the condition is met. In other words, whether or not the target interaction operation corresponding to that condition is executed is controlled based on whether the condition is met.

[0033] Therefore, in order to achieve virtual-real interaction between physical objects and virtual scenes, the positional relationship between physical objects and virtual scenes can be determined; it can be determined whether the positional relationship meets the triggering condition of the target interaction operation; if it is determined that the positional relationship meets the triggering condition, the target interaction operation is triggered to change the virtual scene; if it is determined that the positional relationship does not meet the triggering condition, the virtual scene is maintained.

[0034] In one possible implementation, the positional relationship between the entity object and the virtual scene may include the positional relationship between the entity object and the virtual object (or model) in the virtual scene that serves as the object of operation, the positional relationship between the entity object and the screen displaying the virtual scene, or the positional relationship between the entity object and the target position in the virtual scene or the screen.

[0035] In one possible implementation, the triggering condition may include, but is not limited to, matching the positional relationship between the entity object and the virtual scene with the target interaction operation. For example, if the target interaction operation between the entity object and the virtual scene is the first operation that causes the model in the virtual scene to move following the entity object, then the triggering condition is that the distance between the entity object and the model in the virtual scene is less than a preset value. As another example, if the target interaction operation between the entity object and the virtual scene is the second operation that leaves footprints where the entity object has walked while walking through the virtual scene, then the triggering condition is that the distance between the entity object's feet and the ground in the virtual scene is less than a preset value. For instance, an actor walks close to the screen, and footprints appear on the beach displayed on the screen.

[0036] In one possible implementation, the target interaction operation includes a collision operation, which is triggered in response to a match between the positional relationship between the entity object and the virtual scene and the collision operation.

[0037] In this embodiment, for the collision operation, the positional relationship between the entity object and the virtual object to be collided with in the virtual scene (such as any stone, table, chair, or other collideable virtual object) can be determined, and it can be detected whether the positional relationship matches the collision operation (for example, the distance between the entity object and the virtual object to be collided with in the virtual scene is less than a preset value). If the positional relationship matches the collision operation, the collision operation is triggered.

[0038] For example, if the collision operation includes an entity object pushing an object in the virtual scene or an entity object knocking an object in the virtual scene (such as a boulder) away with its hand, the distance between the entity object's hand and the object in the virtual scene can be determined, and it can be detected whether the distance is less than a preset value. If the distance is less than the preset value, the collision operation is triggered.

[0039] In one possible implementation, the target interaction operation includes a movement operation, and accordingly, in response to a movement operation being detected as triggered based on a match between the positional relationship between the entity object and the virtual scene.

[0040] In this embodiment, for a movement operation, the positional relationship between the entity object and the virtual scene can be determined based on the positioning data, and it can be detected whether the positional relationship matches the movement operation (for example, the distance between the entity object and the virtual object being moved in the virtual scene is less than a preset value). If the positional relationship matches the movement operation, the movement operation is triggered.

[0041] For example, if the movement operation includes leaving footprints where the entity walks in the virtual scene, the distance between the entity's feet and the ground in the virtual scene can be determined, and it can be detected whether the distance is less than a preset value. If the distance is less than the preset value, the movement operation is triggered.

[0042] According to the virtual-real interaction method for virtual shooting in this embodiment, the positional relationship between the physical object and the virtual scene is determined. When the positional relationship meets the conditions for triggering a target interaction operation, the target interaction operation between the physical object and the virtual scene is triggered, and the virtual scene changes in response to the triggering of the target interaction operation. Therefore, whether to trigger the target interaction operation can be controlled based on whether the positional relationship between the physical object and the virtual scene meets the conditions for triggering the target interaction operation, and the virtual scene changes in response to the triggering of the target interaction operation. This enhances the interaction control logic between the physical object and the virtual scene, enabling real-time interaction between the real character and the virtual scene, such as knocking away a wooden box in the virtual scene or leaving footprints on the beach. This improves the presentation effect of the virtual shooting solution, making the virtual shooting effect more realistic, thus meeting the needs of more shooting scenarios and increasing the application scenarios of the virtual shooting solution.

[0043] In one possible implementation, step S120 may include: acquiring positioning data of positioning points installed on the entity object in real time; and determining the positional relationship between the entity object and the virtual scene based on the positioning data.

[0044] In this embodiment, corresponding positioning points can be installed on relevant parts of the physical object according to the virtual-real interaction required in virtual shooting. By adding positioning points to key parts of the physical object, the precise position and orientation of the key parts of the physical object can be obtained, thereby enabling better real-time interaction between the physical object and the virtual scene.

[0045] For example, if the target interaction operation between the entity object and the virtual scene is the first operation that makes the model in the virtual scene move with the entity object, then in order to perform this first operation, one or more positioning points can be installed on the back of the entity object to locate the overall position and orientation of the entity object.

[0046] For example, if the target interaction operation between a physical object and a virtual scene is a second operation that leaves footprints where the physical object walks in the virtual scene, then one or more positioning points can be installed on the actor's feet to perform this second operation.

[0047] For example, if the target interaction operation between the entity object and the virtual scene is a third operation in which the entity object pushes an object in the virtual scene, then one or more positioning points can be installed on the actor's hand in order to perform this third operation.

[0048] In one possible implementation, in order to obtain more accurate positioning data and thus perform more accurate interactive operations, a greater number of positioning points can be installed on the key parts of the entity object. That is, the higher the accuracy of the required positioning data, the denser the positioning points are placed on the key parts.

[0049] After installing the corresponding positioning points on the physical object, the positioning system can be debugged to ensure it functions correctly. A motion capture system, or simply motion capture system, can be used here. It's a system that captures changes in the motion state of an object, most commonly its position and orientation. Commonly used motion capture systems in virtual shooting solutions include OptiTrack and Mosys. The motion capture system can track the position data (positioning data) of each positioning point in real time. The virtual-real interaction system connects to the motion capture system and obtains (queries) the position data of the positioning points from it in real time. The position data of a positioning point includes its spatial location and orientation. It should be understood that if multiple positioning points are installed on the physical object for virtual shooting, the position data of each of these positioning points is obtained from the motion capture system in real time. The positioning points can be any form of device capable of sensing position data, such as tags or sensors.

[0050] After acquiring the location data of the positioning point in real time, it is equivalent to acquiring the location data of the physical object, which can be used to determine the positional relationship between the physical object and the virtual scene. For example, the positioning data can be used to determine whether the physical object is in contact with the ground in the virtual scene. Taking distance as a criterion, the positioning data can locate the position of the physical object in the real world, while the position of the virtual object in the virtual scene can be obtained from the data of the virtual scene. Combining the two, the distance between the physical object and the virtual object can be calculated. Optionally, for virtual objects in the virtual scene that are close to the screen position, the position of the physical object and the distance between it and the virtual object's displayed position on the screen can also be simply calculated to simplify the calculation process.

[0051] In one possible implementation, the above-mentioned virtual-real interaction method may further include: offset correction of the positioning data, and correspondingly, determining the positional relationship between the entity object and the virtual scene based on the positioning data includes: determining the positional relationship between the entity object and the virtual scene based on the offset-corrected positioning data.

[0052] In this embodiment, considering that there is a certain deviation between the positioning data obtained in real time from the motion capture system and the position data in the virtual scene, which may cause the virtual space and the real space to be misaligned, after obtaining the positioning data from the motion capture system, the positioning data is first offset corrected to eliminate the aforementioned deviation (i.e., the deviation between the virtual space and the real space), and then the positional relationship between the entity object and the virtual scene is determined based on the positioning data after eliminating the aforementioned deviation.

[0053] In one possible implementation, triggering a target interaction operation between an entity object and a virtual scene includes: generating an interaction control instruction corresponding to the target interaction operation and sending the interaction control instruction to a rendering engine, wherein the interaction control instruction is used to define the changes that the virtual scene should undergo in response to the triggering of the target interaction operation.

[0054] In this embodiment, once the target interactive operation is triggered, the virtual-real interaction system will determine whether the virtual scene needs to be changed in response to the trigger, such as changing the model in the virtual scene (e.g., moving the model to a new position) or special effects (e.g., triggering the animation effect of the model). After determining the changes that need to be made to the virtual scene, the virtual-real interaction system will abstract the changes into an interactive control command.

[0055] For example, when a collision operation is detected, the virtual-real interaction system determines that the model in the virtual scene needs to be moved to a new position in response to the collision operation. Then, the virtual-real interaction system abstracts this change of moving the model in the virtual scene to a collision interaction control command.

[0056] After generating the interactive control command, the virtual-real interaction system can send the interactive control command to the rendering engine, find the corresponding model through the plugin embedded in the rendering engine, and modify the model's position, orientation and other attributes.

[0057] According to the virtual-real interaction method of this embodiment, the positioning data of the positioning points added to the physical object is acquired in real time. Based on the positioning data, it is detected whether the target interaction operation corresponding to the positioning point is triggered. If the target interaction operation is triggered, an interaction control command corresponding to the target interaction operation is generated and sent to the rendering engine. This allows the rendering engine to modify the virtual scene in response to the target interaction operation based on the interaction control command. Therefore, by adding tracking points to key parts of the physical object, acquiring the real-time position (positioning data) of the key parts of the physical object, and adding interaction control logic, real-time interaction between the real character and the virtual scene can be achieved, such as knocking away a wooden box in the virtual scene or leaving footprints on the beach. This improves the presentation effect of the virtual shooting solution, making the virtual shooting effect more realistic, thus meeting the needs of more shooting scenarios and increasing the application scenarios of the virtual shooting solution.

[0058] In one possible implementation, generating the interactive control instruction corresponding to the target interactive operation includes: determining, based on the positioning data of the positioning point, modifications to the model (e.g., the model's position, orientation, and other attributes) and / or effects of the virtual scene in response to the triggering of the target interactive operation; and generating the interactive control instruction based on the modifications to the model and / or effects of the virtual scene to be made.

[0059] For example, if the target interactive operation is for an actor to knock away a boulder in a virtual scene with a single palm strike, the system can detect whether a collision has occurred between the location of the actor's hand and the boulder model based on real-time acquired positioning data. Once a collision is detected, the target interactive operation is triggered, and an interactive control command can be generated to inertially move the boulder along the direction of the positioning point. It should be understood that, in this example, the modifications to the properties of the virtual scene model in response to the triggering of the target interactive operation of the actor knocking away a boulder in the virtual scene include: causing the boulder to inertially move along the direction of the positioning point.

[0060] For example, if the target interaction is for an actor to leave footprints where they walk, the system can detect whether the location of the corresponding positioning point (e.g., the positioning point is close enough to the ground model) has collided with the real-time positioning data of the actor's feet. Once a collision is detected, the target interaction is triggered, and an interactive control command can be generated to display a footprint model at the collision location and adjust the orientation of the footprint model to match the orientation of the positioning point. It should be understood that in this example, the modifications to the properties of the virtual scene model in response to the triggering of the target interaction of the actor leaving footprints include: displaying a footprint model at the location where the positioning point (corresponding to the positioning data) collides with the ground model, and adjusting the orientation of the footprint model to match the orientation of the positioning point.

[0061] Therefore, the generated interactive control instructions can reflect the modifications that the virtual scene model and / or effects need to make in response to the triggering of the target interactive operation. The rendering engine renders the virtual scene according to such interactive control instructions, which can realize the target interactive operation between the entity object and the virtual scene, thereby achieving a more realistic virtual-real combination effect, such as knocking or pushing objects in the virtual scene, virtual character following, and special effects that follow hand and foot movements, etc.

[0062] In one possible implementation, the virtual-real interaction method may further include: in response to a user's editing operation on an editable interaction template, determining a target interaction template, wherein the target interaction template includes a positioning point corresponding to the target interaction operation and a change method of the virtual scene corresponding to the target interaction operation.

[0063] In one possible implementation, step S140 may include: when the positional relationship satisfies the condition for a target interactive operation, triggering an interactive script generated based on a target interactive template corresponding to the target interactive operation, and executing the interactive script to realize the target interactive operation.

[0064] Considering that users who perform virtual shooting may not understand or be very proficient in technology, in order to enable such users to quickly and easily achieve virtual-real interaction, the virtual-real interaction system abstracts a series of commonly used virtual-real interaction behaviors into general templates, defining a series of interaction templates for users to choose and edit.

[0065] Users can select a template suitable for their desired virtual-real interaction method (target interaction operation) and edit it. Editing can include, but is not limited to, selecting a location point, choosing the model or effect affected by that location point, and selecting the logic for changing the model and effect. Correspondingly, the virtual-real interaction system can obtain the logic for changing the model and effect selected by the user on the interaction template and generate an interaction control script. During filming, the system can respond to the user's editing of the interaction template to determine the target interaction template, which defines the location point corresponding to the target interaction operation and the way the virtual scene changes. This allows for real-time detection of the location data of the location point (reflecting changes in its position and direction), and determines whether the positional relationship between the entity object and the virtual scene meets the conditions for triggering the target interaction operation based on the current position and direction of the location point. If the conditions are met, the corresponding interaction script is triggered and executed, thus achieving the target interaction operation. Templates can also be pre-configured with the above items, allowing users to directly select a pre-configured template to achieve the target interaction operation.

[0066] For example, if the target interactive operation is for an actor to knock away a boulder in a virtual scene with a single palm strike, the virtual-real interaction system can determine the corresponding target interactive template in response to the user's editing operation on the interactive template, and generate the following logic script (interaction script): The system acquires the position and direction of the actor's hand positioning point in real time, uses this positioning point to perform collision detection with the boulder model, and, upon detecting a collision, moves the boulder along the direction of the positioning point using inertia. During filming, the virtual-real interaction system can acquire the positioning data of the actor's hand positioning point in real time. If a collision is detected between the hand and the boulder based on this positioning data, the aforementioned logic script is triggered, indicating that the target interactive operation of the actor knocking away a boulder in the virtual scene has been triggered. The logic script is then executed to make the boulder move along the direction of the positioning point using inertia, thereby achieving the target interactive operation of the actor knocking away a boulder in the virtual scene with a single palm strike.

[0067] For example, if the target interaction is for an actor to leave footprints where they have walked, the virtual-real interaction system can determine the corresponding target interaction template in response to the user's editing operation on the interaction template, and generate the following logic script: Real-time acquisition of the position and direction of the actor's foot positioning point; when this positioning point collides with the ground, the position is recorded; a footprint model is displayed at this position; and the direction of the footprint model is adjusted to match the orientation of the positioning point. During filming, the virtual-real interaction system can acquire the positioning data of the actor's foot positioning point in real time. If a collision between the foot and the ground is detected based on this positioning data, the aforementioned logic script is triggered, indicating that the target interaction of the actor leaving footprints where they have walked has been triggered. The logic script is executed to display a footprint model at the position where the positioning point collides with the ground, and the orientation of the footprint model is adjusted to match the orientation of the positioning point, thereby realizing the target interaction of leaving footprints where they have walked.

[0068] Therefore, a series of commonly used virtual-real interaction behaviors are abstracted into universal templates, enabling users without technical knowledge to quickly design and implement virtual-real interaction operations required for shooting.

[0069] Accordingly, in one possible implementation, executing the interaction script to achieve the target interactive operation may include: determining model data and / or special effects data in the next frame of the virtual scene according to the interaction script; generating interactive control instructions according to the model data and / or special effects data, and sending the interactive control instructions to the rendering engine.

[0070] During filming, the virtual-real interaction system collects changes in all positioning points and detects whether interaction scripts related to those points are triggered. Once an interaction script is triggered, the virtual interaction system controls the model in the virtual scene according to the preset script logic, such as moving the model to a new position or triggering an animation effect on the model. After determining the changes that need to be made to the model or special effects in the virtual scene, this change is abstracted into a control command (interaction control command). This interaction control command can be sent to the rendering engine, which uses plugins embedded in the rendering engine to find the corresponding model and modify its position, orientation, and other attributes. In this way, the target interaction operation between the physical object and the virtual scene can be achieved.

[0071] Fig. 2 A system architecture diagram illustrating a virtual-real interaction method for virtual shooting according to exemplary embodiments of the present disclosure is shown. Fig. 2 As shown, the system mainly includes: camera 301, motion capture system 302, virtual-real interaction control system 303, rendering engine 304, and LED screen 305.

[0072] Based on existing virtual shooting solutions, this embodiment adds a virtual-real interaction system, namely a virtual-real interaction control system 303. The motion capture system 302 (e.g., OptiTrack) tracks the camera 301 and tracks the positioning data of positioning points 1 to N. The virtual-real interaction control system 303 obtains the positioning data of positioning points 1 to N from the motion capture system 302 and executes the virtual-real interaction method of this embodiment. The rendering engine 304 receives the model and special effects control instructions generated by the virtual-real interaction control system 303 and uses the model and special effects control instructions to adjust the model and special effects. The LED screen 305 displays a real-time rendered image, wherein the rendering engine 304 obtains the real-time rendered image after performing processing including rendering. The camera 301 shoots to capture the image of the actor and the real-time rendered image displayed on the LED screen, and stores the captured image in the memory card.

[0073] For example, the virtual-real interaction control system 303 uses the motion capture system 302 to acquire the positioning data of positioning points 1 to N on the physical object in real time. The positioning data includes position and direction. Then, it associates each positioning data with a pre-edited interaction script. The interaction script defines how the model and special effects in the virtual scene respond to changes in the positioning data. The virtual-real interaction control system 303 calculates the state of the model and special effects in the virtual scene in the next frame according to the logic designed in the interaction script. Then, it abstracts the changes in the model and special effects into a control command and sends it to the rendering engine 304. After receiving the control command, the rendering engine 304 adjusts the model and special effects, such as changing the position of the model, and then re-renders the virtual scene and displays it on the LED screen.

[0074] In one possible implementation, the virtual-real interaction control system 303 may include a positioning data collection module, a virtual-real interaction editor, and a virtual scene control module. The positioning data collection module is used to execute step S120, the virtual-real interaction editor is used to generate the interaction script used in step S140, and the virtual scene control module is used to generate the interaction control instructions in step S160.

[0075] It should be understood that the above systems or modules can be set up separately or integrated together. For example, the virtual-real interaction control system 303 and the motion capture system 302 can be set up on different devices or on the same device.

[0076] Fig. 3 A block diagram of a virtual-real interaction device for virtual shooting according to an exemplary embodiment of the present disclosure is shown. Fig. 3As shown, the virtual-real interaction device 300 may include a determining unit 310 and a triggering unit 320. The determining unit 310 is used to determine the positional relationship between a physical object in front of the screen and a virtual scene displayed on the screen during virtual shooting. The triggering unit 320 is connected to the determining unit 310 and is used to trigger a target interaction operation between the physical object and the virtual scene when the positional relationship meets the conditions for triggering a target interaction operation. The virtual scene changes in response to the triggering of the target interaction operation.

[0077] In one possible implementation, the determining unit 310 is configured to: acquire positioning data of positioning points installed on the entity object in real time; and determine the positional relationship between the entity object and the virtual scene based on the positioning data.

[0078] In one possible implementation, the triggering unit 320 is configured to: generate an interaction control instruction corresponding to the target interaction operation and send the interaction control instruction to the rendering engine, wherein the interaction control instruction is used to define the changes to be made in response to the triggering of the target interaction operation in the virtual scene.

[0079] In one possible implementation, the triggering unit 320 is configured to: determine, based on the positioning data of the positioning point, modifications to the model and / or effects of the virtual scene in response to the triggering of the target interactive operation; and generate the interactive control command based on the modifications to the model and / or effects of the virtual scene to be made.

[0080] In one possible implementation, the determining unit 310 is further configured to: determine a target interaction template in response to an editing operation performed by a user on an editable interaction template, wherein the target interaction template includes a positioning point corresponding to the target interaction operation and a change method of the virtual scene corresponding to the target interaction operation.

[0081] In one possible implementation, the triggering unit 320 is configured to: when the positional relationship satisfies the condition for a target interactive operation, trigger an interactive script generated according to a target interactive template corresponding to the target interactive operation, and execute the interactive script to realize the target interactive operation.

[0082] In one possible implementation, the triggering unit 320 is configured to: determine model data and / or special effects data in the next frame of the virtual scene according to the interaction script; generate interactive control instructions according to the model data and / or special effects data, and send the interactive control instructions to the rendering engine.

[0083] In one possible implementation, the virtual-real interaction device 300 may further include: a correction unit (not shown) for offset correction of the positioning data, and correspondingly, the determination unit 310 is configured to: determine the positional relationship between the entity object and the virtual scene based on the offset-corrected positioning data.

[0084] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0085] This disclosure also proposes a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the aforementioned virtual-real interaction method. The computer-readable storage medium can be volatile or non-volatile.

[0086] This disclosure also proposes a virtual-real interaction device, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-described virtual-real interaction method when executing the instructions stored in the memory.

[0087] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A virtual-real interaction method for virtual shooting, characterized in that, include: During virtual shooting, the positioning data of the positioning points installed on the physical object are acquired in real time. The positional relationship between the physical object in front of the screen and the virtual scene displayed on the screen is determined based on the positioning data. The positional relationship includes distance, and the positioning data includes the spatial position and direction of the positioning point. When the positional relationship satisfies the conditions for triggering the target interaction operation, the target interaction operation between the entity object and the virtual scene is triggered; The virtual scene changes in response to the triggering of the target interactive operation. The positional relationships include the positional relationship between the positioning point and a virtual object in the virtual scene that is the object of operation, the positional relationship between the positioning point and the screen displaying the virtual scene, or the positional relationship between the positioning point and a target position in the virtual scene or on the screen. For a virtual object in the virtual scene that is close to the screen, the positional relationship between the positioning point and the virtual object includes the distance between the spatial position of the positioning point and the display position of the virtual object on the screen. Specifically, corresponding positioning points are installed on the parts of the physical object corresponding to the target interactive operation required in the virtual shooting. During the virtual shooting process, the positioning data of the positioning points corresponding to the target interactive operation, which are acquired in real time, are used to detect whether the conditions for triggering the target interactive operation are met. The triggering of the target interaction operation between the entity object and the virtual scene includes: determining the modifications to the model and / or effects of the virtual scene in response to the triggering of the target interaction operation, based on the spatial position and direction of the positioning point. Based on the modifications to the model and / or effects of the virtual scene to be made, an interaction control instruction corresponding to the target interaction operation is generated and sent to the rendering engine, wherein the interaction control instruction is used to define the changes to be made in the virtual scene in response to the triggering of the target interaction operation.

2. The virtual-real interaction method according to claim 1, characterized in that, The virtual-real interaction method also includes: In response to user editing actions on the editable interactive template, determine the target interactive template. The target interaction template includes the positioning point corresponding to the target interaction operation and the way the virtual scene corresponding to the target interaction operation is changed.

3. The virtual-real interaction method according to claim 2, characterized in that, When the positional relationship satisfies the conditions for a target interaction operation, a target interaction operation between the entity object and the virtual scene is triggered, including: When the positional relationship satisfies the conditions for triggering the target interactive operation, the interactive script generated based on the target interactive template corresponding to the target interactive operation is triggered; The target interactive operation is achieved by executing the interactive script.

4. The virtual-real interaction method according to claim 3, characterized in that, The target interactive operation is achieved by executing the interactive script, including: The model data and / or special effects data in the next frame of the virtual scene are determined according to the interaction script; Interactive control commands are generated based on the model data and / or special effects data, and then sent to the rendering engine.

5. The virtual-real interaction method according to claim 1, characterized in that, Also includes: Offset correction is performed on the positioning data. Accordingly, determining the positional relationship between the entity object and the virtual scene based on the positioning data includes: determining the positional relationship between the entity object and the virtual scene based on the offset-corrected positioning data.

6. A virtual-real interactive device for virtual shooting, characterized in that, include: The determining unit is used to acquire positioning data of positioning points installed on a physical object in real time during virtual shooting, and determine the positional relationship between the physical object in front of the screen and the virtual scene displayed on the screen based on the positioning data. The positional relationship includes distance, and the positioning data includes the spatial position and direction of the positioning points. The triggering unit is used to trigger a target interaction operation between the entity object and the virtual scene when the positional relationship satisfies the condition for triggering the target interaction operation; The virtual scene changes in response to the triggering of the target interactive operation. The positional relationships include the positional relationship between the positioning point of the entity object and the virtual object in the virtual scene that serves as the operation object, the positional relationship between the positioning point of the entity object and the screen displaying the virtual scene, or the positional relationship between the positioning point of the entity object and a target position in the virtual scene or the screen. For a virtual object in the virtual scene located near the screen, the positional relationship between the positioning point and the virtual object includes the distance between the spatial position of the positioning point and the display position of the virtual object on the screen. Specifically, corresponding positioning points are installed on the parts of the physical object corresponding to the target interactive operation required in the virtual shooting. During the virtual shooting process, the positioning data of the positioning points corresponding to the target interactive operation, which are acquired in real time, are used to detect whether the conditions for triggering the target interactive operation are met. The triggering of the target interaction operation between the entity object and the virtual scene includes: determining the modifications to the model and / or effects of the virtual scene in response to the triggering of the target interaction operation, based on the spatial position and direction of the positioning point. Based on the modifications to the model and / or effects of the virtual scene to be made, an interaction control instruction corresponding to the target interaction operation is generated and sent to the rendering engine, wherein the interaction control instruction is used to define the changes to be made in the virtual scene in response to the triggering of the target interaction operation.

7. A virtual-real interactive device for virtual shooting, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the virtual-real interaction method according to any one of claims 1 to 5 when executing instructions stored in the memory.

8. A non-volatile computer-readable storage medium storing computer program instructions thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the virtual-real interaction method according to any one of claims 1 to 5.

9. A computer program product comprising computer program instructions, characterized in that, When the computer program instructions are executed by the processor, they implement the virtual-real interaction method according to any one of claims 1 to 5.

Images