Virtual shooting method and related device

Abstract

The invention discloses a virtual shooting method and a related device, the virtual shooting method is applied to a virtual shooting system, and the virtual shooting system comprises shooting equipment and a virtual shooting control device. The method comprises: based on a preset virtual shooting template, obtaining a current expected pose and a current expected virtual picture of a shooting device corresponding to a current shooting time, the virtual shooting template representing a corresponding relationship between poses at different shooting times in a motion trajectory of the shooting device and a virtual picture in a virtual scene; controlling the shooting equipment to be adjusted from the previous pose to the current expected pose, and acquiring a current shooting picture of the shooting equipment at the current expected pose; and fusing the current shot picture and the current expected virtual picture to obtain a current virtual shot picture. According to the scheme, the virtual shooting cost can be reduced, and diversified virtual shooting requirements can be flexibly met.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a virtual shooting method, a virtual shooting control device, a virtual shooting system, an electronic device and a computer-readable storage medium. Background Art

[0002] Virtual shooting technology is widely used in film and television production, TV series shooting, advertising production, game development, live broadcast and other fields.

[0003] Virtual shooting is a shooting method that combines the shooting object in the digital virtual scene with the shooting object in the physical real scene. Specifically, the real shooting picture is obtained by shooting the shooting object in the physical real scene with the shooting device, and the real shooting picture is merged with the corresponding virtual picture in the digital virtual scene to obtain a virtual shooting picture with the virtual picture as the background and the real shooting picture as the foreground.

[0004] At present, the demand for virtual shooting tends to be diversified, that is, virtual shooting needs to be performed in various physical real scenes. However, the existing virtual shooting methods are costly and cannot flexibly adapt to the diverse virtual shooting needs. Summary of the invention

[0005] The present application provides a virtual shooting method, a virtual shooting control device, a virtual shooting system, an electronic device and a computer-readable storage medium, which can solve the problems that the existing virtual shooting methods are high in cost and cannot flexibly adapt to diverse virtual shooting needs.

[0006] On the one hand, the present application provides a virtual shooting method, which is applied to a virtual shooting system. The virtual shooting system includes a shooting device and a virtual shooting control device. The method includes: based on a preset virtual shooting template, obtaining a current expected posture of the shooting device and a current expected virtual picture corresponding to the current shooting time, the virtual shooting template representing the correspondence between the posture of the shooting device at different shooting times in the motion trajectory of the shooting device and the virtual picture in the virtual scene; controlling the shooting device to adjust from the previous posture to the current expected posture, and obtaining the current shooting picture of the shooting device in the current expected posture; fusing the current shooting picture with the current expected virtual picture to obtain the current virtual shooting picture.

[0007] On the other hand, the present application provides a virtual shooting method, which is applied to a virtual shooting system, wherein the virtual shooting system includes a shooting device, a virtual shooting control device and a tracking device, and the method includes: receiving at least one detection posture sent by the tracking device, wherein the at least one detection posture is the posture of the shooting device detected by the tracking device at different shooting times; establishing a correspondence between the at least one detection posture and different virtual images in the virtual scene to synthesize a virtual shooting template of the virtual scene, wherein the virtual shooting template is used to instruct the shooting device to subsequently shoot according to the corresponding detection posture at different shooting times, and to fuse the captured shooting images with the corresponding virtual images to obtain a virtual shooting image.

[0008] On the other hand, the present application provides a virtual shooting control device, which includes: an acquisition module, a control module and a fusion module. The acquisition module is used to acquire the current expected posture of the shooting device and the current expected virtual picture corresponding to the current shooting time based on a preset virtual shooting template, and the virtual shooting template represents the corresponding relationship between the posture of the shooting device at different shooting times in the motion trajectory and the virtual picture in the virtual scene. The control module is used to control the shooting device to adjust from the previous posture to the current expected posture, and acquire the current shooting picture of the shooting device in the current expected posture. The fusion module is used to fuse the current shooting picture with the current expected virtual picture to obtain the current virtual shooting picture.

[0009] On the other hand, the present application provides a virtual shooting control device, which includes: a receiving module and a corresponding module. The receiving module is used to receive at least one detection posture sent by the tracking device, and the at least one detection posture is the posture of the shooting device detected by the tracking device at different shooting times. The corresponding module is used to establish a correspondence between the at least one detection posture and different virtual images in the virtual scene to synthesize a virtual shooting template of the virtual scene, wherein the virtual shooting template is used to instruct the shooting device to shoot according to the corresponding detection posture at different shooting times, and fuse the captured shooting picture with the corresponding virtual picture to obtain a virtual shooting picture.

[0010] On the other hand, the present application provides a virtual shooting system, which includes a shooting device and a virtual shooting control device; the shooting device is used to shoot pictures under the control of the virtual shooting control device; the virtual shooting control device is pre-configured with a virtual shooting template for executing the aforementioned virtual shooting method.

[0011] In another aspect, the present application provides an electronic device, including a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the above-mentioned virtual shooting method.

[0012] In another aspect, the present application provides a computer-readable storage medium having program instructions stored thereon, and the program instructions implement the above-mentioned virtual shooting method when executed by a processor.

[0013] In the above scheme, the present application presets a virtual shooting template before virtual shooting, and the preset virtual shooting template represents the corresponding relationship between the posture of the shooting device at different shooting times in the motion trajectory and the virtual picture in the virtual scene. During the virtual shooting process, the virtual shooting template is reused to obtain the current expected posture and the current expected virtual picture corresponding to the current shooting time, the shooting device is controlled to adjust from the previous posture to the current expected posture, and the current shooting picture of the shooting device in the current expected posture is obtained, and the current expected shooting picture and the current shooting picture are merged to obtain the current virtual shooting picture, thereby realizing virtual shooting. Therefore, the virtual shooting method provided by the present application is to obtain the posture (motion trajectory) of the shooting device and the corresponding virtual picture (virtual scene) by reusing the preset virtual shooting template, without obtaining the posture of the shooting device through the tracking device, and determining the corresponding virtual picture through the posture obtained by the tracking device. Thereby, the cost of virtual shooting can be reduced and the diversified virtual shooting needs can be flexibly adapted.

[0014] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0016] Figure 1 is a structural schematic diagram of an embodiment of a virtual shooting system of the first form of the present application;

[0017] Figure 2 is a structural schematic diagram of an embodiment of a virtual shooting system according to a second aspect of the present application;

[0018] Figure 3 It is a flowchart of an embodiment of a virtual shooting method provided by the present application;

[0019] Figure 4 is a flowchart of another embodiment of the virtual shooting method provided by the present application;

[0020] Figure 5 is a schematic diagram of a specific example of a virtual shooting template system according to the first aspect of the present application;

[0021] Figure 6 is a schematic diagram of a specific example of a virtual shooting template system according to the second aspect of the present application;

[0022] Figure 7 is a structural schematic diagram of an embodiment of a virtual shooting control device provided by the present application;

[0023] Figure 8 It is a structural schematic diagram of another embodiment of the virtual shooting control device provided by the present application.

[0024] Fig. 9 It is a structural schematic diagram of an embodiment of the electronic device of the present application;

[0025] Fig.10 It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0026] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0027] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0028] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the previously associated objects are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B and C.

[0029] The result of virtual shooting can be a single-frame virtual shooting picture, multiple-frame virtual shooting pictures, or a video composed of multiple consecutive frames of virtual shooting pictures. In order to simplify the description, the following text of this application only takes the acquisition of a single-frame virtual picture as an example for explanation.

[0030] In the related art, the virtual shooting method includes: obtaining the current shooting picture of the shooting device, and obtaining the current posture of the shooting device detected by the tracking device; determining the current virtual picture corresponding to the current posture from the virtual scene; and fusing the current shooting picture with the current virtual picture to obtain the current virtual shooting picture. Therefore, the related virtual shooting method needs to obtain the posture of the shooting device through the tracking device to determine the virtual picture corresponding to the posture.

[0031] The inventor of the present application has found through long-term research that the related virtual shooting method has at least the following technical problems:

[0032] 1. The cost of virtual shooting is high. Specifically, the tracking device is expensive, so the cost of obtaining the position of the shooting device through the tracking device is high, and thus the cost of virtual shooting is high.

[0033] 2. Unable to adapt to the diverse needs of virtual shooting. Specifically, virtual shooting needs to be carried out in a variety of virtual shooting scenes (real scenes). The tracking device is large in size, and some virtual shooting scenes such as exhibition halls and exhibitions do not support the deployment of tracking devices, resulting in the inability of virtual shooting to proceed normally. Therefore, the relevant virtual shooting method cannot adapt to the diverse needs of virtual shooting.

[0034] In order to solve the above technical problems, the present application provides a virtual shooting method, which does not require tracking equipment, but implements virtual shooting based on a preset virtual shooting template, which can reduce the cost of virtual shooting and flexibly adapt to diverse virtual shooting needs. The details are as follows:

[0035] The virtual shooting method provided in the present application can be implemented based on a virtual shooting system. On the one hand, the present application provides a first form of virtual shooting system, which is used to perform virtual shooting in a second real scene to obtain a virtual shooting template. On the other hand, a second form of virtual shooting system is provided, and the second form of virtual shooting system is pre-configured with a virtual shooting template, which is used to perform virtual shooting in a first real scene to obtain a virtual shooting picture. The virtual shooting template pre-configured by the second form of virtual shooting system is obtained based on the first form of virtual shooting system.

[0036] Figure 1 FIG. 1 is a schematic diagram of the structure of an embodiment of a virtual shooting system according to the first aspect of the present application. Figure 1 As shown, the first form of the virtual shooting system 10 includes a virtual shooting control device 11, a shooting device 12, a display screen 13 and a tracking device 14. The virtual shooting control device 11 establishes communication connections with the shooting device 12, the display screen 13 and the tracking device 14 respectively.

[0037] The shooting device 12 includes a shooting body (not shown) and a moving carrier (not shown). The shooting body is a video / image shooting source. The shooting body can be a movie camera, a TV camera, a SLR camera, a smart phone, a webcam, etc. The shooting body is physically connected to the moving carrier. The movement of the moving carrier can drive the shooting body to move, so that the shooting body can shoot a picture of the shooting object of the second real scene. The moving carrier can be a mobile device including a tripod, a pan head, and a rocker arm, and can also be a track robot, a track manipulator arm, etc.

[0038] The display screen 13 is arranged on the stage of the second real scene, and can be used to display the virtual picture sent by the virtual shooting control device 11, so as to provide an immersive shooting experience for the users of the virtual shooting (such as the director, the crew, and the actors). The display screen 13 can be an LED display screen, an LCD display screen, a projection screen, a transparent screen, a curved display screen, a ring display screen, a mobile display screen, etc. In the case where the stage of the second real scene does not need to display the virtual picture, the first form of the virtual shooting system 10 may not include the display screen 13.

[0039] The tracking device 14 can be used to detect the position and posture of the shooting device 12, and the detected position and posture is used to generate a virtual shooting template. The tracking device 14 can be an optical tracking device, an inertial tracking device, a hybrid tracking device, etc. The optical tracking device uses a laser to detect the position and posture of the shooting device 12. The inertial tracking device uses sensors such as accelerometers, gyroscopes, and magnetometers to detect the position and posture of the shooting device 12. The hybrid tracking device combines the characteristics of the optical tracking device and the inertial tracking device to detect the position and posture of the shooting device 12. In some specific examples, the tracking device 14 can be Mosys, RedSpy, etc.

[0040] The virtual shooting control device 11 can be a control device with virtual shooting capabilities such as a virtual shooting server, a computer, a mobile phone, a switch, and a control machine. The virtual shooting control device 11 can be used to receive the shooting picture of the shooting device 12, receive the posture detected by the tracking device 14, establish a corresponding relationship between the shooting picture and the virtual picture, merge the shooting picture and the virtual picture into a virtual shooting picture, send the virtual shooting picture to the display screen 13 for display, etc., so as to execute the virtual shooting method in the second real scene to obtain a virtual shooting template. The second real scene supports the deployment of the tracking device 14.

[0041] Figure 2 FIG. 1 is a schematic diagram of the structure of an embodiment of a virtual shooting system according to the second aspect of the present application. Figure 2 As shown, the second form of the virtual shooting system 20 includes a virtual shooting control device 21, a shooting device 22, and a display screen 23. The virtual shooting control device 21 establishes communication connections with the shooting device 22 and the display screen 23 respectively.

[0042] Different from the virtual shooting system 10 of the first form mentioned above, the virtual shooting system 20 of the second form does not include a tracking device. The virtual shooting control device 21 is pre-configured with a virtual shooting template, which is used to execute a virtual shooting method based on the virtual shooting template in the first real scene to obtain a virtual shooting picture. The first real scene can be any virtual shooting scene, and the first real scene is the same as or different from the second real scene. Regarding the virtual shooting method, please refer to the following embodiments, which will not be repeated here. The shooting device 22 can be used to move under the control of the virtual shooting control device 11 to shoot the picture.

[0043] Other detailed descriptions about the second form of the virtual shooting system 20 are similar to those of the first form of the virtual shooting system 10 and are not repeated here.

[0044] It is understandable that the virtual shooting control device 21 and the virtual shooting control device 11 may be the same or different. The shooting device 12 and the shooting device 22 may be the same or different. The display screen 13 and the display screen 23 may be the same or different. Under the same circumstances, the second form of the virtual shooting system 10 and the second form of the shooting system 20 may be converted to each other. Specifically, the first form of the virtual shooting system 10 may be deployed in the second real scene to obtain a virtual shooting template. After obtaining the virtual shooting template, the tracking device 14 in the first form of the virtual shooting system 10 may be removed and converted to the second form of the virtual shooting system 20. The second form of the virtual shooting system 20 may be deployed in the first real scene to realize the virtual shooting of the first real scene and obtain a virtual shooting picture.

[0045] Figure 3 1 is a flow chart of an embodiment of a virtual shooting method provided by the present application. Figure 3 As shown, in this embodiment, the virtual shooting method may include the following steps:

[0046] S31: Based on a preset virtual shooting template, obtain a current expected posture of the shooting device and a current expected virtual picture corresponding to the current shooting time.

[0047] The execution subject of this embodiment is a virtual shooting control device.

[0048] The virtual shooting template represents the corresponding relationship between the postures of the shooting device at different shooting times in the motion trajectory and the virtual images in the virtual scene.

[0049] The shooting time is relative to the overall shooting time range of the first real scene. The overall shooting time range of the first real scene includes multiple shooting times, and each shooting time has a sequence relative to the overall shooting time range.

[0050] The motion trajectory of the shooting device is the motion trajectory of the shooting device when it is applied to the virtual shooting of the first real scene. The motion trajectory of the shooting device includes several postures at shooting time. The virtual scene is a digital scene obtained according to 3D modeling technology, including several virtual pictures. In a specific example, the virtual scene can be obtained by 3D modeling through Unreal Engine. The posture at a shooting time in the motion trajectory corresponds to a virtual picture in the virtual scene. The virtual picture corresponding to the posture is the field of view of the shooting device in the virtual scene when it is in the posture.

[0051] The virtual shooting template is obtained by the first form of the virtual shooting system in the second real scene supporting the deployment of the tracking device. In addition, the virtual shooting template can be obtained in the process of executing other virtual shooting pictures, or can be obtained by executing a special virtual shooting template recording task.

[0052] In some embodiments, the virtual shooting template is a mapping table of motion trajectories and virtual scenes. The mapping table records the virtual images corresponding to the postures at each shooting time. For example, the motion trajectory includes postures B1-B5 at shooting times A1-A5, and the virtual scene includes corresponding virtual images C1-C5. The mapping table records the correspondence between A1-A5, B1-B5, and C1-C5. By looking up the mapping table, it can be determined that A1 corresponds to B1 and C1, A2 corresponds to B2 and C2, A3 corresponds to B3 and C3, A4 corresponds to B4 and C4, and A5 corresponds to B5 and C5.

[0053] In this case, S31 includes S311 (not shown). S311: searching the current expected posture and the current expected virtual picture corresponding to the current shooting time from the mapping table of the motion trajectory and the virtual scene.

[0054] In some embodiments, the virtual shooting template includes virtual posture fusion pictures at each shooting time. The virtual posture fusion pictures at the shooting time represent the corresponding relationship between the posture at the shooting time and the virtual picture, and are obtained by fusing the posture at the shooting time and the corresponding virtual picture.

[0055] In this case, S31 includes S312-S313 (not shown). S312: Acquire the virtual posture fusion picture of the current shooting time from the virtual shooting template. S313: Acquire the current expected posture and the current expected virtual picture from the virtual posture fusion picture of the current shooting time.

[0056] It can be understood that by fusing the posture at the time of shooting and the corresponding virtual picture, the posture can be hidden in the virtual picture to obtain a virtual posture fusion picture. Compared with the mapping table, it is more secure. The fusion of the posture at the time of shooting and the corresponding virtual picture can be implemented based on the least significant bit algorithm (LSB), the algorithm based on matrix coding, the algorithm based on the transform domain, the algorithm based on the statistical characteristics of the image, the algorithm based on machine learning, the algorithm based on chaos theory, the algorithm based on compressed sensing, the algorithm based on visual psychology, etc. Taking the least significant bit algorithm and the algorithm based on matrix coding as examples, the least significant bit algorithm is to write the posture as secret information by modifying the pixel value of the least significant bit of the virtual picture. The algorithm based on matrix coding is to write the posture as secret information into the virtual picture through matrix operation.

[0057] In S313, the method of obtaining the current desired posture and the current desired virtual picture from the virtual posture fusion picture corresponds to the fusion method of the posture and the virtual picture.

[0058] In some embodiments, the virtual pose fusion picture at the shooting time is obtained by fusing the pose at the shooting time and the corresponding virtual picture through the least significant bit algorithm. S313 includes: obtaining the current expected pose and the current expected virtual picture from the virtual pose fusion picture at the current shooting time through the least significant bit algorithm.

[0059] S32: Control the shooting device to adjust from the previous posture to the current expected posture, and obtain the current shooting picture of the shooting device in the current expected posture.

[0060] The previous pose may be the pose at the previous shooting time or the initial pose.

[0061] The current shooting picture includes a shooting object, which may be a person, an animal, a static object, etc. The current shooting picture is obtained by shooting the shooting object in the first real scene at the current expected posture by the shooting device.

[0062] The first real scene is a stage scene of a virtual shooting scene. The stage scene may be a green screen scene, a blue screen scene, or a virtual scene displayed on a display screen arranged on the stage.

[0063] In some embodiments, a control signal carrying the current desired posture may be sent to the photographing device. The photographing device receives the control signal and responds to the control signal to adjust from the previous posture to the current desired posture.

[0064] S33: Merge the current shooting picture with the current expected virtual picture to obtain the current virtual shooting picture.

[0065] The current virtual shooting picture is a picture with the shooting object in the current shooting picture as the foreground and the current desired virtual picture as the background.

[0066] In some embodiments, the current shooting picture may be segmented to obtain foreground pixels corresponding to the shooting object; and the pixels at corresponding positions in the current virtual shooting picture may be replaced by the foreground pixels to obtain the current virtual shooting picture.

[0067] Through the embodiments of S31-S33, the present application presets a virtual shooting template before virtual shooting, and the preset virtual shooting template represents the corresponding relationship between the posture of the shooting device at different shooting times in the motion trajectory and the virtual picture in the virtual scene. During the virtual shooting process, the virtual shooting template is reused to obtain the current expected posture and the current expected virtual picture corresponding to the current shooting time, the shooting device is controlled to adjust from the previous posture to the current expected posture, and the current shooting picture of the shooting device in the current expected posture is obtained, and the current expected shooting picture and the current shooting picture are merged to obtain the current virtual shooting picture, thereby realizing virtual shooting. Therefore, the virtual shooting method provided by the present application is to obtain the posture (motion trajectory) of the shooting device and the corresponding virtual picture (virtual scene) by reusing the preset virtual shooting template, without obtaining the posture of the shooting device through the tracking device, and determining the corresponding virtual picture through the posture obtained by the tracking device. Thereby, the cost of virtual shooting can be reduced and the diversified virtual shooting needs can be flexibly adapted.

[0068] In some embodiments, the first real scene is a virtual scene displayed on a display screen arranged on the stage of the virtual shooting scene. In this case, after S31, it also includes: sending the current expected virtual picture to the display screen, and controlling the display screen to display the current expected virtual picture on the first real scene. That is, the display screen arranged on the stage of the first real scene can control the display screen to display the current expected virtual picture on the stage of the first real scene, so as to provide an immersive shooting experience for users of virtual shooting (such as directors, crews, actors).

[0069] In some embodiments, S41-S42 (not shown) is also included before S31. S41: Receive at least one detection posture sent by the tracking device, and at least one detection posture is the posture of the shooting device detected by the tracking device at different shooting times. S42: Establish a correspondence between at least one detection posture and different virtual pictures in the virtual scene, so as to synthesize a virtual shooting template of the virtual scene. Among them, the shooting device is arranged in the first real scene during the virtual shooting picture generation process, and is arranged in the second real scene during the tracking device detection process. The first real scene and the second real scene are the same or different. For other detailed descriptions of S41-S42, please refer to the relevant descriptions of other embodiments, which will not be repeated here. Through S41-S42, a virtual shooting template can be obtained before virtual shooting. The virtual shooting template can be used to implement virtual shooting.

[0070] It can be understood that in S41-S42, the shooting device is arranged in the second real scene to obtain a virtual shooting template. In S31-S32, the shooting device is arranged in the first real scene to realize virtual shooting of the first real scene. That is to say, in the second real scene that supports the arrangement of the tracking device, S41-S42 can be implemented based on the virtual shooting system of the first form to obtain the virtual shooting template. After obtaining the virtual shooting template, in any second real scene, the tracking device in the virtual shooting template of the first form can be removed to obtain the virtual shooting system of the second form, and S31-S32 can be implemented based on the virtual shooting system of the second form to realize virtual shooting.

[0071] Figure 4 1 is a flow chart of another embodiment of the virtual shooting method provided by the present application. The virtual shooting method provided by the present embodiment is used to obtain a virtual shooting template. The virtual shooting method provided by the present embodiment can be applied to the first form of the virtual shooting system. Figure 4 As shown, in this embodiment, the virtual shooting method may include the following steps:

[0072] S51: Receive at least one detected pose sent by a tracking device.

[0073] At least one detected posture is a posture of the shooting device detected by the tracking device at different shooting times.

[0074] The execution subject of this embodiment is a virtual shooting control device.

[0075] The shooting time is relative to the overall shooting time range of the second real scene. The overall shooting time range of the second real scene is consistent with the overall shooting time range of the first real scene. That is, the duration of the overall shooting time range is consistent, and the relative relationship between each shooting time included and the overall shooting time range is consistent. A detected posture is the posture of the shooting device when shooting the shooting object of the second real scene at a shooting time.

[0076] S52: Establishing a correspondence between at least one detected posture and different virtual images in the virtual scene to synthesize a virtual shooting template of the virtual scene.

[0077] Among them, the virtual shooting template is used to instruct the shooting device to shoot according to the corresponding detection posture at different shooting times, and merge the captured shooting picture with the corresponding virtual picture to obtain a virtual shooting picture.

[0078] In some embodiments, the virtual shooting template is a mapping table of the detected postures and virtual images at each shooting time. In this case, S52 includes S521: for each detected posture, storing the corresponding relationship between the detected posture and a virtual image in the mapping table.

[0079] In some embodiments, the virtual shooting template includes a virtual posture fusion picture at each shooting time, and the virtual posture fusion picture represents the corresponding relationship between the detection posture and the virtual picture. In this case, S52 includes S522. S522: For each detection posture, fuse the detection posture with a virtual picture to obtain a virtual posture fusion picture.

[0080] It can be understood that by fusing the posture at the time of shooting with the corresponding virtual picture, the posture can be hidden in the virtual picture to obtain a virtual posture fusion picture. Compared with the mapping table, it is more secure.

[0081] The fusion of the posture at the time of shooting and the corresponding virtual picture can be achieved based on the least significant bit algorithm (LeastSignificant Bit, LSB), the algorithm based on matrix coding, the algorithm based on transform domain, the algorithm based on image statistical characteristics, the algorithm based on machine learning, the algorithm based on chaos theory, the algorithm based on compressed sensing, the algorithm based on visual psychology, etc.

[0082] In some embodiments, the virtual posture fusion picture is obtained by fusion using a least significant bit algorithm. In this case, S522 includes: writing the detected posture into the virtual picture using a least significant bit algorithm to obtain the virtual posture fusion picture.

[0083] Through the embodiments of S51-S52, the present application can receive at least one detected posture sent by the tracking device in a second real scene supporting the deployment of the tracking device, wherein the at least one detected posture is the posture of the shooting device detected by the tracking device at different shooting times; establish a corresponding relationship between the at least one detected posture and different virtual images in the virtual scene to synthesize a virtual shooting template of the virtual scene. The virtual shooting template can be reused for virtual shooting of the second real scene without the need for a tracking device, so as to reduce the cost of virtual shooting and flexibly adapt to diverse virtual shooting needs.

[0084] For ease of understanding, the virtual shooting method provided by the present application is described below in the form of a specific example.

[0085] 1. Get a virtual shooting template

[0086] Build a first form of virtual shooting system in the first real scene. Figure 5 FIG. 1 is a schematic diagram of a specific example of a virtual shooting template system according to the first embodiment of the present application. Figure 5As shown, the first form of the virtual shooting template system is arranged in the second real scene, and the stage of the second real scene is provided with an LED screen for displaying virtual images. The first form of the virtual shooting template system includes a virtual shooting server, a shooting device, an LED screen and a tracking device. The shooting device includes a track robot and a camera. The track robot can move along the track to drive the camera to move.

[0087] A1: The camera shoots a subject of the second real scene at a shooting time of the second real scene to obtain a shooting picture. The tracking device detects the position of the camera at the shooting time and sends it to the virtual shooting server.

[0088] A2: The virtual shooting server receives the position and posture at the shooting time sent by the tracking device.

[0089] A3: The virtual shooting server determines the virtual picture corresponding to the detected posture at the shooting time in the virtual scene, and fuses the detected posture at the shooting time with the corresponding virtual picture through the least significant bit algorithm to obtain the virtual posture fusion picture at the shooting time. The virtual posture fusion pictures of each shooting time are synthesized into a virtual shooting template. And the virtual picture is sent to the LED screen for display.

[0090] A4: Encode the virtual shooting template and transmit it to the virtual shooting server of the second form of the virtual shooting template system. It can be real-time transmission or non-real-time transmission.

[0091] 2. Reuse virtual shooting templates for virtual shooting

[0092] Figure 6 FIG. 1 is a schematic diagram of a specific example of a virtual shooting template system according to the second embodiment of the present application. Figure 6 As shown, the second form of the virtual shooting template system is arranged in the first real scene, and the stage of the first real scene is provided with an LED screen for displaying a virtual picture. The second form of the virtual shooting template system includes a virtual shooting server, a shooting device, and an LED screen. The shooting device includes a track robot and a camera. The track robot can move along the track under the control of the virtual shooting server, thereby driving the camera to move.

[0093] B 1: Receive the encoded virtual shooting template and decode it to obtain the current virtual posture fusion image.

[0094] B2: Extract the current expected posture of the shooting device and the current expected virtual picture corresponding to the current shooting time from the current virtual posture fusion picture through the least significant bit algorithm.

[0095] B3: Send the current expected virtual image to the LED screen for display. Control the camera to adjust from the previous position to the current expected position, and obtain the current shooting image of the camera at the current expected position.

[0096] B4: The current expected virtual picture and the current shooting picture are merged to obtain the current virtual shooting picture. The virtual shooting pictures of each shooting time within the overall shooting time range constitute the final virtual shooting result.

[0097] Figure 7 Schematic diagram of the structure of an embodiment of a virtual shooting control device provided by the present application. Figure 7 As shown, the virtual shooting control device 70 includes an acquisition module 71 , a control module 72 and a fusion module 73 .

[0098] The acquisition module 71 is used to obtain the current expected posture of the shooting device and the current expected virtual image corresponding to the current shooting time based on a preset virtual shooting template. The virtual shooting template represents the correspondence between the posture of the shooting device at different shooting times in the motion trajectory and the virtual image in the virtual scene.

[0099] The control module 72 is used to control the camera to adjust from the previous posture to the current desired posture, and obtain the current shooting picture of the camera in the current desired posture.

[0100] The fusion module 73 is used to fuse the current shooting picture with the current expected virtual picture to obtain the current virtual shooting picture.

[0101] For other detailed descriptions of the virtual shooting control device 70 , please refer to the previous embodiments, which will not be described here in detail.

[0102] Figure 8 FIG. 1 is a schematic diagram of the structure of another embodiment of the virtual shooting control device provided by the present application. Figure 8 As shown, the virtual shooting control device 80 includes a receiving module 81 and a corresponding module 82 .

[0103] The receiving module 81 is used to receive at least one detected posture sent by the tracking device, where the at least one detected posture is the posture of the shooting device detected by the tracking device at different shooting times.

[0104] The corresponding module 82 is used to establish a corresponding relationship between at least one detected posture and different virtual images in the virtual scene to synthesize a virtual shooting template of the virtual scene, wherein the virtual shooting template is used to instruct the shooting device to subsequently shoot according to the corresponding detected posture at different shooting times, and to fuse the captured shooting picture with the corresponding virtual picture to obtain a virtual shooting picture.

[0105] For other detailed descriptions of the virtual shooting control device 80 , please refer to the previous embodiments, which will not be described here in detail.

[0106] Fig. 9 Schematic diagram of the structure of an embodiment of the electronic device of the present application. Fig. 9 As shown, the electronic device 90 includes a memory 91 and a processor 92, and the processor 92 is used to execute the program instructions stored in the memory 91 to implement the steps in any of the above method embodiments. In a specific implementation scenario, the electronic device 90 may include but is not limited to: a microcomputer, a control device, and in addition, the electronic device 90 may also include a carrier device such as a laptop computer and a tablet computer, which is not limited here.

[0107] Specifically, the processor 92 is used to control itself and the memory 91 to implement the steps in any of the above method embodiments. The processor 92 can also be called a CPU (Central Processing Unit). The processor 92 may be an integrated circuit chip with signal processing capabilities. The processor 92 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 92 can be implemented by an integrated circuit chip.

[0108] See also Fig.10 , Fig.10 The computer-readable storage medium 100 stores program instructions 101, which implement the steps in any of the above method embodiments when executed by a processor.

[0109] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0110] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0111] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, units or components can be combined or integrated into another system, or some features can be ignored or not executed. In another image position, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0112] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a control device, or a network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

Claims

1. A virtual shooting method, characterized in that: Applied to a virtual shooting system, the virtual shooting system includes a shooting device and a virtual shooting control device, and the method includes: Based on a preset virtual shooting template, a current expected posture of the shooting device and a current expected virtual picture corresponding to the current shooting time are obtained, wherein the virtual shooting template represents the corresponding relationship between the postures at different shooting times in the motion trajectory of the shooting device and the virtual pictures in the virtual scene; Controlling the photographing device to adjust from a previous posture to the current desired posture, and obtaining a current photographing picture of the photographing device at the current desired posture; The current shooting picture and the current expected virtual picture are merged to obtain a current virtual shooting picture.

2. The method according to claim 1, characterized in that The virtual shooting template includes a virtual posture fusion picture at each shooting time, wherein the virtual posture fusion picture at the shooting time represents the corresponding relationship between the posture at the shooting time and the virtual picture, and is obtained by fusing the posture at the shooting time and the corresponding virtual picture; The method of obtaining the current expected posture and the current expected virtual picture of the shooting device corresponding to the current shooting time based on the preset virtual shooting template includes: Acquire the virtual posture fusion picture of the current shooting time from the virtual shooting template; The current expected posture and the current expected virtual picture are obtained from the virtual posture fusion picture at the current shooting time.

3. The method according to claim 2, characterized in that The virtual posture fusion picture at the shooting time is obtained by fusing the posture at the shooting time and the corresponding virtual picture through the least significant bit algorithm; The step of fusing the virtual posture image at the current shooting time to obtain the current expected posture and the current expected virtual image includes: The current expected posture and the current expected virtual picture are obtained from the virtual posture fusion picture at the current shooting time through the least significant bit algorithm.

4. The method according to claim 1, characterized in that: Before obtaining the current expected posture and the current expected virtual picture of the shooting device corresponding to the current shooting time based on the preset virtual shooting template, the method further includes: Receiving at least one detected posture sent by a tracking device, wherein the at least one detected posture is a posture of the shooting device detected by the tracking device at different shooting times; respectively establishing corresponding relationships between the at least one detected posture and different virtual images in the virtual scene to synthesize a virtual shooting template of the virtual scene; The shooting device is arranged in a first real scene during the generation of the virtual shooting picture, and is arranged in a second real scene during the detection of the tracking device. The first real scene and the second real scene are the same or different.

5. The method according to claim 1, characterized in that The virtual shooting system further includes a display screen, and after obtaining the current expected posture and the current expected virtual picture of the shooting device corresponding to the current shooting time based on the preset virtual shooting template, further includes: The current desired virtual picture is sent to the display screen, and the display screen is controlled to display the current desired virtual picture in a first real scene.

6. A virtual shooting method, characterized in that: Applied to a virtual shooting system, the virtual shooting system includes a shooting device, a virtual shooting control device and a tracking device, including: receiving at least one detected posture sent by the tracking device, wherein the at least one detected posture is a posture of the shooting device detected by the tracking device at different shooting times; A correspondence is established between the at least one detected posture and different virtual images in the virtual scene to synthesize a virtual shooting template of the virtual scene, wherein the virtual shooting template is used to instruct the shooting device to subsequently shoot according to the corresponding detected posture at different shooting times, and the captured shooting image is merged with the corresponding virtual image to obtain a virtual shooting image.

7. The method according to claim 6, characterized in that The step of establishing a correspondence between the at least one detected posture and different virtual images in the virtual scene to synthesize a virtual shooting template of the virtual scene includes: For each of the detected postures, the detected posture is fused with a virtual picture to obtain a virtual posture fusion picture, and the virtual posture fusion picture represents the corresponding relationship between the detected posture and the virtual picture.

8. The method according to claim 7, characterized in that The step of fusing the detected posture with the virtual picture to obtain a virtual posture fusion picture includes: The detected posture is written into the virtual picture through the least significant bit algorithm to obtain the virtual posture fusion picture.

9. A virtual shooting control device, characterized in that: include: An acquisition module, used to acquire a current expected posture of a shooting device and a current expected virtual picture corresponding to a current shooting time based on a preset virtual shooting template, wherein the virtual shooting template represents a corresponding relationship between the posture of the shooting device at different shooting times in a motion trajectory and a virtual picture in a virtual scene; A control module, used for controlling the photographing device to adjust from a previous posture to the current desired posture, and obtaining a current photographing picture of the photographing device at the current desired posture; The fusion module is used to fuse the current shooting picture with the current expected virtual picture to obtain the current virtual shooting picture.

10. A virtual shooting control device, characterized in that: include: A receiving module, configured to receive at least one detected posture sent by a tracking device, wherein the at least one detected posture is a posture of a shooting device detected by the tracking device at different shooting times; A corresponding module is used to establish a corresponding relationship between the at least one detected posture and different virtual images in the virtual scene to synthesize a virtual shooting template of the virtual scene, wherein the virtual shooting template is used to instruct the shooting device to subsequently shoot according to the corresponding detected posture at different shooting times, and to fuse the captured shooting picture with the corresponding virtual picture to obtain a virtual shooting picture.

11. A virtual shooting system, characterized in that: It includes shooting equipment and virtual shooting control device; The shooting device is used to shoot pictures under the control of the virtual shooting control device; The virtual shooting control device is pre-configured with a virtual shooting template for executing the method according to any one of claims 1-4.

12. The system according to claim 11, characterized in that The virtual shooting control device is a virtual shooting server; and / or the virtual shooting system further includes a tracking device, the tracking device is used to detect the position and posture of the shooting device, and the detected position and posture is used to generate the virtual shooting template.

13. An electronic device, characterized in that: The invention comprises a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the method according to any one of claims 1 to 8.

14. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images