Video shooting method and device, electronic equipment and readable storage medium

Abstract

Embodiments of the present application provide a video shooting method and device, electronic equipment and readable storage medium. The method comprises: in the process of video shooting of the real camera, if the current shooting object of the real camera comprises a rendered picture in the display component, a target distance is obtained; the target distance is the distance between the real camera in the current shooting direction and the display component; the focusing distance in the shooting parameter of the virtual camera is adjusted to the target distance, so as to update the rendered picture according to the adjusted shooting parameter of the virtual camera, and update the depth-of-field effect of the rendered picture according to the actual position of the real camera. The depth-of-field effect of the rendered picture shot by the real camera can be improved while avoiding the unchanged depth-of-field effect of the rendered picture.

Description

Technical Field

[0001] This invention relates to the field of video technology, and in particular to a video shooting method, apparatus, electronic device, and readable storage medium. Background Technology

[0002] With the development of video production technology, virtual production technology based on screens is being used more and more in the film and television production field. Virtual production based on screens refers to displaying virtual images on a screen, allowing real cameras to simultaneously capture real objects and virtual images, achieving a composite effect of reality and virtuality during filming without the need for post-production processing using green screens.

[0003] In existing technologies, virtual images are typically generated in a virtual scene and then transmitted to a screen for display. However, the image on the screen is on a flat plane in the actual scene. When a real camera moves in the real scene, the depth of field of the image on the screen remains fixed, resulting in poor depth of field in the virtual image during video recording. Summary of the Invention

[0004] The purpose of this invention is to provide a video shooting method, apparatus, electronic device, and readable storage medium to achieve the technical objective of improving the depth-of-field effect in video shooting. The specific technical solution is as follows:

[0005] In a first aspect of this invention, a video shooting method is provided. This method is applied to a video shooting system comprising a real camera and a virtual camera. The virtual camera generates a rendered image by moving within a virtual scene. The rendered image is transmitted to a display component in the real scene for display. The spatial position of the virtual camera in the virtual scene is the same as the spatial position of the real camera in the real scene, and the focal length of the virtual camera is the same as the focal length of the real camera. The method includes:

[0006] During video recording by the real camera, if the current shooting object of the real camera includes the rendered image in the display component, the target distance is obtained; the target distance is the distance between the real camera and the display component in the current shooting direction.

[0007] The focus distance in the shooting parameters of the virtual camera is adjusted to the target distance, so as to update the rendered image according to the adjusted shooting parameters of the virtual camera.

[0008] In a second aspect of the invention, a video shooting device is also provided, applied to a video shooting system. The video shooting system includes a real camera and a virtual camera. The virtual camera generates a rendered image by moving within a virtual scene. The rendered image is transmitted to a display component in the real scene for display. The spatial position of the virtual camera in the virtual scene is consistent with the spatial position of the real camera in the real scene. The focal length of the virtual camera is consistent with the focal length of the real camera. The device includes:

[0009] The distance acquisition module is used to acquire a target distance if the current shooting object of the real camera includes the rendered image in the display component during the video shooting process of the real camera; the target distance is the distance between the real camera and the display component in the current shooting direction.

[0010] The parameter setting module is used to adjust the focus distance in the shooting parameters of the virtual camera to the target distance, so as to update the rendered image according to the adjusted shooting parameters of the virtual camera.

[0011] In a third aspect of the present invention, an electronic device is also provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus.

[0012] Memory, used to store computer programs;

[0013] When a processor executes a program stored in memory, it implements the method described in the first aspect above.

[0014] In a fourth aspect of the invention, a computer-readable storage medium is also provided, wherein instructions are stored therein, which, when executed on a computer, cause the computer to perform the method described in the first aspect.

[0015] In a fifth aspect of the invention, a computer program product comprising instructions is also provided, which, when run on a computer, causes the computer to perform the method described in the first aspect above.

[0016] The video shooting method provided in this invention sets up a virtual camera in a virtual scene with the same spatial position as the real camera. The virtual camera provides a rendered image for the real scene, and the distance between the real camera and the display component is set as the focusing distance of the virtual camera. This allows the depth-of-field effect of the rendered image to be updated according to the actual position of the real camera when the real camera moves in the real scene. This avoids the depth-of-field effect of the rendered image remaining unchanged, while making the depth-of-field effect of the rendered image captured by the real camera more consistent with the visual experience of the real camera's location, thereby improving the depth-of-field effect of video shooting. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0018] Figure 1 This is a flowchart illustrating the steps of a video shooting method according to an embodiment of the present invention;

[0019] Figure 2 This is a schematic diagram of the structure of a video shooting system according to an embodiment of the present invention;

[0020] Figure 3 This is a schematic diagram of a fuzzy circle in an embodiment of the present invention;

[0021] Figure 4 This is a schematic diagram of another blur circle in an embodiment of the present invention;

[0022] Figure 5 This is a flowchart illustrating a video shooting method according to an embodiment of the present invention;

[0023] Figure 6 This is a schematic diagram of the structure of a video shooting device according to an embodiment of the present invention;

[0024] Figure 7 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0025] The technical solutions of the present invention will now be described with reference to the accompanying drawings in the embodiments of the present invention.

[0026] Figure 1This is a flowchart illustrating the steps of a video shooting method according to an embodiment of the present invention. This method can be applied to a video shooting system, which includes a real camera and a virtual camera. The virtual camera generates a rendered image by moving within a virtual scene. The rendered image is transmitted to a display component in the real scene for display. The spatial position of the virtual camera in the virtual scene is the same as the spatial position of the real camera in the real scene, and the focal length of the virtual camera is the same as the focal length of the real camera. Figure 1 As shown, the method includes:

[0027] Step 101: During the video shooting process of the real camera, if the current shooting object of the real camera includes the rendered image in the display component, obtain the target distance; the target distance is the distance between the real camera and the display component in the current shooting direction.

[0028] Step 102: Adjust the focus distance in the shooting parameters of the virtual camera to the target distance, so as to update the rendered image according to the adjusted shooting parameters of the virtual camera.

[0029] A virtual camera, in this context, refers to a camera used in a virtual environment. It can simulate the shooting action of a real camera within a virtual setting, generating virtual images. Specifically, Figure 2 This is a schematic diagram of the structure of a video shooting system according to an embodiment of the present invention, such as... Figure 2 As shown, the video shooting system provided in this embodiment of the invention may further include a camera tracking subsystem and a real-time rendering subsystem. Both the camera tracking subsystem and the real-time rendering subsystem can be software-based and can run on a computer device. The computer device establishes a communication connection with the real camera and display components in the real scene for data transmission. The real camera's shooting object includes the display components. A virtual camera also runs on the computer device, and the real-time rendering subsystem is used to control the virtual camera. Specifically, the camera tracking subsystem can acquire parameters of the real camera in real time and send them to the real-time rendering subsystem. These parameters may include intrinsic and extrinsic parameters. Intrinsic parameters refer to the shooting parameters of the real camera, such as focal length, aperture, and focus distance. Extrinsic parameters refer to the position and orientation of the real camera. The real-time rendering subsystem can then set the parameters of the virtual camera based on the acquired parameters of the real camera and render a virtual image according to these parameters. The virtual image can be transmitted in real time to the display components in the real scene.

[0030] The real-time rendering subsystem can use the acquired extrinsic parameters of the real camera to set parameters such as the position and orientation of the virtual camera, thereby ensuring that the spatial position of the virtual camera in the virtual scene is consistent with that of the real camera in the real scene. Specifically, the virtual scene can be pre-matched with the real scene, that is, a virtual scene of the same size as the real scene can be created. At the same time, the positions of the zero plane, zero point, and reference point are set in the real scene to form the real three-dimensional coordinate system of the real scene. Correspondingly, the zero plane, zero point, and reference point are set in the virtual scene to be consistent with the real scene, forming the virtual three-dimensional coordinate system of the virtual scene. At this time, the coordinate systems of the two scenes are exactly the same, with the same origin and three-dimensional orientation, which facilitates the tracking of the real camera's real-time position in the virtual scene, ensuring that the spatial position of the virtual camera in the virtual scene is consistent with that of the real camera in the real scene.

[0031] Furthermore, the real-time rendering subsystem can also set the focal length of the virtual camera to that of the real camera. By keeping the focal lengths of the two consistent, the resulting virtual image can be more closely resemble the image captured by the real camera, thus improving the video shooting effect.

[0032] The aforementioned rendered image refers to the virtual image rendered by the real-time rendering subsystem according to the parameters of the virtual camera. The aforementioned display component refers to the component used to display the virtual image in the real scene, which can be a light-emitting diode (LED) screen. Specifically, in a real-world scenario, the rendered image generated in the virtual scene is usually used as a virtual background. By displaying the rendered image on an LED screen in the real scene, the real camera can simultaneously capture both the foreground actors in the real scene and the virtual background.

[0033] The aforementioned shooting objects refer to objects included within the current shooting range of the real camera. Specifically, during the shooting process, the real camera can shoot both the real scene and the rendered image according to actual needs. In one scenario, the real camera can only shoot the foreground actors in the real scene. Since the real camera does not shoot the rendered image on the display component in this case, no adjustment to the rendered image is required. In another scenario, the real camera can shoot both the foreground actors in the real scene and the virtual background simultaneously. In this case, the rendered image needs to be adjusted in real time according to the parameters of the real camera. This embodiment of the invention is performed under the condition that the real camera can simultaneously shoot both the foreground actors and the virtual background.

[0034] Understandably, in actual film and television production, the position and shooting parameters of the real camera are not fixed. If the rendered image on the display remains unchanged during actual shooting, the depth-of-field effect of the rendered image captured by the real camera will not conform to visual experience. For example, if the distance between the real camera and the display component increases while the shooting parameters of the real camera remain unchanged, according to visual experience, the depth-of-field effect of the rendered image captured by the real camera should be further blurred, and the degree of blurring of virtual objects at different distances in the rendered image should also be different.

[0035] In step 101 above, the current shooting object including the rendered image in the display component refers to the situation where the real camera is simultaneously shooting the foreground actor and the virtual background. In this case, the embodiment of the present invention can obtain the distance between the real camera and the display component in the current shooting direction as the target distance. Specifically, the above-mentioned operation of obtaining the target distance can be obtained through the extrinsic parameters of the real camera and the position of the display component in the real scene. The extrinsic parameters include the three-dimensional position and three-dimensional orientation of the real camera, that is, the six degrees of freedom (6DOF) pose of the real camera. In the above-mentioned real three-dimensional coordinate system, the embodiment of the present invention can take the three-dimensional position of the real camera as the starting point and the three-dimensional orientation of the real camera as the direction vector. The display component can be equivalent to a plane in the real three-dimensional coordinate system. Through the above-mentioned starting point and direction vector, the endpoint coordinates of the straight line formed by the starting point along the three-dimensional orientation in the plane where the display component is located can be determined, thereby obtaining the above-mentioned target distance based on the starting point coordinates and the endpoint coordinates.

[0036] Furthermore, after obtaining the target distance, this embodiment of the invention can set the focusing distance of the virtual camera to the aforementioned target distance. This allows the real-time rendering subsystem to render according to the adjusted shooting parameters of the virtual camera, resulting in a new rendered image. The depth-of-field effect of the new rendered image better matches the visual experience of the actual camera's location. In this embodiment, the focusing distance refers to the object distance, which is the distance from the focus point to the optical center of the lens.

[0037] It should be noted that the above-mentioned operation of obtaining the target distance can be performed at a frequency higher than the shooting frame rate, so that the rendered image can be updated in real time as the actual camera moves. The shooting frame rate mentioned above is preset according to actual shooting needs; different shooting needs correspond to different shooting frame rates. For example, the shooting frame rate for a TV series can be set to 25 FPS, while the shooting frame rate for a movie can be 24 FPS.

[0038] In summary, the video shooting method provided by the embodiments of the present invention, by setting a virtual camera in a virtual scene with the same spatial position as the real camera, provides a rendered image for the real scene through the virtual camera, and sets the distance between the real camera and the display component as the focusing distance of the virtual camera, so that when the real camera moves in the real scene, the depth-of-field effect of the rendered image is updated according to the actual position of the real camera. This can avoid the depth-of-field effect of the rendered image remaining unchanged, while making the depth-of-field effect of the rendered image captured by the real camera more consistent with the visual experience of the real camera's location, thereby improving the video shooting effect.

[0039] Optionally, after obtaining the target distance in step 101 above, this embodiment of the invention may further include the following steps:

[0040] S21. Obtain the focal length of the real camera, the focusing distance of the real camera, and the aperture value of the real camera.

[0041] S22. Determine a virtual aperture value based on the focal length of the real camera, the focusing distance of the real camera, the aperture value of the real camera, and the target distance, so that the difference between the diameter of the first blur circle and the diameter of the second blur circle is equal to the diameter of the third blur circle.

[0042] Wherein, the first blur circle diameter is the actual blur circle diameter formed when the real camera captures the rendered image located at the target distance according to the real camera's focal length, focus distance, and aperture value; the second blur circle diameter is the ideal blur circle diameter when the real camera captures the sub-object in the rendered image located at the target distance according to the real camera's focal length, focus distance, and aperture value; and the third blur circle diameter is the blur circle diameter corresponding to the sub-object when the virtual camera generates the rendered image according to the virtual aperture value.

[0043] After adjusting the focus distance in the shooting parameters of the virtual camera to the target distance in step 102 above, this embodiment of the invention may further include the following steps:

[0044] S22. Adjust the aperture value in the shooting parameters of the virtual camera to the virtual aperture value.

[0045] In this process, step S21 can be implemented through the camera tracking subsystem, which can acquire the focal length, focus distance, and aperture value of the real camera in real time. The circle of blur refers to the range of the circular light spot formed on the camera's image plane when the camera is shooting an object located before or after the focus point. A larger circle of blur diameter indicates a greater depth of field and a deeper degree of bokeh. The circle of blur diameter is related to the camera's lens aperture diameter, focal length, focus distance, and the distance between the subject and the camera, while the aperture is related to the lens aperture diameter and focal length.

[0046] Accordingly, the aforementioned first blur circle diameter refers to the diameter of the blur circle formed by the real camera when capturing the rendered image. It's understandable that, since the rendered image is usually used as a virtual background, the focus of the real camera is often on the real actor in front of the display component. Therefore, during the shooting process, the rendered image will form a blur circle on the image plane of the real camera. It should be noted that, since the rendered image in a real scene is located on the same display component, different virtual objects contained in the rendered image will form blur circles of the same diameter on the image plane of the real camera. The aforementioned ideal blur circle diameter refers to the diameter of the blur circle that any sub-object or any virtual object contained in the rendered image should form on the image plane of the real camera.

[0047] Figure 3 This is a schematic diagram of a fuzzy circle in an embodiment of the present invention, as shown below. Figure 3 As shown, Figure 3 In this context, "Lens" refers to an idealized model of a camera lens; "Physical Camera Sensor" refers to the light-sensitive sensor of a real camera, equivalent to the image plane of a real camera; "Physical Actor" refers to a real actor; "Physical LED Screen" refers to the display component in a real scene; and "Virtual Env" refers to a virtual object in the rendered image. p This represents the aperture diameter of the actual camera lens. The actual camera's focusing distance is S. P When the focus point is on the real actor, the real camera can get a clear image of the real actor, who is a point on the image plane.

[0048] Figure 3 This assumes that all virtual objects in the rendered image are located in the real scene, and that a certain virtual object is located at a distance S from the camera. XP The location. C L The actual camera focusing distance is S PAt that time, the diameter of the blur circle formed by an object at the location of the LED curtain wall on the camera sensor is the diameter of the first blur circle mentioned above. C X The actual camera focusing distance is S P At that time, the diameter of the blur circle that the virtual scene should form on the camera sensor is the diameter of the second blur circle mentioned above.

[0049] It can be seen that the diameter of the blur circle formed by the real camera capturing the rendered image differs from the diameter of the blur circle that should form around the virtual objects in the rendered image, resulting in a blurring effect that does not match reality. In this case, the embodiments of the present invention can compensate for this difference using a virtual camera. Specifically, the aperture value of the virtual camera can be determined by the difference between the diameter of the first blur circle and the diameter of the second blur circle, so that the portion of the blur circle diameter missing when the real camera captures a virtual object in the LED screen wall is compensated for by the diameter of the blur circle formed by the virtual camera.

[0050] Specifically, by Figure 3 It can be deduced that the diameter of the aforementioned blur circle can be obtained from the lens aperture diameter and the distance between the lens element and the photosensitive sensor:

[0051]

[0052]

[0053] Among them, the above a p The above d represents the aperture diameter of a real camera lens. p The distance between the lens of a real camera and the light sensor, d above. L For real cameras to film in S L When considering the distance to an object, the distance between the intersection of the reflected light rays and the photosensor, as mentioned above, d. XP For real cameras to film in S XP When considering the distance to the object, the distance between the intersection of the reflected rays and the photosensitive sensor is used. From the above formulas (1) and (2), the difference between the diameter of the first blur circle and the diameter of the second blur circle can be obtained as C. X -C L .

[0054] Figure 4 This is a schematic diagram of another blur circle in an embodiment of the present invention, such as... Figure 4 As shown, Figure 4In this context, "VirtualLED Screen" refers to a virtual component built within a virtual scene that is identical to the display components. Its distance from the virtual camera is the same as the distance between the real camera and the display components. "Virtual Camera Sensor" refers to the light-sensitive sensor of the virtual camera, essentially the image plane of the virtual camera. v C is the aperture diameter of the virtual camera lens. V This refers to the situation where the virtual camera's focus distance is S. v This refers to the diameter of the blur circle formed on the virtual camera sensor when the virtual camera focuses on the virtual component; it is also the diameter of the third blur circle mentioned above.

[0055] In this embodiment of the invention, the diameter of the blur circle formed by the virtual scene on the virtual camera sensor is the difference between the diameter of the first blur circle and the diameter of the second blur circle. This allows the missing portion of the blur circle diameter when the real camera captures a virtual scene in the LED screen to be compensated by the diameter of the blur circle formed by the virtual camera, thus making C... V =C X -C L .

[0056] Furthermore, based on the properties of optical lens imaging:

[0057]

[0058]

[0059]

[0060]

[0061]

[0062]

[0063]

[0064]

[0065]

[0066]

[0067] Among them, the above a v The aperture diameter of the virtual camera lens, d above v The distance between the lens of the virtual camera and the photosensor, d above. XV For virtual cameras to capture S located in a virtual sceneXV When considering the distance to an object, the distance is the distance between the intersection of reflected light rays and the photosensor. Since the focal length and spatial position of the real camera and the virtual camera are identical, S... L ≈S V ,d L ≈d V ,S XP ≈S XV ,d XP ≈d XV To simplify calculations, they can be considered equal. The focal lengths of the real camera and the virtual camera are the same, that is, f0. V =f P Therefore, the virtual aperture value can be obtained through the above formula.

[0068] Where N is the aperture value, and correspondingly, N v N represents the aperture value of the virtual camera. p This is the aperture value of the actual camera, which can be obtained in real time through the camera tracking subsystem.

[0069] In this embodiment of the invention, a virtual aperture value is determined based on the focal length, focusing distance, aperture value, and target distance of the real camera, so that the difference between the diameter of the first blur circle and the diameter of the second blur circle is equal to the diameter of the third blur circle. The first blur circle diameter is the diameter of the blur circle formed when the real camera captures the rendered image; the second blur circle diameter is the ideal blur circle diameter when the real camera captures a sub-object in the rendered image; and the third blur circle diameter is the blur circle diameter corresponding to the sub-object when the virtual camera generates the rendered image according to the virtual aperture value. The aperture value in the shooting parameters of the virtual camera is adjusted to the virtual aperture value. Thus, by making the difference between the diameter of the first blur circle and the diameter of the second blur circle equal to the diameter of the third blur circle, the rendered image generated by the virtual camera can, to some extent, compensate for the missing blur circle diameter when the real camera captures a sub-object in the rendered image, making the depth-of-field effect of the rendered image more consistent with natural optical effects and further improving the shooting effect.

[0070] Optionally, before recording video using the real camera, embodiments of the present invention may further include the following steps:

[0071] S31. Construct a virtual component in the virtual scene, the attributes of the virtual component being the same as those of the display component; the attributes include at least position attributes, shape attributes, and size attributes.

[0072] The operation of obtaining the target distance in step 101 above may include:

[0073] S32. Obtain the straight-line distance between the virtual camera and the virtual component in multiple specified directions, wherein the multiple specified directions are the directions formed by the virtual camera and multiple corners of the shooting range of the virtual camera.

[0074] S33. Obtain the target distance based on the straight-line distances in the multiple specified directions.

[0075] The aforementioned virtual components refer to virtual 3D models, which can be built using any 3D software (e.g., 3ds Max). They can be built according to the position, shape, and size attributes of display components in the real scene, thus enabling the construction of virtual components in the virtual scene that are completely identical to the display components in the real scene.

[0076] Furthermore, since the virtual component and the display component have the same attributes, the distance between the virtual camera and the virtual component in the virtual scene is the same as the distance between the real camera and the display component in the real scene. Therefore, in this embodiment of the invention, the target distance can be obtained by using the virtual camera and the virtual component.

[0077] Specifically, the aforementioned shooting range refers to the field of view of the virtual camera, which is consistent with the shooting range of a real camera. Specifically, the aforementioned shooting range can be determined by first obtaining the focal length through the camera tracking subsystem, then calculating the lateral and longitudinal fields of view based on the focal length and the size of the camera sensor, and finally determining the vector representation in space of each corner of the captured image based on the lateral and longitudinal field of view.

[0078] Furthermore, multiple rays can be constructed using the coordinates of the virtual camera and the vector representations of its corners. By determining whether these rays intersect the plane containing the virtual component and obtaining the intersection coordinates, the straight-line distances in different specified directions can be obtained using the coordinates of the virtual camera and the various intersection coordinates. Optionally, if none of the rays intersect the plane containing the virtual component, it indicates that the current subject of the real camera does not include the rendered image in the display component. In this case, it is not necessary to set the focus distance of the virtual camera, or the focus distance of the virtual camera can be set to the default value.

[0079] Optionally, step S33 above may include: S331, obtaining the target distance based on the straight-line distances in the plurality of specified directions and the number of specified directions.

[0080] Specifically, in order to reduce the amount of calculation, the embodiment of the present invention can directly use the average value of the straight-line distances in multiple specified directions as the target distance, that is, the target distance is obtained based on the ratio of the straight-line distances in multiple specified directions to the number of specified directions.

[0081] In this embodiment of the invention, a virtual component is constructed in the virtual scene, and the attributes of the virtual component are the same as those of the display component. The attributes include at least position, shape, and size attributes. The straight-line distances between the virtual camera and the virtual component are obtained in multiple specified directions, where the multiple specified directions are the directions formed by the virtual camera and multiple corners of the virtual camera's shooting range. The target distance is obtained based on the straight-line distances in the multiple specified directions. Thus, by constructing a virtual component with the same attributes as the display component in the virtual scene, the target distance can be directly obtained through the virtual camera and the virtual component, reducing the complexity of obtaining the target distance.

[0082] Optionally, the operation of obtaining the straight-line distance between the virtual camera and the virtual component in multiple specified directions in S31 above may further include the following steps in this embodiment of the invention:

[0083] S41. Using the current spatial position of the virtual camera in the virtual scene as the starting point, perform collision detection to obtain the coordinates of the collision points located in the multiple specified directions.

[0084] S42. Based on the starting point and the coordinates of the collision points in the multiple specified directions, obtain the straight-line distance between the virtual camera and the virtual component in the multiple specified directions.

[0085] The aforementioned collision detection refers to emitting a ray at the starting position and detecting whether the ray collides with the target plane. Specifically, collision detection can be implemented through the collision detection module in the real-time rendering subsystem. The current spatial position of the virtual camera can be used as the collision starting point, the location of the virtual component as the target plane, and multiple specified directions as collision direction vectors. This allows multiple rays to be emitted along the specified directions, resulting in multiple collision point coordinates.

[0086] Furthermore, by calculating the distance between the starting point coordinates and multiple collision point coordinates using the spatial distance calculation formula, the straight-line distance between the virtual camera and the virtual component in multiple specified directions can be obtained.

[0087] In this embodiment of the invention, collision detection is performed using the current spatial position of the virtual camera in the virtual scene as the starting point to obtain the coordinates of collision points located in multiple specified directions. Based on the starting point and the coordinates of the collision points in the multiple specified directions, the straight-line distances between the virtual camera and the virtual component in the multiple specified directions are obtained. Thus, by obtaining the collision point coordinates through collision detection and obtaining the straight-line distances between the virtual camera and the virtual component based on the collision point coordinates and the starting point, the accuracy of distance acquisition can be improved.

[0088] Optionally, after determining the virtual aperture value based on the focal length of the real camera, the focusing distance of the real camera, the aperture value of the real camera, and the target distance, the embodiments of the present invention may further include the following steps:

[0089] S51. Determine the target aperture value based on the preset coefficient and the virtual aperture value.

[0090] The operation of adjusting the aperture value in the shooting parameters of the virtual camera to the virtual aperture value may specifically include: S52, adjusting the aperture value in the shooting parameters of the virtual camera to the target aperture value.

[0091] The aforementioned preset coefficients can be pre-set according to actual needs, and this embodiment of the invention does not impose any restrictions on this. For example, a value less than 1 can be set to make the captured video more in line with human visual perception, or a value greater than 1 can be set to make sub-objects in the background of the captured video more blurred.

[0092] Specifically, the target aperture value can be α is the preset coefficient mentioned above. It should be noted that in the above formula... The calculation result is usually between 0 and 1. α can be set to a value greater than 0. The larger α is, the smaller the target aperture value N, and the more blurred the sub-objects in the background of the captured video. Conversely, the smaller α is, the larger the target aperture value N, and the clearer the sub-objects in the background of the captured video.

[0093] In this embodiment of the invention, the target aperture value is determined based on a preset coefficient and the virtual aperture value; the aperture value in the shooting parameters of the virtual camera is adjusted to the target aperture value, and the determined virtual aperture value can be fine-tuned by the preset coefficient.

[0094] Figure 5 This is a flowchart illustrating a video shooting method according to an embodiment of the present invention, which may specifically include preparations before shooting and the shooting process.

[0095] Specifically, before filming, a 1:1 3D model of the LED curtain wall can be created and placed in a virtual scene, unifying the spatial coordinate system of the camera tracking subsystem and the real-time rendering subsystem.

[0096] During the shooting process, for the nth frame, the coordinates and focal length of the real camera are obtained and assigned to the virtual camera. If the real camera's shooting object does not include the rendered screen of the display component, the focus distance of the virtual camera is not updated. Otherwise, the target distance is obtained and the focus distance of the virtual camera is set to the target distance. Based on the target distance, the aperture, focal length, and focus distance of the real camera, the virtual aperture value is obtained and assigned to the virtual camera. The rendered screen is regenerated based on the virtual camera and transmitted to the display component before the next frame is shot.

[0097] The video shooting method provided in this invention uses the distance from the shooting direction of the real camera to the LED screen wall as the focusing distance of the virtual camera, ensuring that the virtual camera's focus is always locked on the location of the LED screen wall. This better conforms to the logic of the layering of reality and illusion in virtual shooting, achieving a better virtual depth-of-field effect. By determining the virtual aperture value, the diameter of the blurred circle missing when the real camera shoots virtual objects on the LED screen wall is supplemented by the diameter of the blurred circle formed by the virtual camera. This allows virtual depth-of-field blurring to be superimposed with optical depth-of-field blurring, more accurately simulating the natural optical depth-of-field effect when shooting real scenes.

[0098] Figure 6 This is a schematic diagram of a video shooting device according to an embodiment of the present invention. The device is applied to a video shooting system, which includes a real camera and a virtual camera. The virtual camera generates a rendered image by moving within a virtual scene. The rendered image is transmitted to a display component in the real scene for display. The spatial position of the virtual camera in the virtual scene is the same as the spatial position of the real camera in the real scene, and the focal length of the virtual camera is the same as the focal length of the real camera. Figure 6 As shown, the device 20 may include:

[0099] The distance acquisition module 201 is used to acquire a target distance if the current shooting object of the real camera includes the rendered image in the display component during the video shooting process of the real camera; the target distance is the distance between the real camera and the display component in the current shooting direction.

[0100] The parameter setting module 202 is used to adjust the focus distance in the shooting parameters of the virtual camera to the target distance, so as to update the rendered image according to the adjusted shooting parameters of the virtual camera.

[0101] Optionally, the device 20 further includes:

[0102] The parameter acquisition module is used to acquire the focal length of the real camera, the focusing distance of the real camera, and the aperture value of the real camera;

[0103] An aperture value determination module is used to determine a virtual aperture value based on the focal length of the real camera, the focusing distance of the real camera, the aperture value of the real camera, and the target distance, so that the difference between the diameter of the first blur circle and the diameter of the second blur circle is equal to the diameter of the third blur circle; wherein, the diameter of the first blur circle is the diameter of the blur circle formed when the real camera captures the rendered image, the diameter of the second blur circle is the ideal blur circle diameter when the real camera captures the sub-object in the rendered image, and the diameter of the third blur circle is the diameter of the blur circle corresponding to the sub-object when the virtual camera generates the rendered image according to the virtual aperture value;

[0104] An aperture adjustment module is used to adjust the aperture value in the shooting parameters of the virtual camera to the virtual aperture value.

[0105] Optionally, the device 20 further includes:

[0106] A construction module is used to construct virtual components in the virtual scene, wherein the attributes of the virtual components are the same as those of the display components; the attributes include at least position attributes, shape attributes, and size attributes.

[0107] The distance acquisition module includes:

[0108] The first acquisition submodule is used to acquire the straight-line distance between the virtual camera and the virtual component in multiple specified directions, wherein the multiple specified directions are the directions formed by the virtual camera and multiple corners of the shooting range of the virtual camera;

[0109] The second acquisition submodule is used to acquire the target distance based on the straight-line distances in the multiple specified directions.

[0110] Optionally, the aperture value determination module is specifically used for:

[0111]

[0112] Wherein, Nv represents the virtual aperture value, fp represents the focal length of the real camera, Sp represents the focusing distance of the real camera, Sv represents the target distance, and Np represents the aperture value of the real camera.

[0113] Optionally, the second acquisition submodule is specifically used for:

[0114] The target distance is obtained based on the straight-line distances in the specified directions and the number of specified directions.

[0115] Optionally, the first acquisition submodule includes:

[0116] The collision detection unit is used to perform collision detection with the current spatial position of the virtual camera in the virtual scene as the starting point, so as to obtain the coordinates of the collision points located in the multiple specified directions respectively;

[0117] The distance acquisition unit is used to acquire the straight-line distance between the virtual camera and the virtual component in multiple specified directions based on the starting point and the coordinates of the collision points in the multiple specified directions.

[0118] Optionally, the device 20 further includes:

[0119] The target acquisition module is used to determine the target aperture value based on a preset coefficient and the virtual aperture value;

[0120] The aperture adjustment module is specifically used to adjust the aperture value in the shooting parameters of the virtual camera to the target aperture value.

[0121] In summary, the video shooting device provided in this embodiment of the invention, by setting a virtual camera in a virtual scene with the same spatial position as the real camera, provides a rendered image for the real scene through the virtual camera, and sets the distance between the real camera and the display component as the focusing distance of the virtual camera, allows the depth-of-field effect of the rendered image to be updated according to the actual position of the real camera when the real camera moves in the real scene. This avoids the depth-of-field effect of the rendered image remaining unchanged, while making the depth-of-field effect of the rendered image captured by the real camera more consistent with the visual experience of the real camera's location, thereby improving the video shooting effect.

[0122] This invention also provides an electronic device, such as... Figure 7 As shown, it includes a processor 9001, a communication interface 9002, a memory 9003, and a communication bus 9004. The processor 9001, communication interface 9002, and memory 9003 communicate with each other via the communication bus 9004.

[0123] The 9003 memory is used to store computer programs;

[0124] When processor 9001 executes the program stored in memory 9003, it performs the following steps:

[0125] During video recording by the real camera, if the current shooting object of the real camera includes the rendered image in the display component, the target distance is obtained; the target distance is the distance between the real camera and the display component in the current shooting direction.

[0126] The focus distance in the shooting parameters of the virtual camera is adjusted to the target distance, so as to update the rendered image according to the adjusted shooting parameters of the virtual camera.

[0127] Optionally, after obtaining the target distance, the method further includes:

[0128] Obtain the focal length, focusing distance, and aperture value of the real camera;

[0129] A virtual aperture value is determined based on the focal length of the real camera, the focusing distance of the real camera, the aperture value of the real camera, and the target distance, so that the difference between the diameter of the first blur circle and the diameter of the second blur circle is equal to the diameter of the third blur circle.

[0130] Wherein, the first blur circle diameter is the actual blur circle diameter formed when the real camera captures the rendered image located at the target distance according to the focal length, focusing distance, and aperture value of the real camera; the second blur circle diameter is the ideal blur circle diameter when the real camera captures the sub-object in the rendered image located at the target distance according to the focal length, focusing distance, and aperture value of the real camera; and the third blur circle diameter is the blur circle diameter corresponding to the sub-object when the virtual camera generates the rendered image according to the virtual aperture value.

[0131] After adjusting the focus distance in the shooting parameters of the virtual camera to the target distance, the method further includes:

[0132] Adjust the aperture value in the shooting parameters of the virtual camera to the virtual aperture value.

[0133] Optionally, before capturing video using the real camera, the method further includes: constructing a virtual component in the virtual scene, the virtual component having the same attributes as the display component; the attributes including at least position, shape, and size attributes; obtaining the target distance includes:

[0134] Obtain the straight-line distance between the virtual camera and the virtual component in multiple specified directions, wherein the multiple specified directions are the directions formed by the virtual camera and multiple corners of the shooting range of the virtual camera;

[0135] The target distance is obtained based on the straight-line distances in the specified directions.

[0136] Optionally, determining the virtual aperture value based on the focal length of the real camera, the focusing distance of the real camera, the aperture value of the real camera, and the target distance includes:

[0137]

[0138] Wherein, the N v f represents the virtual aperture value. p S represents the focal length of the actual camera. p S represents the focusing distance of the actual camera. v N represents the target distance. p This represents the aperture value of the actual camera.

[0139] Optionally, obtaining the target distance based on the straight-line distances in the plurality of specified directions includes:

[0140] The target distance is obtained based on the straight-line distances in the specified directions and the number of specified directions.

[0141] Optionally, obtaining the straight-line distance between the virtual camera and the virtual component in multiple specified directions includes:

[0142] The current spatial position of the virtual camera in the virtual scene is used as the starting point for collision detection, so as to obtain the coordinates of the collision points located in the multiple specified directions respectively;

[0143] Based on the starting point and the coordinates of the collision points in the multiple specified directions, the straight-line distances between the virtual camera and the virtual component in the multiple specified directions are obtained.

[0144] Optionally, after determining the virtual aperture value based on the focal length of the real camera, the focusing distance of the real camera, the aperture value of the real camera, and the target distance, the method further includes:

[0145] The target aperture value is determined based on a preset coefficient and the virtual aperture value;

[0146] The step of adjusting the aperture value in the shooting parameters of the virtual camera to the virtual aperture value includes: adjusting the aperture value in the shooting parameters of the virtual camera to the target aperture value.

[0147] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0148] The communication interface is used for communication between the aforementioned terminal and other devices.

[0149] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0150] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0151] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores instructions that, when executed on a computer, cause the computer to perform any of the video capture methods described in the above embodiments.

[0152] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the video capture methods described in the above embodiments.

[0153] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0154] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0155] It should be noted that the various data-related processes in the embodiments of this application are carried out in compliance with the relevant data protection laws and policies of the country where the location is located, and with the authorization granted by the owner of the corresponding device.

[0156] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0157] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A video shooting method, characterized in that, An application is made to a video shooting system, which includes a real camera and a virtual camera. The virtual camera generates a rendered image by moving within a virtual scene. The rendered image is transmitted to a display component in the real scene for display. The spatial position of the virtual camera in the virtual scene is the same as that of the real camera in the real scene, and the focal length of the virtual camera is the same as that of the real camera. The method includes: During video recording by the real camera, if the current shooting object of the real camera includes the rendered image in the display component, the target distance is obtained; the target distance is the distance between the real camera and the display component in the current shooting direction. The focus distance in the shooting parameters of the virtual camera is adjusted to the target distance, so as to update the rendered image according to the adjusted shooting parameters of the virtual camera; After obtaining the target distance, the method further includes: Obtain the focal length, focusing distance, and aperture value of the real camera; A virtual aperture value is determined based on the focal length of the real camera, the focusing distance of the real camera, the aperture value of the real camera, and the target distance, so that the difference between the diameter of the first blur circle and the diameter of the second blur circle is equal to the diameter of the third blur circle. The diameters of the first and second blur circles are obtained through the lens aperture diameter and the distance between the lens and the photosensor, including: Among them, the The aperture diameter of the lens of the actual camera is described. The distance between the lens of the real camera and the photosensor. The real camera was used to film the location. When considering the distance to an object, the distance between the intersection of the reflected light rays and the photosensitive sensor is... The real camera is shooting at the location When considering the distance to an object, the distance between the intersection of the reflected light rays and the photosensitive sensor; The difference between the diameter of the first blur circle and the diameter of the second blur circle is The The diameter of the first blur circle; The diameter of the second blur circle; After adjusting the focus distance in the shooting parameters of the virtual camera to the target distance, the method further includes: Adjust the aperture value in the shooting parameters of the virtual camera to the virtual aperture value.

2. The method according to claim 1, characterized in that, The first blur circle diameter is the actual blur circle diameter formed when the real camera captures the rendered image located at the target distance using the real camera's focal length, focus distance, and aperture value. The second blur circle diameter is the ideal blur circle diameter when the real camera captures the sub-object in the rendered image located at the target distance using the real camera's focal length, focus distance, and aperture value. The third blur circle diameter is the blur circle diameter corresponding to the sub-object when the virtual camera generates the rendered image using the virtual aperture value.

3. The method according to claim 1, characterized in that, Before capturing video using the real camera, the method further includes: constructing a virtual component in the virtual scene, the virtual component having the same attributes as the display component; the attributes including at least position, shape, and size attributes; obtaining the target distance includes: Obtain the straight-line distance between the virtual camera and the virtual component in multiple specified directions, wherein the multiple specified directions are the directions formed by the virtual camera and multiple corners of the shooting range of the virtual camera; The target distance is obtained based on the straight-line distances in the specified directions.

4. The method of claim 1, wherein, The process of determining the virtual aperture value based on the focal length of the real camera, the focusing distance of the real camera, the aperture value of the real camera, and the target distance includes: Among them, the This represents the virtual aperture value. This indicates the focal length of the actual camera. This indicates the focusing distance of the actual camera. Indicates the target distance. This represents the aperture value of the actual camera.

5. The method of claim 3, wherein, The step of obtaining the straight-line distance between the virtual camera and the virtual component in multiple specified directions includes: The current spatial position of the virtual camera in the virtual scene is used as the starting point for collision detection, so as to obtain the coordinates of the collision points located in the multiple specified directions respectively; Based on the starting point and the coordinates of the collision points in the multiple specified directions, the straight-line distances between the virtual camera and the virtual component in the multiple specified directions are obtained.

6. The method according to claim 1, characterized in that, After determining the virtual aperture value based on the focal length of the real camera, the focusing distance of the real camera, the aperture value of the real camera, and the target distance, the method further includes: The target aperture value is determined based on a preset coefficient and the virtual aperture value; The step of adjusting the aperture value in the shooting parameters of the virtual camera to the virtual aperture value includes: adjusting the aperture value in the shooting parameters of the virtual camera to the target aperture value.

7. The method of claim 3, wherein, The step of obtaining the target distance based on the straight-line distances in the multiple specified directions includes: The target distance is obtained based on the straight-line distances in the specified directions and the number of specified directions.

8. A video camera device, characterized by comprising: The device is applied to a video shooting system, which includes a real camera and a virtual camera. The virtual camera generates a rendered image by moving within a virtual scene. The rendered image is transmitted to a display component in the real scene for display. The spatial position of the virtual camera in the virtual scene is the same as that of the real camera in the real scene, and the focal length of the virtual camera is the same as that of the real camera. The device includes: The distance acquisition module is used to acquire a target distance if the current shooting object of the real camera includes the rendered image in the display component during the video shooting process of the real camera; the target distance is the distance between the real camera and the display component in the current shooting direction. The parameter setting module is used to adjust the focus distance in the shooting parameters of the virtual camera to the target distance, so as to update the rendered image according to the adjusted shooting parameters of the virtual camera; The video recording device also includes: The parameter acquisition module is used to acquire the focal length of the real camera, the focusing distance of the real camera, and the aperture value of the real camera; An aperture value determination module is used to determine a virtual aperture value based on the focal length of the real camera, the focusing distance of the real camera, the aperture value of the real camera, and the target distance, so that the difference between the diameter of the first blur circle and the diameter of the second blur circle is equal to the diameter of the third blur circle; the diameters of the first blur circle and the diameter of the second blur circle are obtained through the lens aperture diameter and the distance between the lens and the photosensitive sensor, including: , wherein The aperture diameter of the lens of the actual camera is described. The distance between the lens of the real camera and the photosensor. The real camera was used to film the location. When considering the distance to an object, the distance between the intersection of the reflected light rays and the photosensitive sensor is... The real camera is shooting at the location When considering the distance to an object, the distance between the intersection of reflected light rays and the photosensor; the difference between the diameter of the first blur circle and the diameter of the second blur circle is... - The The diameter of the first blur circle; The diameter of the second blur circle; An aperture adjustment module is used to adjust the aperture value in the shooting parameters of the virtual camera to the virtual aperture value.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-7.

Images