Virtual image optimization methods, devices, and mobile terminals for virtual shooting systems

Abstract

This disclosure relates to a method, apparatus, and mobile terminal for optimizing virtual images in a virtual shooting system. The method includes: displaying the current focus parameters of a virtual camera, including a focus distance, in an interactive interface of the mobile terminal; the virtual camera being used to generate a virtual image; adjusting the current focus parameters of the virtual camera in response to an adjustment operation on the focus parameters; and displaying the virtual image on a physical screen in the virtual shooting system, wherein the depth-of-field effect of the virtual image is related to the focus parameters. According to embodiments of this disclosure, users can conveniently optimize the depth-of-field effect of the virtual image displayed on a real-world screen, thereby making the virtual image displayed on the physical screen, which serves as a background, more realistic.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of virtual shooting, in particular to a virtual picture optimization method and device for a virtual shooting system and a mobile terminal. BACKGROUND

[0002] Traditional film and television production needs to spend a lot of time and manpower to select shooting locations, make props, and build shooting scenes in order to meet shooting requirements. The virtual shooting (or virtual production) technology replaces real setting by rendering visual effect pictures on a built screen (such as an LED screen), reduces the dependence of film and television shooting on locations and settings, and reduces the cost and production cycle of film and television shooting.

[0003] In the virtual shooting technology, a virtual camera is usually placed in a virtual scene rendered by a rendering engine to simulate the displacement and rotation of a real camera. In the virtual shooting process, a virtual picture generated based on the virtual camera is projected onto an LED screen in the real world, and the LED screen displaying the virtual picture is used as a background board for shooting with a real camera. However, the virtual picture displayed on the LED screen as a background board is less realistic compared to the real scene, and usually needs to be optimized by professionals on a computer, which is not convenient. SUMMARY

[0004] Therefore, the present disclosure provides a virtual picture optimization method and device for a virtual shooting system and a mobile terminal, which can enable a user to conveniently adjust the focus parameter of a virtual camera to optimize the depth of field effect of a virtual picture, so that the virtual picture displayed on an entity screen as a background board is more realistic.

[0005] According to an aspect of the present disclosure, a virtual picture optimization method for a virtual shooting system is provided, which is applied to a mobile terminal and includes: displaying a current focus parameter of a virtual camera in an interactive interface of the mobile terminal, the focus parameter including a focusing distance, and the virtual camera being used to generate a virtual picture; in response to an adjustment operation on the focus parameter, adjusting the current focus parameter of the virtual camera, and displaying the virtual picture on an entity screen in the virtual shooting system, a depth of field effect of the virtual picture being related to the focus parameter.

[0006] In a possible implementation, the interactive interface displays at least two adjustment precisions of the focusing distance, and in response to the adjustment operation on the focus parameter, adjusting the current focus parameter of the virtual camera includes: in response to an adjustment operation of adjusting the focusing distance at any adjustment precision in the interactive interface, adjusting the current focusing distance of the virtual camera.

[0007] In a possible implementation, the focus parameter further includes a focus size and / or a brightness and darkness degree, the focus size is used to control a focus range of the virtual camera, and the brightness and darkness degree is used to control a brightness and darkness degree of a picture out of the focus range in a field of view of the virtual camera, wherein a size of a focus area in which a focus point of a virtual picture displayed on the entity screen is related to the focus size, and a brightness and darkness effect of a picture out of the focus area is related to the brightness and darkness degree.

[0008] In a possible implementation, the method further includes: displaying, in the interactive interface, a preview picture corresponding to a virtual picture in a field of view of the virtual camera, the preview picture being used to represent a display effect of the virtual picture generated based on the focus parameter.

[0009] In a possible implementation, at least two virtual cameras are arranged in the virtual scene, and the displaying, in the interactive interface of the mobile terminal, of the current focus parameter of the virtual camera includes: in response to selection of any virtual camera in the interactive interface of the mobile terminal, displaying the current focus parameter of the selected target virtual camera, and wherein the adjusting, in response to the adjustment operation on the focus parameter, of the current focus parameter of the virtual camera includes: in response to an adjustment operation of adjusting the current focus parameter of the selected target virtual camera in the interactive interface, adjusting the current focus parameter of the target virtual camera.

[0010] In a possible implementation, the virtual shooting system further includes a real camera associated with the virtual camera, and the method further includes: displaying, in the interactive interface, a camera parameter of the real camera associated with the virtual camera, and representing the current camera parameter of the virtual camera by the camera parameter of the real camera.

[0011] According to another aspect of the present disclosure, a virtual picture optimization apparatus for a virtual shooting system is provided, the apparatus being applied to a mobile terminal and including: a display module configured to display, in an interactive interface of the mobile terminal, a current focus parameter of a virtual camera, the focus parameter including a focusing distance, the virtual camera being configured to generate a virtual picture; and an adjustment module configured to adjust, in response to an adjustment operation on the focus parameter, the current focus parameter of the virtual camera, and display the virtual picture on an entity screen in the virtual shooting system, a depth of field effect of the virtual picture being related to the focus parameter.

[0012] In a possible implementation, at least two adjustment precisions of the focusing distance are displayed in the interactive interface, and wherein the adjusting, in response to the adjustment operation on the focus parameter, of the current focus parameter of the virtual camera includes: in response to an adjustment operation of adjusting the focusing distance at any adjustment precision in the interactive interface, adjusting the current focusing distance of the virtual camera.

[0013] In one possible implementation, the focus parameter further includes focus size and / or brightness, wherein the focus size is used to control the focus range of the virtual camera, and the brightness is used to control the brightness of the image outside the focus range within the virtual camera's field of view. The size of the focus area in the virtual image displayed on the physical screen is related to the focus size, and the brightness effect of the image outside the focus area is related to the brightness.

[0014] In one possible implementation, the device further includes a preview module, used to display a preview image corresponding to the virtual image within the field of view of the virtual camera in the interactive interface, the preview image being used to characterize the display effect of the virtual image generated based on focus parameters.

[0015] In one possible implementation, the virtual scene includes at least two virtual cameras. Displaying the current focus parameters of the virtual cameras in the interactive interface of the mobile terminal includes: displaying the current focus parameters of the selected target virtual camera in response to selecting any virtual camera in the interactive interface of the mobile terminal; wherein adjusting the current focus parameters of the virtual camera in response to an adjustment operation on the focus parameters includes: adjusting the current focus parameters of the target virtual camera in response to an adjustment operation on adjusting the current focus parameters of the selected target virtual camera in the interactive interface.

[0016] In one possible implementation, the virtual shooting system further includes a real camera associated with and bound to the virtual camera, and the device further includes: a camera parameter display module, used to display the camera parameters of the real camera associated with and bound to the virtual camera in the interactive interface, and to represent the current camera parameters of the virtual camera through the camera parameters of the real camera.

[0017] According to another aspect of this disclosure, a mobile terminal is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-described method when executing instructions stored in the memory.

[0018] According to another aspect of this disclosure, a non-volatile computer-readable storage medium is provided that stores computer program instructions thereon, wherein the computer program instructions, when executed by a processor, implement the above-described method.

[0019] According to another aspect of this disclosure, a computer program product is provided, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of a mobile terminal, the processor in the mobile terminal performs the above-described method.

[0020] According to embodiments of this disclosure, users can conveniently optimize the depth-of-field effect of the virtual image by adjusting the focus parameters of the virtual camera on a mobile terminal, thereby making the virtual image displayed on the physical screen, which serves as a background, more realistic. This does not require professional expertise. Furthermore, users can flexibly adjust the focus distance in real time to change the depth-of-field effect of the virtual image displayed on the physical screen, thereby achieving a zoom effect for the virtual image. This also satisfies the user's zoom requirements for the virtual image displayed on the physical screen during virtual shooting, which is beneficial for improving the efficiency of virtual shooting.

[0021] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0022] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this disclosure together with the specification and serve to explain the principles of this disclosure.

[0023] Figure 1 A schematic diagram illustrating an application scenario according to an embodiment of the present disclosure is shown.

[0024] Figure 2 A schematic diagram of a virtual shooting system according to an embodiment of the present disclosure is shown.

[0025] Figure 3 A flowchart illustrating a virtual image optimization method for a virtual shooting system according to an embodiment of the present disclosure is shown.

[0026] Figure 4 A schematic diagram of an interactive interface according to an embodiment of the present disclosure is shown.

[0027] Figure 5a and Figure 5b A schematic diagram showing a virtual screen according to an embodiment of the present disclosure.

[0028] Figure 6 A schematic diagram of an interactive interface according to an embodiment of the present disclosure is shown.

[0029] Figure 7 A block diagram of a virtual image optimization apparatus for a virtual shooting system according to an embodiment of the present disclosure is shown.

[0030] Figure 8 A block diagram of a mobile terminal 800 according to an embodiment of the present disclosure is shown. Detailed Implementation

[0031] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0032] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0033] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0034] Figure 1 The diagram illustrates an application scenario according to an embodiment of the present disclosure, such as... Figure 1 As shown, the virtual shooting system includes a rendering device 01, a physical screen 02, and a real camera 03. The mobile terminal 00 can establish a communication connection with the rendering device 01, and the rendering device 01 can establish a communication connection with the physical screen 02. The mobile terminal 00 can execute the virtual image optimization method of this embodiment to obtain the adjusted focus parameters and send them to the rendering device 01. The rendering device 01 can render the virtual image within the field of view of the virtual camera according to the focus parameters sent by the mobile terminal 00 and control the physical screen 02 to display the rendered virtual image.

[0035] The rendering device can establish a virtual scene, in which a virtual camera is set. The virtual camera is associated and bound to a real camera 03 in the virtual shooting system to simulate the displacement and rotation of the real camera. The virtual camera follows the movement of the real camera, and the rendering device can generate virtual images based on the focus parameters of the virtual camera under different poses. The user can control the real camera 03 to take pictures of the physical screen 02 displaying the virtual images, and then merge the captured real images with the virtual images to obtain a virtual-real fused image, achieving the purpose of virtual shooting. The real camera mentioned in this embodiment can be a video camera, such as a wide-angle camera or a telephoto camera. This embodiment does not limit the type or number of cameras.

[0036] In the above application scenario, the user can hold a mobile terminal 00 in front of a physical screen 02. Based on the depth-of-field effect of the virtual image displayed in real time on the physical screen 02, the user can adjust the focus parameters of the virtual camera in real time through the mobile terminal 00 to flexibly adjust the depth-of-field effect of the virtual image on the physical screen. By adjusting the depth-of-field effect of the virtual image on the physical screen, the zoom effect of the virtual image can be achieved, which can simulate real-world scenes. The operation is simple and convenient, without the need for professional personnel to operate on a computer based on human experience. The adjustment is highly flexible and the optimization efficiency of the virtual image is high. In this way, when the real camera 03 is focused on the physical screen 02 and a shot is taken, the depth-of-field effect of the virtual image on the physical screen can be captured, which is beneficial to improving the virtual-real fusion effect.

[0037] The physical screen 02 used in the virtual shooting system can be of the type of LED screen, LCD screen, etc., and can have a curved screen or flat screen structure. It should be understood that those skilled in the art can customize the type, quantity, size, resolution, etc. of the physical screen 02 in the virtual shooting system according to actual needs, and this disclosure embodiment does not limit this. It should be understood that this disclosure embodiment does not limit the communication connection method between devices.

[0038] Optionally, when the physical screen 02 is an LED screen, this disclosure embodiment also provides Figure 2 A virtual shooting system is shown, such as Figure 2 As shown, the virtual shooting system may include an LED screen 101, a server cluster 102 (including a main server 1021 and multiple servers 1022), an LED broadcast control processor 103, a camera 104, a pose tracker 105, a network switch 106, and a synchronization signal generator 107; the rendering device 01 of this embodiment may include the server cluster 102 and the LED broadcast control processor 103; wherein, the mobile terminal 00 can establish a communication connection with the main server 1021, so that the mobile terminal 00 can send focus parameters to the main server 1021, and the main server 1021 can synchronize the focus parameters to the other multiple servers 1022.

[0039] In this system, each server in server cluster 102 can render virtual images based on focus parameters and send the rendering results to the corresponding LED broadcast control processor 103. Since the LED screen 101 is composed of multiple LED cabinets, one server can correspond to one or more LED cabinets; that is, one server can render the images required to be displayed by one or more corresponding LED cabinets. The LED broadcast control processor 103 is a hardware device used to control the LED screen 101. One LED broadcast control processor can correspond to one or more LED cabinets, or in other words, one LED broadcast control processor can control one or more corresponding LED cabinets to display images.

[0040] The pose tracker 105 is attached to the camera 104. The pose tracker 105 can track the position and attitude of the camera 104 in real time and broadcast the tracked pose information to the local area network (LAN) via the network. The network switch 106 can establish the LAN, enabling communication between devices within the LAN. For example, the network switch 106 can receive the pose information broadcast by the pose tracker 105 and send it to the server cluster 102. The synchronization signal generator 107 can generate synchronization signal pulses and send the synchronization signals to the camera 104, pose tracker 105, server cluster 110, and LED broadcast control processor 103, enabling synchronization among these components. The camera 104 can capture real-world footage in real time and transmit it back to the server cluster 102 in real time. This allows for the fusion of the real-world footage captured by the camera 104 with the visual effects rendered in the server cluster 102 to generate a virtual fused image, thus achieving virtual shooting.

[0041] The virtual screen optimization method of this disclosure can be deployed on various mobile terminals through software or hardware modifications. For example, mobile terminals may include, but are not limited to, handheld devices, tablets, and PDAs. The mobile terminal involved in this disclosure can refer to a device with wireless and / or wired connection capabilities. Wireless connection capability means that it can connect to other devices via wireless connection methods such as Wi-Fi and Bluetooth. The mobile terminal involved in this disclosure can also communicate with other devices via wired connection capabilities. The mobile terminal involved in this disclosure can be touchscreen or non-touchscreen. Touchscreen mobile terminals can be controlled by clicking or swiping on the display using fingers or styluses. Non-touchscreen mobile terminals can be connected to input devices such as mice, keyboards, and touch panels to control the mobile terminal. This disclosure does not limit the scope of the mobile terminal.

[0042] Figure 3 A flowchart illustrates a virtual image optimization method for a virtual shooting system according to an embodiment of the present disclosure. The method is applied to a mobile terminal, such as… Figure 3 As shown, the method includes steps S31 to S32.

[0043] Step S31: Display the current focus parameters of the virtual camera in the interactive interface of the mobile terminal. The focus parameters include the focus distance. The virtual camera is used to generate virtual images.

[0044] As described above, the virtual image optimization method of this disclosure can be deployed in a mobile terminal via software, for example, in the form of an application (APP). When a user wants to optimize the virtual image, they can open the application to enter the interactive interface for optimizing the virtual image. In this process, the mobile terminal can obtain the current focus parameters of the virtual camera from an electronic device that stores the current focus parameters of the virtual camera and display them in the interactive interface. For example, the current focus parameters of the virtual camera can be obtained directly from the rendering device. This disclosure does not limit this. Displaying the focus parameters means displaying the specific values ​​of the focus parameters.

[0045] As we know, focusing distance refers to the distance between the object being photographed and the imaging plane. Focusing distance can control the depth of field to which the camera focuses, or in other words, whether the camera focuses on the foreground or the background. The farther the focusing distance, the deeper the depth of field, and the more blurred the foreground scene and the clearer the background scene in the virtual image. Conversely, the closer the focusing distance, the shallower the depth of field, and the clearer the foreground scene and the clearer the background scene.

[0046] In practical applications, after associating and binding a virtual camera with a real camera, users can control the real camera's focus to move to the physical screen to ensure that the physical screen in the real-world image captured by the real camera is clear. At this point, the camera parameters (including intrinsic and extrinsic parameters) of the virtual and real cameras are consistent, or their pose, focal length, and focus distance are identical. However, at the current focus distance of the virtual camera, the virtual camera's focus may be on distant objects in the virtual scene, resulting in blurred foreground objects in the generated virtual image, failing to meet the user's shooting needs. In this case, the user can use the virtual image optimization method of this disclosure to adjust the current focus distance of the virtual camera. For example, the virtual camera's focus distance can be reduced so that the virtual camera's focus is on foreground objects in the virtual scene, making the foreground objects in the generated virtual image clearer. It should be noted that the focus parameters adjusted in this disclosure embodiment do not affect the real camera; they mainly affect the virtual camera's generation of virtual images.

[0047] Considering that users may also need to adjust the focus distance to different levels of precision, one possible implementation is that the user interface can display at least two levels of focus distance adjustment precision. For example, it could display adjustment precision in meters and adjustment precision in centimeters. This would allow users to more easily adjust to the desired focus distance.

[0048] Exemplary embodiments of this disclosure provide, as follows: Figure 4 An interactive interface is shown, such as Figure 4As shown, users can adjust the focus distance in meters at "Focus Coarse Adjustment (meters)" and in centimeters at "Focus Coarse Adjustment (cm)". The "8M" and "12CM" currently displayed in the interactive interface represent that the virtual camera's current focus distance is 8.12 meters.

[0049] Optionally, such as Figure 4 As shown, the interactive interface can also display a preview of the virtual image within the virtual camera's field of view. This preview represents the display effect of the virtual image generated based on the focus parameters. It should be understood that before adjusting the focus parameters, the preview represents the display effect (e.g., depth of field) of the virtual image generated based on the current focus parameters; after adjusting the focus parameters, the preview represents the display effect (e.g., depth of field) of the virtual image generated based on the adjusted focus parameters. This allows users to easily adjust the focus parameters based on the preview to meet their desired depth of field in the virtual image.

[0050] Optionally, the interface can also display the camera parameters of the real camera associated with the virtual camera. These real camera parameters represent the current camera parameters of the virtual camera. Displaying the real camera parameters means displaying their values, which may include, but are not limited to, parameters such as the real camera's focus position, focus distance, aperture value, and focal length. Figure 4 As shown, the interactive interface displays "Physical camera focus position (i.e., real camera focus position) in front of / on the LED screen", "Physical camera focus distance (i.e., real camera focus distance) 0.7M", "Aperture (f-number) f / 16", and "Focal length 0.7M". The f-number (i.e., aperture value) is a relative value obtained by dividing the focal length of the camera lens by the light-transmitting diameter of the lens. In practical applications, the above camera parameters of the real camera can be obtained by a focus tracker (i.e., follow focus) set on the real camera. This embodiment of the disclosure does not limit the method of obtaining the camera parameters.

[0051] It should be understood that, under normal circumstances, the camera parameters of the virtual camera and the real camera are consistent during virtual shooting. When the real camera parameters change, the virtual camera parameters change accordingly. However, the focus distance of the real camera may not be suitable for the virtual camera, resulting in the image captured by the virtual camera not meeting the shooting requirements. Therefore, after the real camera parameters are fixed (e.g., no longer changed), according to the embodiments of this application, the focus distance of the virtual camera can be adjusted without affecting the focus distance of the real camera. This allows the image rendered on the screen to change with the focus distance of the virtual camera, resulting in a more realistic effect. In this case, displaying the camera parameters of the real camera is equivalent to displaying the current camera parameters of the virtual camera, excluding the focus distance. When the parameters of the real camera change again, the parameters of the virtual camera can follow the real camera parameters. For example, the focus distance of the virtual camera may become consistent with the focus distance of the real camera again, and it can still be adjusted. Alternatively, the focusing distance of the virtual camera may not change with the focusing distance of the real camera, but only other parameters may change with the parameters of the real camera. That is, the adjusted focusing distance of the virtual camera is maintained, and the focusing distance of the virtual camera can still be adjusted based on the method of the embodiments of this application, so as to further simplify the adjustment operation.

[0052] It is understood that at least two real cameras from different positions may be used in a virtual shooting scene. Therefore, at least two virtual cameras can be set up in the virtual scene and associated with at least two real cameras. Thus, the focus parameters of different virtual cameras can be adjusted separately. Based on this, the above-mentioned display of the current focus parameters of the virtual cameras in the interactive interface of the mobile terminal can include: in response to selecting any virtual camera in the interactive interface of the mobile terminal, displaying the current focus parameters of the selected target virtual camera.

[0053] For example, such as Figure 4 As shown, the interactive interface currently displays the focus distance, preview image, and camera parameters of the real camera corresponding to "Virtual Camera 1 (leftmost)". That is, the user's currently selected target virtual camera is "Virtual Camera 1 (leftmost)". Users can select either virtual camera by clicking the page tabs "Virtual Camera Action" and "Virtual Camera 2" at the top of the interactive interface. For example, when the user clicks "Virtual Camera 2", the interactive interface can switch between displaying the focus distance, preview image, and camera parameters of the real camera corresponding to that "Virtual Camera 2". It should be understood that this embodiment does not limit the method of selecting the virtual camera.

[0054] Optionally, the focus parameters of the virtual camera may also include focus size and / or brightness. Focus size is used to control the focus range of the virtual camera, and brightness is used to control the brightness of the image outside the focus range within the field of view of the virtual camera. The size of the focus area in the virtual image displayed on the physical screen is related to the focus size, and the brightness effect of the image outside the focus area is related to the brightness.

[0055] It is known that the image within the camera's field of view is usually the sharpest, and the image becomes increasingly blurry towards the edge of the field of view. In some shooting scenarios, users may need to increase the focus area to include more sharp areas in the virtual image, or they may need to decrease the focus area to highlight specific areas within the virtual image. Therefore, providing users with a focus size adjustment function allows them to adjust the virtual camera's focus area, thereby affecting the size of the focused area in the virtual image generated by the virtual camera. The focused area can be understood as the area occupied by the focus area within the virtual image. For example, Figure 5a and Figure 5b The image shown can be a virtual image generated under two different focus sizes. Figure 5a The focal area enclosed by the middle dashed line is smaller than Figure 5b The focal area enclosed by the dashed line.

[0056] In practice, users may also need to reduce the brightness of the image outside the focus range within the virtual camera's field of view, or darken the image outside the focus range. Therefore, a brightness adjustment function can be provided to users. By adjusting the brightness, the brightness of the image outside the focus range within the virtual camera's field of view can be adjusted, thereby affecting the brightness effect of the image outside the focus area in the virtual image generated by the virtual camera. For example, it can... Figure 5a and Figure 5b The brightness of the image outside the focal area can be darkened, or it can be brightened; this disclosure does not limit this aspect.

[0057] Since real cameras typically do not include camera parameters such as focus size and / or brightness, virtual camera parameters, which merely follow real camera parameters, often do not include these two parameters, or these two parameters are not adjustable. In this embodiment, one or more of these two parameters of the virtual camera can be adjusted via a mobile terminal, thereby achieving more diversified adjustments to the display effect on the screen to meet shooting needs.

[0058] It should be understood that the above Figure 4The illustrated interactive interface is one possible implementation provided by this embodiment. In fact, those skilled in the art can design the layout, style, etc. of the interactive interface according to actual needs, and this embodiment does not impose any limitations on this. Furthermore, those skilled in the art can independently design the display and adjustment methods of the aforementioned focus distance, focus size, and brightness in the interactive interface according to actual needs, and this embodiment does not impose any limitations on this.

[0059] Step S32: In response to the adjustment operation for the focus parameters, the current focus parameters of the virtual camera are adjusted, and a virtual image is displayed on the physical screen of the virtual shooting system. The depth-of-field effect of the virtual image is related to the focus parameters.

[0060] Understandably, users can adjust the focus parameters multiple times. The mobile terminal can send the focus parameters obtained from each adjustment operation to the rendering device. This allows the rendering device to render the virtual image within the virtual camera's field of view based on the real-time adjusted focus parameters and display it on the physical screen. This allows users to use the depth-of-field effect of the virtual image displayed on the physical screen as feedback to adjust the virtual camera's focus parameters in real time, thus meeting the user's needs for depth-of-field effects in the virtual image.

[0061] As described above, the interactive interface displays at least two adjustment accuracies for the focus distance. Based on this, adjusting the current focus parameter of the virtual camera in response to an adjustment operation on the focus parameter can include: adjusting the current focus distance of the virtual camera in response to an adjustment operation on the focus distance at any adjustment accuracies in the interactive interface. It should be understood that a single user adjustment operation can adjust the focus distance at any adjustment accuracies. Each time the focus distance is adjusted, the mobile terminal can send the adjusted focus distance to the rendering device to optimize the depth-of-field effect of the rendered virtual image displayed on the physical screen. By supporting focus distance adjustment operations at different accuracies, the need for fine-tuning the focus distance can be met.

[0062] For example, if the user is Figure 4 In the "Focus Coarse Adjustment (meters)" setting, adjusting 8M to 2M is equivalent to adjusting the focus distance in meters, which will yield the desired result. Figure 6 The interactive interface shown is as follows: Figure 6 As shown, the adjusted focus distance is 2.12 meters, and compared to... Figure 4 The depth-of-field effect of the virtual image represented in the preview screen. Figure 6 The preview screen shows that the foreground objects in the virtual scene are clearer.

[0063] As described above, at least two virtual cameras can be set up in the virtual scene. Users can select either virtual camera through the mobile terminal's interactive interface, which displays the current focus parameters of the selected target virtual camera. Based on this, adjusting the current focus parameters of the virtual camera in response to an adjustment operation on the focus parameters can include: adjusting the current focus parameters of the target virtual camera in response to an adjustment operation on the selected target virtual camera's current focus parameters in the interactive interface. It should be understood that adjusting the focus parameters of the target virtual camera optimizes the virtual image within the target virtual camera's field of view; in other words, adjusting the focus parameters of any one target virtual camera does not affect the focus parameters of other virtual cameras. In this way, the depth-of-field effect of the corresponding virtual image can be adjusted for virtual cameras at different positions to meet the user's shooting needs for real cameras at different positions.

[0064] According to embodiments of this disclosure, users can conveniently optimize the depth-of-field effect of the virtual image by adjusting the focus parameters of the virtual camera on a mobile terminal, thereby making the virtual image displayed on the physical screen, which serves as a background, more realistic. This does not require professional expertise. Furthermore, users can flexibly adjust the focus distance in real time to change the depth-of-field effect of the virtual image displayed on the physical screen, thereby achieving a zoom effect for the virtual image. This also satisfies the user's zoom requirements for the virtual image displayed on the physical screen during virtual shooting, which is beneficial for improving the efficiency of virtual shooting.

[0065] According to embodiments of this disclosure, no professional personnel are required to optimize the virtual image using a computer; the adjustment operation can be completed on a mobile terminal. Utilizing visual language, the focus distance is simplified into coarse and fine adjustments, enabling non-professionals to quickly optimize the virtual image by manipulating the focus parameters of the virtual camera. Furthermore, it satisfies users' demands for control over the virtual camera's focus in virtual shooting scenes, reducing the difference between the virtual image displayed on the physical screen and the surrounding real-world scene, making the virtual image displayed on the physical screen more realistic and lifelike. By adjusting the focus distance of the virtual camera, the depth-of-field effect of the virtual image can be changed, and a zoom effect can be achieved by changing the focus on objects at different distances. Additionally, users can use the camera parameters of the real camera collected by the follow focus device to perform zoom operations on the virtual camera.

[0066] Figure 7 This diagram illustrates a block diagram of a virtual image optimization apparatus for a virtual shooting system according to an embodiment of the present disclosure, the apparatus being applied to a mobile terminal, such as... Figure 7 As shown, the device includes:

[0067] The display module 701 is used to display the current focus parameters of the virtual camera in the interactive interface of the mobile terminal. The focus parameters include the focus distance. The virtual camera is used to generate virtual images.

[0068] The adjustment module 702 is used to adjust the current focus parameter of the virtual camera in response to the adjustment operation of the focus parameter, and to display the virtual image on the physical screen of the virtual shooting system, wherein the depth-of-field effect of the virtual image is related to the focus parameter.

[0069] In one possible implementation, the interactive interface displays at least two adjustment accuracies for the focus distance, wherein adjusting the current focus parameter of the virtual camera in response to an adjustment operation for the focus parameter includes: adjusting the current focus distance of the virtual camera in response to an adjustment operation in the interactive interface that adjusts the focus distance at any adjustment accuracies.

[0070] In one possible implementation, the focus parameter further includes focus size and / or brightness, wherein the focus size is used to control the focus range of the virtual camera, and the brightness is used to control the brightness of the image outside the focus range within the virtual camera's field of view. The size of the focus area in the virtual image displayed on the physical screen is related to the focus size, and the brightness effect of the image outside the focus area is related to the brightness.

[0071] In one possible implementation, the device further includes a preview module, used to display a preview image corresponding to the virtual image within the field of view of the virtual camera in the interactive interface, the preview image being used to characterize the display effect of the virtual image generated based on focus parameters.

[0072] In one possible implementation, the virtual scene includes at least two virtual cameras. Displaying the current focus parameters of the virtual cameras in the interactive interface of the mobile terminal includes: displaying the current focus parameters of the selected target virtual camera in response to selecting any virtual camera in the interactive interface of the mobile terminal; wherein adjusting the current focus parameters of the virtual camera in response to an adjustment operation on the focus parameters includes: adjusting the current focus parameters of the target virtual camera in response to an adjustment operation on adjusting the current focus parameters of the selected target virtual camera in the interactive interface.

[0073] In one possible implementation, the virtual shooting system further includes a real camera associated with and bound to the virtual camera, and the device further includes: a camera parameter display module, used to display the camera parameters of the real camera associated with and bound to the virtual camera in the interactive interface, and to represent the current camera parameters of the virtual camera through the camera parameters of the real camera.

[0074] According to embodiments of this disclosure, users can conveniently optimize the depth-of-field effect of the virtual image by adjusting the focus parameters of the virtual camera on a mobile terminal, thereby making the virtual image displayed on the physical screen, which serves as a background, more realistic. This does not require professional expertise. Furthermore, users can flexibly adjust the focus distance in real time to change the depth-of-field effect of the virtual image displayed on the physical screen, thereby achieving a zoom effect for the virtual image. This also satisfies the user's zoom requirements for the virtual image displayed on the physical screen during virtual shooting, which is beneficial for improving the efficiency of virtual shooting.

[0075] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0076] This disclosure also proposes a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the above-described method. The computer-readable storage medium can be volatile or non-volatile.

[0077] This disclosure also proposes an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0078] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the above-described method.

[0079] Figure 8 A block diagram of a mobile terminal 800 according to an embodiment of the present disclosure is shown. (Refer to...) Figure 8 The mobile terminal 800 may include one or more of the following components: processing component 802, memory 804, power supply component 806, multimedia component 808, audio component 810, input / output interface 812 (I / O interface), sensor component 814, and communication component 816.

[0080] Processing component 802 typically controls the overall operation of mobile terminal 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0081] Memory 804 is configured to store various types of data to support operation on mobile terminal 800. Examples of this data include instructions for any application or method operating on mobile terminal 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0082] The power supply component 806 provides power to various components of the mobile terminal 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the mobile terminal 800.

[0083] Multimedia component 808 includes a screen that provides an output interface between the mobile terminal 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the mobile terminal 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0084] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when mobile terminal 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0085] Input / output interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0086] Sensor assembly 814 includes one or more sensors for providing status assessments of various aspects of mobile terminal 800. For example, sensor assembly 814 can detect the on / off state of mobile terminal 800, the relative positioning of components such as the display and keypad of mobile terminal 800, changes in position of mobile terminal 800 or a component of mobile terminal 800, the presence or absence of user contact with mobile terminal 800, orientation or acceleration / deceleration of mobile terminal 800, and temperature changes of mobile terminal 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0087] Communication component 816 is configured to facilitate wired or wireless communication between mobile terminal 800 and other devices. Mobile terminal 800 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0088] In an exemplary embodiment, the mobile terminal 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0089] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 804 including computer program instructions, which can be executed by the processor 820 of the mobile terminal 800 to perform the above method.

[0090] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0091] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0092] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0093] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0094] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0095] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0096] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0097] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0098] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for optimizing a virtual picture for a virtual shooting system, characterized in that, The method is applied to a mobile terminal and comprises: displaying a current focus parameter of a virtual camera in an interactive interface of the mobile terminal, the focus parameter comprising a focusing distance, the virtual camera being used to generate a virtual picture; in response to an adjustment operation on the focus parameter, adjusting the current focus parameter of the virtual camera, displaying the virtual picture on a physical screen in the virtual shooting system, the depth-of-field effect of the virtual picture being related to the focus parameter, wherein a real camera shoots the physical screen on which the virtual picture is displayed, and a real-virtual fusion picture is obtained by fusing a real picture shot by the real camera and the virtual picture; the virtual shooting system further comprises a real camera associated with the virtual camera, and the method further comprises: displaying a camera parameter of the real camera associated with the virtual camera in the interactive interface; wherein when the camera parameter of the real camera changes, the camera parameter of the virtual camera changes accordingly, after the camera parameter of the real camera is fixed, the focusing distance of the virtual camera is allowed to be adjusted, and the focusing distance of the real camera is not affected, and when the camera parameter of the real camera changes again, the camera parameter of the virtual camera follows the camera parameter of the real camera.

2. The method of claim 1, wherein, The interactive interface displays at least two adjustment accuracies of the focusing distance, wherein in response to an adjustment operation on the focus parameter, adjusting the current focus parameter of the virtual camera comprises: in response to an adjustment operation on the focusing distance in any adjustment accuracy in the interactive interface, adjusting the current focusing distance of the virtual camera.

3. The method of claim 1, wherein, The focus parameter further comprises a focus size and / or a light-dark degree, the focus size being used to control the focus range of the virtual camera, and the light-dark degree being used to control the light-dark degree of a picture outside the focus range in the field of view of the virtual camera, wherein the size of a focus area in which the focus point of the virtual picture displayed on the physical screen is related to the focus size, and the light-dark effect of a picture outside the focus area is related to the light-dark degree.

4. The method of claim 1, wherein, The method further comprises: displaying a preview picture corresponding to the virtual picture in the field of view of the virtual camera in the interactive interface, the preview picture being used to represent the display effect of the virtual picture generated based on the focus parameter.

5. The method of claim 1, wherein, At least two virtual cameras are arranged in a virtual scene, and the displaying of the current focus parameter of the virtual camera in the interactive interface of the mobile terminal comprises: in response to selecting any virtual camera in the interactive interface of the mobile terminal, displaying the current focus parameter of the selected target virtual camera; wherein in response to an adjustment operation on the focus parameter, adjusting the current focus parameter of the virtual camera comprises: in response to an adjustment operation on the current focus parameter of the selected target virtual camera in the interactive interface, adjusting the current focus parameter of the target virtual camera.

6. The method of claim 1, wherein, The method further comprises: representing the current camera parameter of the virtual camera by the camera parameter of the real camera.

7. A virtual picture optimization apparatus for a virtual shooting system, characterized by, The device is applied to a mobile terminal and comprises: The display module is configured to display a current focal parameter of a virtual camera in an interactive interface of the mobile terminal, the focal parameter including a focusing distance, the virtual camera being configured to generate a virtual picture; The adjustment module is configured to adjust the current focal parameter of the virtual camera in response to an adjustment operation on the focal parameter, display the virtual picture on a physical screen in the virtual shooting system, the depth-of-field effect of the virtual picture being related to the focal parameter, and shoot the physical screen displaying the virtual picture by a real camera, and fuse a real picture obtained by shooting with the virtual picture to obtain a virtual-real fused picture. The virtual shooting system further includes a real camera associated with the virtual camera, and the device further includes: A camera parameter display module is configured to display a camera parameter of the real camera associated with the virtual camera in the interactive interface. When the camera parameter of the real camera changes, the camera parameter of the virtual camera changes accordingly, after the camera parameter of the real camera is fixed, the focusing distance of the virtual camera is allowed to be adjusted, and the focusing distance of the real camera is not affected, and when the camera parameter of the real camera changes again, the camera parameter of the virtual camera follows the camera parameter of the real camera.

8. The apparatus of claim 7, wherein, The interactive interface displays at least two adjustment precisions of the focusing distance, and adjusting the current focal parameter of the virtual camera in response to an adjustment operation on the focal parameter includes: Adjusting the current focusing distance of the virtual camera in response to an adjustment operation on the focusing distance of any adjustment precision in the interactive interface.

9. A mobile terminal, characterized by It includes: A processor; A memory for storing processor-executable instructions; The processor is configured to implement the method of any one of claims 1 to 6 when executing the instructions stored in the memory.

10. A non-transitory computer readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions are executed by the processor to implement the method of any one of claims 1 to 6.

Images