Parameter processing method and device of virtual camera, electronic equipment and storage medium

Abstract

The application provides a parameter processing method and device of a virtual camera, electronic equipment and a storage medium. The method comprises: obtaining camera parameters of a first virtual camera; wherein the first virtual camera has a binding relationship with a physical camera in a real scene; performing smoothing processing on the camera parameters of the first virtual camera to obtain target camera parameters; configuring a second virtual camera in a virtual scene, wherein the focal point of the second virtual camera corresponds to the focal point of the first virtual camera; and adjusting the camera parameters of the second virtual camera based on the target camera parameters to obtain an adjusted second virtual camera; wherein the adjusted second virtual camera is used to render an image of the virtual scene including the object based on the image data. Through the application, the stability of the virtual camera can be effectively improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a parameter processing method, apparatus, electronic device, and storage medium for a virtual camera. Background Technology

[0002] Virtual-real fusion technology is a technology that cleverly integrates virtual and real scenes. It is widely used in many technical fields such as multimedia video production, 3D modeling, online conferencing, real-time registration, intelligent interaction, and sensing. Virtual-real fusion is mainly reflected in the combination of virtual and real, real-time interaction, and 3D interaction matching. It is mainly achieved through display technology, interaction technology, sensing technology, and computer graphics and image technology.

[0003] In related technologies, camera stabilization is usually achieved through hardware stabilizers (e.g., camera gimbals). Hardware stabilizers are extremely expensive, resulting in very high hardware costs and poor stability performance in related technologies. Summary of the Invention

[0004] This application provides a parameter processing method, apparatus, electronic device, computer-readable storage medium, and computer program product for a virtual camera, which can effectively improve the stability of the virtual camera.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides a parameter processing method for a virtual camera, including:

[0007] The camera parameters of the first virtual camera are obtained. The camera parameters include at least one of the attitude angle parameters, field of view parameters, and position parameters. The first virtual camera is bound to a physical camera in the real scene. The physical camera is used to collect image data of objects in the real scene to obtain image data of the objects.

[0008] The camera parameters of the first virtual camera are smoothed to obtain the target camera parameters;

[0009] A second virtual camera is configured in the virtual scene, and the focus of the second virtual camera corresponds to the focus of the first virtual camera;

[0010] Based on the target camera parameters, the camera parameters of the second virtual camera are adjusted to obtain the adjusted second virtual camera;

[0011] The adjusted second virtual camera is used to render an image of a virtual scene including the object based on the image data.

[0012] This application provides a parameter processing device for a virtual camera, comprising:

[0013] The acquisition module is used to acquire camera parameters of a first virtual camera. The camera parameters include at least one of attitude angle parameters, field of view angle parameters, and position parameters. The first virtual camera is bound to a physical camera in the real scene. The physical camera is used to acquire image data of objects in the real scene to obtain image data of the objects.

[0014] A smoothing module is used to smooth the camera parameters of the first virtual camera to obtain the target camera parameters;

[0015] A configuration module is used to configure a second virtual camera in the virtual scene, wherein the focus of the second virtual camera corresponds to the focus of the first virtual camera;

[0016] An adjustment module is used to adjust the camera parameters of the second virtual camera based on the target camera parameters to obtain an adjusted second virtual camera; wherein, the adjusted second virtual camera is used to render an image of a virtual scene including the object based on the image data.

[0017] In some embodiments, when the camera parameters include attitude angles, the acquisition module is further configured to acquire the target position of the object in the world coordinate system and the position of the first virtual camera; determine a target direction vector based on the target position of the object in the world coordinate system and the position of the first virtual camera, wherein the target direction vector is used to indicate the direction in which the first virtual camera points to the object in the world coordinate system; and determine the attitude angle of the first virtual camera based on the target direction vector.

[0018] In some embodiments, the attitude angle includes pitch angle and yaw angle; the acquisition module is further configured to determine the cosine of the vertical component of the target direction vector as the pitch angle of the first virtual camera, wherein the vertical component is the component of the target direction vector on the vertical axis of the world coordinate system; determine the ratio of the ordinate component and the abscissa component of the target direction vector as a reference ratio, wherein the ordinate component is the component of the target direction vector on the vertical axis of the world coordinate system, and the abscissa component is the component of the target direction vector on the abscissa of the world coordinate system; and determine the tangent of the reference ratio as the yaw angle of the first virtual camera.

[0019] In some embodiments, when the number of objects is one, the above-mentioned acquisition module is further configured to acquire the coordinates of multiple skeletal points of the object in the world coordinate system; and to perform a weighted summation of the multiple skeletal point coordinates to obtain the target position of the object in the world coordinate system.

[0020] In some embodiments, when the number of objects is at least two, the acquisition module is further configured to perform the following processing for each object: acquire the coordinates of multiple skeletal points of the object in the world coordinate system; perform a weighted summation of the multiple skeletal point coordinates to obtain the position of the object in the world coordinate system; and determine the target position based on the position of each object in the world coordinate system, wherein the distance between the target position and the position of each object in the world coordinate system is less than a distance threshold.

[0021] In some embodiments, the first virtual camera has n smoothing moments, where n is a positive integer greater than 1; the smoothing module is further configured to obtain a smoothing index and a (n-1)th target camera parameter when the camera parameters of the first virtual camera include the camera parameters at the nth smoothing moment; wherein, the smoothing index is used to indicate the smoothness of the camera parameters; the (n-1)th target camera parameter is the target camera parameter obtained by smoothing the camera parameters of the first virtual camera at the (n-1)th smoothing moment; based on the smoothing index and the (n-1)th target camera parameter, the camera parameters at the nth smoothing moment are smoothed to obtain the nth target camera parameter, and the nth target camera parameter is used as the target camera parameter.

[0022] In some embodiments, the smoothing index is between 0 and 1. The smoothing module is further configured to determine the product of the camera parameter at the nth smoothing time and the smoothing index as a first reference parameter; determine the product of the (n-1)th target camera parameter and the supplementary smoothing index as a second reference parameter, wherein the supplementary smoothing index is the difference between the smoothing index and 1; add the first reference parameter and the second reference parameter to obtain the nth target camera parameter, and use the nth target camera parameter as the target camera parameter.

[0023] In some embodiments, when the camera parameters include the attitude angles, the attitude angles include pitch angle, yaw angle, and roll angle; the parameter processing device for the virtual camera further includes: a locking module, configured to lock the target angle in response to a locking command for a target angle in the attitude angles; wherein the target angle includes at least one of the pitch angle, yaw angle, and roll angle; the smoothing module is further configured to smooth the portion of the attitude angles other than the target angle to obtain target camera parameters.

[0024] In some embodiments, the adjustment module is further configured to adjust the current camera parameters of the second virtual camera to the target camera parameters to obtain the adjusted second virtual camera; the parameter processing device of the virtual camera further includes: an instruction adjustment module, configured to adjust the target camera parameters in response to an adjustment instruction for the target camera parameters to obtain the adjusted second virtual camera.

[0025] In some embodiments, when the camera parameters include the attitude angle, the smoothing module is further configured to obtain the data type of the attitude angle, wherein the data type includes quaternion type and Euler angle type; when the data type is the quaternion type, each element in the attitude angle of the quaternion type is smoothed to obtain the reference attitude angle of the quaternion type; the reference attitude angle of the quaternion type is converted to a different data type to obtain the reference attitude angle of the Euler angle type, and the reference attitude angle of the Euler angle type is determined as the target attitude angle.

[0026] In some embodiments, the parameter processing device for the virtual camera further includes: a selection module, configured to determine a plurality of configured virtual cameras in the virtual scene, wherein each configured virtual camera is bound to a different physical camera; and in response to a selection operation for the plurality of configured virtual cameras, to determine the selected virtual camera as the first virtual camera.

[0027] In some embodiments, the parameter processing device for the virtual camera further includes: a second acquisition module, configured to acquire camera parameters of a third virtual camera, wherein the third virtual camera is not bound to a physical camera in the real scene; a second smoothing module, configured to smooth the camera parameters of the third virtual camera to obtain target camera parameters of the third virtual camera; a second configuration module, configured to configure a fourth virtual camera in the virtual scene, wherein the focus of the fourth virtual camera corresponds to the focus of the third virtual camera; and a second adjustment module, configured to adjust the camera parameters of the fourth virtual camera based on the target camera parameters of the third virtual camera to obtain an adjusted fourth virtual camera; wherein the adjusted fourth virtual camera is used to render an image of the virtual scene.

[0028] In some embodiments, when the camera parameters of the third virtual camera include the attitude angle, the second acquisition module is further configured to acquire the position parameters of the focal position of the third virtual camera; determine the direction vector of the third virtual camera based on the position parameters of the focal position and the position parameters of the third virtual camera; and determine the attitude angle of the third virtual camera based on the direction vector of the third virtual camera.

[0029] In some embodiments, when the camera parameters include the field of view, the second acquisition module is further configured to acquire a virtual distance when a virtual object exists within the field of view of the third virtual camera, wherein the virtual distance is the distance between a first position and a second position, the first position being the position of the third virtual camera in the world coordinate system, and the second position being the position of the virtual object in the world coordinate system; and to determine the field of view of the third virtual camera based on the virtual distance, wherein the value of the virtual distance is proportional to the value of the field of view.

[0030] In some embodiments, the parameter processing device for the virtual camera further includes: a shaking module, configured to add the shaking parameters to the camera parameters of the adjusted fourth virtual camera in response to a shaking parameter addition instruction for the adjusted fourth virtual camera, so that the image rendered by the fourth virtual camera with the added shaking parameters produces a shaking effect.

[0031] This application provides an electronic device, including:

[0032] Memory is used to store executable instructions or computer programs.

[0033] The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the parameter processing method for the virtual camera provided in the embodiments of this application.

[0034] This application provides a computer-readable storage medium storing computer-executable instructions for inducing a processor to execute and implement the parameter processing method for a virtual camera provided in this application.

[0035] This application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the virtual camera parameter processing method described above in this application.

[0036] The embodiments of this application have the following beneficial effects:

[0037] By acquiring the camera parameters of a first virtual camera bound to a physical camera, smoothing the camera parameters of the first virtual camera to obtain the target camera parameters, and adjusting the camera parameters of a second virtual camera based on the target camera parameters to obtain the adjusted second virtual camera, the image data captured by the physical camera is rendered using the adjusted second virtual camera to obtain an image of a virtual scene including objects. In the rendering process of the combined virtual and real scene, since the first virtual camera is bound to the physical camera in the real scene, the first virtual camera and the physical camera have the same camera parameters. Therefore, smoothing the camera parameters of the first virtual camera is equivalent to smoothing the camera parameters of the physical camera. Furthermore, by configuring a second virtual camera with a focus corresponding to the first virtual camera, the target camera parameters obtained from the smoothing process are transferred to the second virtual camera. In this way, the physical camera in the real scene does not need the assistance of a hardware stabilizer. Even if the physical camera shakes, the camera parameters of the second virtual camera can remain stable, effectively improving the stability performance of the virtual camera and saving the hardware cost of adding a hardware stabilizer to the physical camera, thereby significantly reducing hardware costs. Attached Figure Description

[0038] Figures 1A to 1B This is a schematic diagram illustrating the effects of the related technologies;

[0039] Figure 2 This is a schematic diagram of the architecture of the virtual camera parameter processing system provided in the embodiments of this application;

[0040] Figure 3 This is a schematic diagram of the structure of the parameter processing device for the virtual camera provided in the embodiments of this application;

[0041] Figure 4 This is a flowchart illustrating the parameter processing method for a virtual camera provided in an embodiment of this application;

[0042] Figure 5 This is a schematic diagram illustrating the principle of the parameter processing method for a virtual camera provided in the embodiments of this application;

[0043] Figures 6 to 7 This is a flowchart illustrating the parameter processing method for a virtual camera provided in an embodiment of this application;

[0044] Figure 8 This is a schematic diagram illustrating the principle of the parameter processing method for a virtual camera provided in the embodiments of this application;

[0045] Figures 9 to 11 This is a flowchart illustrating the parameter processing method for a virtual camera provided in an embodiment of this application;

[0046] Figures 12 to 14This is a schematic diagram illustrating the principle of the parameter processing method for a virtual camera provided in the embodiments of this application. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0049] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0051] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0052] 1) Virtual Scene: A virtual scene is a scene displayed (or provided) by an application when it runs on an electronic device. A virtual scene can be a simulation of the real world, a semi-simulated / semi-fictional virtual environment, or a purely fictional virtual environment. A virtual scene can be any of a two-dimensional, 2.5-dimensional, or three-dimensional virtual scene; this application does not limit the dimension of the virtual scene. For example, a virtual scene may include the sky, land, ocean, etc., and the land may include environmental elements such as deserts and cities. Users can control virtual objects to move within this virtual scene.

[0053] 2) Virtual Objects: These are interactive representations of people and objects within a virtual scene, or movable objects within the virtual scene. These movable objects can be virtual characters, virtual animals, anime characters, etc., such as people or animals displayed in a virtual scene. A virtual object can be a virtual avatar representing the user within the virtual scene. A virtual scene can include multiple virtual objects, each with its own shape and volume, occupying a portion of the space within the virtual scene.

[0054] 3) Virtual-Real Fusion: Virtual-real fusion technology is a technology that cleverly integrates virtual and real scenes. It is widely used in many technical fields such as multimedia video production, 3D modeling, online conferencing, real-time registration, intelligent interaction, and sensing. Virtual-real fusion is mainly reflected in the combination of virtual and real, real-time interaction, and 3D interaction matching. It is mainly achieved through display technology, interaction technology, sensing technology, and computer graphics and image technology.

[0055] 4) Virtual Camera: A virtual camera is a "camera" set up in computer animation software or virtual engine. In animation, the virtual camera's role in representing the viewpoint is equivalent to a traditional camera. While the subjects of a virtual camera and a physical camera are completely different, their functions are extremely similar. A physical camera captures real people or actual, constructed scenes, while a virtual camera captures models built in 3D software, allowing for limitless possibilities. Virtual cameras are presented as icons in the virtual engine and also have parameters such as lens, focal length, focus, aperture, and depth of field. They can perform camera actions such as "push, pull, pan, tilt, track, flick, rise, fall, and combined movements," achieving shooting effects that are difficult or impossible for a physical camera, such as passing through walls, keyholes, or objects. The parameters that need to be adjusted for a physical camera are distributed on the camera body and require manual operation. The camera parameters of a virtual camera are integrated into buttons or numerical input fields on the panel. The operator only needs to input parameters or drag the mouse. Sometimes a few keyframes can determine the motion path of the virtual camera. In actual shooting, physical cameras often need stabilizers or motion control systems, and even then, the image shake still exists.

[0056] 5) Virtual Engine: A virtual engine refers to a pre-written, editable computer virtual system or the core component of an interactive real-time graphics application. These systems provide virtual scene designers with various tools needed to create virtual scenes, with the aim of enabling designers to easily and quickly write programs. Virtual engines include rendering engines (including 2D and 3D rendering engines), physics engines, collision detection engines, sound engines, script engines, animation engines, artificial intelligence engines, network engines, and scene management engines, etc.

[0057] 6) Camera Parameters: Camera parameters include at least one of attitude angle parameters, field of view parameters, and position parameters. Attitude angle parameters are defined according to Euler's concept and are therefore also called Euler angles. Attitude angles include roll, pitch, and yaw angles. Different rotation sequences will result in different coordinate transformation matrices. Typically, the spatial rotation of the camera coordinate system relative to the object coordinate system is represented in the order of yaw, pitch, and roll. The value of the field of view parameter is positively correlated with the virtual camera's field of view. The position parameter characterizes the camera's three-dimensional position coordinates.

[0058] 7) Metaverse: This is a digital living space that uses technology to link and create, mapping the real world into an interactive virtual world, possessing a new social system. Metaverse is a new type of internet application and social form that integrates multiple new technologies, providing immersive experiences based on extended reality technology and generating a mirror image of the real world through digital twin technology. It builds an economic system through blockchain technology, closely integrating the virtual and real worlds in social systems, identity systems, and other systems, and allowing each user to produce and edit content.

[0059] 8) Responding to: used to indicate the conditions or states on which the operation is performed. When the conditions or states on which the operation is performed are met, one or more operations may be performed in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations are performed.

[0060] During the implementation of the embodiments of this application, the applicant discovered the following problems with the related technology:

[0061] See Figure 1A and Figure 1B , Figure 1A and Figure 1B This is a schematic diagram illustrating the effects of related technologies, which typically utilize hardware stabilizers (e.g., Figure 1A The released hardware stabilizer 40, Figure 1B The hardware stabilizer 41 shown achieves camera image stabilization. The hardware stabilizer can be achieved by adding additional camera accessories to realize functions such as automatic subject tracking. However, the cost of the hardware stabilizer will increase with the increase of the camera cost. Therefore, the cost of image stabilization is extremely high.

[0062] In related technologies, feature point detection algorithms can be used to detect feature points in each frame of the image. The motion trajectories of the same feature points are then smoothed to obtain a smoothed result. Based on this smoothed result, each frame can be enlarged, reduced, rotated, cropped, or distorted to obtain a stabilized video. However, these stabilization algorithms often consume significant computational resources and have poor stabilization effects.

[0063] The parameter processing method for the virtual camera provided in this application embodiment, compared with the aforementioned related technologies, can achieve the same high-quality stabilization effect as a hardware stabilizer, with extremely low complexity, real-time operation, and no consumption of computing resources. Furthermore, it requires no complex installation or debugging operations; it can be used immediately simply by connecting to the camera screen, and its ease of use and low barrier to entry make it a complete replacement for physical stabilizers, thereby saving costs.

[0064] This application provides a parameter processing method, apparatus, electronic device, computer-readable storage medium, and computer program product for a virtual camera. It can significantly reduce hardware costs while effectively improving the stability of the virtual camera and its anti-shake effect. The following describes an exemplary application of the parameter processing system for the virtual camera provided in this application.

[0065] See Figure 2 , Figure 2 This is a schematic diagram of the architecture of the virtual camera parameter processing system 100 provided in the embodiments of this application. The terminal (terminal 400 is shown as an example) connects to the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0066] Terminal 400 is used by a user to display images of a virtual scene on a graphical interface 410-1 (graphical interface 410-1 is shown as an example). For example, the graphical interface 410-1 displays images of a virtual scene from an online game app. Another example is that terminal 400 runs a web conferencing application app and displays images of a virtual scene on the graphical interface 410-1. Yet another example is that terminal 400 runs a video app and displays images of a virtual scene on the graphical interface 410-1. Terminal 400 and server 200 are interconnected via a wired or wireless network.

[0067] In some embodiments, server 200 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminal 400 can be a smartphone, tablet, laptop, desktop computer, smart speaker, smart TV, smartwatch, in-vehicle terminal, etc., but is not limited to these. The electronic device provided in this application embodiment can be implemented as a terminal or a server. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited in this application embodiment.

[0068] In some embodiments, the parameter processing method for the virtual camera provided in this application can also be applied to the display of images in virtual scenes related to augmented reality.

[0069] For example, a virtual scene image is displayed using a terminal 400 (such as a virtual wearable terminal). The terminal 400 displays icons of virtual cameras that the user can select in a graphical interface 410-1. In response to the user's selection operation of the icons displayed in the graphical interface 410-1, the selected virtual camera is determined. The terminal 400 sends the selected virtual camera to the server 200 through the network 300. The server 200 performs smoothing processing on the camera parameters of the selected virtual camera to obtain the target camera parameters. The virtual camera configured with the target camera parameters is used to render an image of the virtual scene. The server 200 sends the rendered image of the virtual scene to the terminal 400, and the terminal 400 displays the rendered image of the virtual scene in the graphical interface 410-1.

[0070] In some embodiments, the terminal 400 displays icons of virtual cameras that can be selected by the user in the graphical interface 410-1. In response to the user's selection operation of the icons displayed in the graphical interface 410-1, the terminal 400 obtains the camera parameters of the selected virtual camera, performs smoothing processing on the camera parameters of the selected virtual camera to obtain the target camera parameters, renders an image of the virtual scene through the virtual camera configured with the target camera parameters, and displays the rendered image of the virtual scene in the graphical interface 410-1.

[0071] In other embodiments, terminal 400 displays an icon of a virtual camera that the user can select in a graphical interface 410-1. In response to the user's selection operation of the icon displayed in the graphical interface 410-1, terminal 400 sends the camera parameters of the selected virtual camera to server 200 through network 300. Server 200 receives the camera parameters of the selected virtual camera and sends the smoothed target camera parameters to terminal 400. Terminal 400 receives the smoothed target camera parameters and renders an image of the virtual scene using the virtual camera configured with the target camera parameters. Terminal 400 then displays the image of the virtual scene in the graphical interface 410-1.

[0072] In other embodiments, the embodiments of this application can be implemented with the aid of cloud technology, which refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or local area network to realize the computation, storage, processing, and sharing of data.

[0073] Cloud technology is a general term encompassing network technology, information technology, integration technology, management platform technology, and application technology based on the cloud computing business model. It can form resource pools, allowing for on-demand use with flexibility and convenience. Cloud computing technology will become a crucial support. The backend services of cloud computing systems require substantial computing and storage resources.

[0074] See Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device 500 for smoothing processing provided in an embodiment of this application, wherein, Figure 3 The electronic device 500 shown can be Figure 2 Server 200 or terminal 400 in the middle, Figure 3 The illustrated electronic device 500 includes at least one processor 410, a memory 450, and at least one network interface 420. The various components in the electronic device 500 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 3 The general labeled all buses as Bus System 440.

[0075] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0076] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0077] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0078] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0079] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0080] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, such as Bluetooth, WiFi, and Universal Serial Bus.

[0081] In some embodiments, the parameter processing device for the virtual camera provided in this application can be implemented in software. Figure 3 A parameter processing device 455 for a virtual camera stored in memory 450 is shown. This device can be software in the form of programs and plugins, and includes the following software modules: acquisition module 4551, smoothing module 4552, configuration module 4553, and adjustment module 4554. These modules are logically linked and can therefore be arbitrarily combined or further separated according to their implemented functions. The functions of each module will be described below.

[0082] In other embodiments, the parameter processing device for the virtual camera provided in this application can be implemented in hardware. As an example, the parameter processing device for the virtual camera provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the parameter processing method for the virtual camera provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0083] In some embodiments, the terminal or server can implement the parameter processing method for the virtual camera provided in this application by running a computer program or computer-executable instructions. For example, the computer program can be a native program in the operating system (e.g., a dedicated image deblurring program) or a software module, such as a deblurring module that can be embedded in any program (e.g., an instant messaging client, a photo album program, an electronic map client, a navigation client); or it can be a native application (APP), i.e., a program that needs to be installed in the operating system to run. In summary, the above-mentioned computer program can be any form of application, module, or plugin.

[0084] The parameter processing method for a virtual camera provided in this application will be described in conjunction with exemplary applications and implementations of the server or terminal provided in the embodiments of this application.

[0085] See Figure 4 , Figure 4 This is a flowchart illustrating the parameter processing method for a virtual camera provided in an embodiment of this application, which will be combined with... Figure 4 Steps 101 to 104 are described below. The parameter processing method for the virtual camera provided in this application embodiment can be implemented by the server or the terminal alone, or by the server and the terminal working together. The following description will take the implementation by the server alone as an example.

[0086] In step 101, the camera parameters of the first virtual camera are obtained.

[0087] In some embodiments, camera parameters include at least one of attitude angle, field of view angle, and camera position. The first virtual camera is bound to a physical camera in the real scene. The physical camera is used to acquire image data of objects in the real scene to obtain image data of the objects.

[0088] In some embodiments, the binding relationship between the first virtual camera and the physical camera in the real scene means that the first virtual camera and the bound physical camera have the same camera parameters. That is, the camera parameters of the first virtual camera change with the camera parameters of the bound physical camera and are always the same as the camera parameters of the bound physical camera.

[0089] As an example, see Figure 5 , Figure 5 This is a schematic diagram illustrating the principle of the parameter processing method for a virtual camera provided in this application embodiment. The first virtual camera can be a virtual camera 2 that is bound to a physical camera 1 in a real scene.

[0090] In some embodiments, a virtual camera is used to acquire image data of virtual objects in a virtual scene to obtain image data of the virtual objects.

[0091] In some embodiments, the virtual camera includes: a virtual camera that is bound to a physical camera and a virtual camera that is not bound to a physical camera.

[0092] In some embodiments, camera parameters include at least one of attitude angle, field of view angle, and camera position. Attitude angle includes pitch angle, yaw angle, and roll angle. Different rotation sequences of the virtual camera will result in different coordinate transformation matrices. Typically, the spatial rotation of the virtual camera's coordinate system relative to the geographic coordinate system is represented in the order of yaw angle, pitch angle, and roll angle. The size of the field of view angle determines the virtual camera's field of view. The field of view angle is the angle formed by the two edges of the maximum range through which the image of the target object can pass through the lens, with the lens as the vertex.

[0093] In some embodiments, prior to step 101 above, a first virtual camera may be selected by: determining a plurality of configured virtual cameras in a virtual scene, wherein each configured virtual camera is bound to a different physical camera; and in response to a selection operation for the plurality of configured virtual cameras, determining the selected virtual camera as the first virtual camera.

[0094] In some embodiments, in the application scenario of virtual-real fusion, multiple real cameras are configured in the real scene, and different real cameras collect data from objects in different real scenes. Virtual cameras are configured in the virtual scene and bound to each real camera. In this way, virtual cameras act as a bridge between the real scene and the virtual scene, thus realizing virtual-real fusion.

[0095] In this way, by configuring virtual cameras bound to each real camera in the virtual scene and treating these bound virtual cameras as configured virtual cameras, in response to a selection operation for multiple configured virtual cameras, the selected virtual camera is determined as the first virtual camera. This allows the camera parameters of the virtual cameras that need to be smoothed to be determined based on the selection. When there are many physical cameras, the number of virtual cameras bound to them will also increase dramatically. By selecting the configured virtual cameras, the smoothing target is determined, so that the camera parameters of every virtual camera are not smoothed. Instead, virtual cameras are selected selectively, and the camera parameters of the selected virtual cameras are smoothed. This effectively reduces the number of virtual cameras that need to have their camera parameters smoothed, effectively reduces the amount of computation, and improves the smoothing efficiency.

[0096] In some embodiments, when the camera parameters include attitude angles, see [reference needed]. Figure 6 , Figure 6This is a schematic flowchart of the parameter processing method for a virtual camera provided in an embodiment of this application. Figure 6 Step 101 shown can be achieved by performing steps 1011 to 1013.

[0097] In step 1011, the target position of the object in the world coordinate system and the position of the first virtual camera are obtained.

[0098] In some embodiments, the world coordinate system refers to the system's absolute coordinate system. Before a user coordinate system is established, the coordinates of all points on the screen are determined by the origin of this coordinate system.

[0099] In some embodiments, the physical camera and the bound virtual camera are located in the same position in the world coordinate system. The physical camera acquires image data of objects in the real scene and sends the acquired image data to the virtual camera bound to the physical camera in the virtual engine.

[0100] In some embodiments, the number of objects in the real scene can be at least one. When there are multiple objects in the real scene, different objects can have their image data acquired through the same physical camera, or different objects can have their image data acquired through different physical cameras.

[0101] In some embodiments, when the number of objects within the field of view of the physical camera bound to the first virtual camera is one, that is, when the number of objects is one, obtaining the target position of the object in the world coordinate system in step 1011 above can be achieved as follows: obtaining the coordinates of multiple skeleton points of the object in the world coordinate system; performing a weighted summation of the multiple skeleton point coordinates to obtain the target position of the object in the world coordinate system.

[0102] In some embodiments, objects in a real-world scene include multiple skeletal points. Different skeletal points are located at different positions in the world coordinate system. Skeletal points are the skeletal support points of the object's external shape. Skeletal points are located at the turning points of the shape and play a crucial role in the modeling.

[0103] In some embodiments, the expression for the target position of an object in the world coordinate system can be:

[0104] X = a1X1 + a2X2 + a3X3 + ... + a n X n (1)

[0105] Where X represents the target position of the object in the world coordinate system, a1 represents the weight corresponding to the skeletal point 1 of the object, X1 represents the position of the skeletal point 1 of the object in the world coordinate system, a2 represents the weight corresponding to the skeletal point 2 of the object, X2 represents the position of the skeletal point 2 of the object in the world coordinate system, a3 represents the weight corresponding to the skeletal point 3 of the object, X3 represents the position of the skeletal point 3 of the object in the world coordinate system, and a n The weights X corresponding to the skeletal point n of the representation object n The position of the skeletal point n of the object in the world coordinate system represents the number of skeletal points of the object.

[0106] In some embodiments, the weights of the weighted summation corresponding to different bone points are different. The weights of the weighted summation corresponding to bone points can be specifically set according to the actual situation, and the sum of the weights of the weighted summation corresponding to each bone point can be equal to 1.

[0107] In this way, by weighted summing of the coordinates of multiple skeleton points, the target position of the object in the world coordinate system is obtained, thereby accurately determining the target position of the object in the world coordinate system. This facilitates the subsequent determination of the attitude angle of the first virtual camera based on the target position, effectively improving the accuracy of the determined attitude angle of the first virtual camera.

[0108] In some embodiments, when the number of objects within the field of view of the physical camera bound to the first virtual camera is at least two, that is, when the number of objects is at least two, obtaining the target position of the object in the world coordinate system in step 1011 above can be achieved in the following way: performing the following processing for each object: obtaining the coordinates of multiple skeleton points of the object in the world coordinate system; performing a weighted summation of the multiple skeleton point coordinates to obtain the position of the object in the world coordinate system; determining the target position based on the position of each object in the world coordinate system, wherein the distance between the target position and the position of each object in the world coordinate system is less than a distance threshold.

[0109] In some embodiments, the distance threshold can be set according to the specific application scenario.

[0110] Thus, when there are at least two objects, the position of each object in the world coordinate system can be determined separately, and the position where the distance between the positions of each object in the world coordinate system is less than a distance threshold can be determined as the target position. This accurately determines the target position, which facilitates the subsequent determination of the attitude angle of the first virtual camera based on the target position, effectively improving the accuracy of the determined attitude angle of the first virtual camera.

[0111] In step 1012, the target direction vector is determined based on the target position of the object in the world coordinate system and the position of the first virtual camera.

[0112] In some embodiments, the target direction vector is used to indicate the direction in which the first virtual camera points at the object in the world coordinate system.

[0113] In some embodiments, the target direction vector is a vector in the world coordinate system that starts at the position of the first virtual camera and ends at the target position of the object in the world coordinate system.

[0114] In step 1013, the attitude angle of the first virtual camera is determined based on the target direction vector.

[0115] In some embodiments, the expression for the target direction vector can be:

[0116] V = (x, y, z) (2)

[0117] Where V represents the target direction vector, x represents the horizontal component of the target direction vector, y represents the vertical component of the target direction vector, and z represents the vertical component of the target direction vector.

[0118] In some embodiments, the attitude angles include pitch angle and yaw angle; step 1013 can be implemented as follows: the cosine of the vertical component of the target direction vector is determined as the pitch angle of the first virtual camera, where the vertical component is the component of the target direction vector on the vertical axis of the world coordinate system; the ratio of the ordinate component to the abscissa component of the target direction vector is determined as a reference ratio, where the ordinate component is the component of the target direction vector on the abscissa of the world coordinate system, and the abscissa component is the component of the target direction vector on the abscissa of the world coordinate system; and the tangent of the reference ratio is determined as the yaw angle of the first virtual camera.

[0119] In some embodiments, the attitude angle of the first virtual camera includes a roll angle, the magnitude of which can be 0.

[0120] In some embodiments, the pitch angle of the first virtual camera can be expressed as:

[0121] Q=b1sin(z) (3)

[0122] Where b1 represents the pitch angle coefficient, z represents the vertical component of the target direction vector, and Q represents the pitch angle of the first virtual camera.

[0123] In some embodiments, the expression for the heading angle of the first virtual camera can be:

[0124] W = b²tan²(y, x) (4)

[0125] Where W represents the heading angle of the first virtual camera, b2 represents the heading angle coefficient, x represents the horizontal component of the target direction vector, and y represents the vertical component of the target direction vector.

[0126] In this way, by accurately determining the pitch and yaw angles of the first virtual camera based on the target direction vector, it is easier to perform smoothing processing based on the accurate pitch and yaw angles, thus effectively improving the accuracy of the pitch and yaw angles.

[0127] In step 102, the camera parameters of the first virtual camera are smoothed to obtain the target camera parameters.

[0128] In some embodiments, the camera parameters of the first virtual camera include at least one of attitude angle, field of view angle, and camera position, wherein the attitude angle includes pitch angle, yaw angle, and roll angle. Step 102 above can be implemented by performing at least one of the following processes: smoothing the attitude angle of the first virtual camera to obtain target attitude angle parameters; smoothing the field of view angle of the first virtual camera to obtain target field of view angle parameters; and smoothing the camera position of the first virtual camera to obtain target camera position parameters.

[0129] In some embodiments, the above-mentioned smoothing of the attitude angles of the first virtual camera to obtain the target attitude angle parameters can be achieved in the following ways: smoothing the pitch angle of the first virtual camera to obtain the target pitch angle parameters; smoothing the yaw angle of the first virtual camera to obtain the target yaw angle parameters; and smoothing the roll angle of the first virtual camera to obtain the target roll angle parameters.

[0130] In some embodiments, smoothing refers to a process that reduces the difference between two adjacent parameters to be smoothed at a smoothing moment in order to achieve parameter smoothing.

[0131] In some embodiments, the first virtual camera has n smoothing moments, where n is a positive integer greater than 1. See also Figure 6 , Figure 6 This is a schematic flowchart of the parameter processing method for a virtual camera provided in an embodiment of this application. Figure 6 Step 102 shown can be achieved by performing steps 1021 to 1022.

[0132] In step 1021, when the camera parameters of the first virtual camera include the camera parameters at the nth smoothing time, the smoothing index and the (n-1)th target camera parameters are obtained.

[0133] In some embodiments, the camera parameters at the nth smoothing time are the camera parameters before the smoothing process is applied to the camera parameters of the first virtual camera at the nth smoothing time.

[0134] In some embodiments, the smoothing index is used to indicate the degree of smoothing of the camera parameters. The (n-1)th target camera parameter is the target camera parameter obtained by smoothing the camera parameters of the first virtual camera at the (n-1)th smoothing time.

[0135] As an example, when n=2, the smoothing index and the first target camera parameters are obtained, where the first target camera parameters are the target camera parameters obtained by smoothing the camera parameters of the first virtual camera at the first smoothing time.

[0136] As an example, when n=3, the smoothing index and the second target camera parameters are obtained, where the second target camera parameters are the target camera parameters obtained by smoothing the camera parameters of the first virtual camera at the second smoothing time.

[0137] As an example, when n=3, the smoothing index and the second target camera parameters are obtained, where the second target camera parameters are the target camera parameters obtained by smoothing the camera parameters of the first virtual camera at the second smoothing time.

[0138] In step 1022, based on the smoothing index and the (n-1)th target camera parameters, the camera parameters at the nth smoothing time are smoothed to obtain the nth target camera parameters, and the nth target camera parameters are used as the target camera parameters.

[0139] In some embodiments, the smoothing index is between 0 and 1. The smoothing index is used to indicate the degree of smoothness of the camera parameters. The higher the smoothing index, the higher the degree of smoothness of the corresponding camera parameters. The specific setting of the smoothing index can be set according to different application scenarios.

[0140] In some embodiments, the smoothing index is between 0 and 1, and step 1022 above can be implemented as follows: the product of the camera parameter at the nth smoothing time and the smoothing index is determined as the first reference parameter; the product of the (n-1)th target camera parameter and the supplementary smoothing index is determined as the second reference parameter, wherein the supplementary smoothing index is the difference between the smoothing index and 1; the first reference parameter and the second reference parameter are added together to obtain the nth target camera parameter, and the nth target camera parameter is used as the target camera parameter.

[0141] In some embodiments, the expression for the first reference parameter can be:

[0142] T1=k1β n (5)

[0143] Where T1 represents the first reference parameter, k1 represents the smoothing exponent, and β1 represents the camera parameter at the nth smoothing time.

[0144] In some embodiments, the expression for the second reference parameter can be:

[0145] T2=(1-k1)α n-1 (6)

[0146] Where T2 represents the second reference parameter, k1 represents the smoothing exponent, and α n-1 The parameter represents the (n-1)th target camera parameter, and 1-k1 represents the supplementary smoothing exponent.

[0147] In some embodiments, the expression for the nth target camera parameter can be:

[0148] α n =T1+T2=k1β n +(1-k1)α n-1 (7)

[0149] Where, α n T1 represents the parameters of the nth target camera, T2 represents the first reference parameter, and T2 represents the second reference parameter.

[0150] Thus, by smoothing each of the n smoothing moments of the first virtual camera, the camera parameters of the first virtual camera at each smoothing moment are smoothed, so that the difference between the camera parameters of any two adjacent smoothing moments will not change abruptly, thereby achieving smoothing of the camera parameters.

[0151] In some embodiments, when the camera parameters include attitude angles, the attitude angles include pitch angle, yaw angle, and roll angle. Prior to step 102 above, the target angle can also be locked in the following manner: in response to a locking command for the target angle in the attitude angles, the target angle is locked; wherein the target angle includes at least one of pitch angle, yaw angle, and roll angle.

[0152] As an example, in response to a lock command for the pitch and yaw angles in the attitude angles, the pitch and yaw angles are locked.

[0153] As an example, in response to a lock command for the pitch angle in the attitude angle, the pitch angle is locked.

[0154] As an example, in response to a lock command for pitch and roll angles in the attitude angles, the pitch and roll angles are locked.

[0155] In some embodiments, the target angle is locked to prevent it from being smoothed.

[0156] In some embodiments, step 102 above can also be implemented by smoothing the portion of the attitude angle other than the target angle to obtain the target camera parameters.

[0157] In this way, by smoothing the part of the attitude angle other than the target angle, the gradual or partial smoothing of different attitude angles is achieved, thereby ensuring the controllability of attitude angle smoothing and meeting the smoothing requirements of various application scenarios. At the same time, progressive smoothing can be achieved, which reduces the error rate in the smoothing process and improves the accuracy of smoothing.

[0158] In some embodiments, when the camera parameters include attitude angles, see [link to relevant documentation]. Figure 7 , Figure 7 This is a schematic flowchart of the parameter processing method for a virtual camera provided in an embodiment of this application. Figure 7 Step 102 shown can be achieved by performing steps 1023 to 1025.

[0159] In step 1023, the data type of the attitude angle is obtained, wherein the data type includes quaternion type and Euler angle type.

[0160] In some embodiments, the attitude angle of the Euler angle type, the Euler angle is a set of three independent angular parameters used to determine the position of the virtual camera, consisting of nutation angle, precession angle and rotation angle.

[0161] In some embodiments, attitude angles are of the quaternion type, where a quaternion is a noncommutative extension of a complex number. If the set of quaternions is considered as a multidimensional real space, then a quaternion represents a four-dimensional space, as opposed to a two-dimensional space for a complex number.

[0162] As an example, the expression for a quaternion-type attitude angle can be:

[0163] ai+bj+ck+d (8)

[0164] In this quaternion, a, b, c, and d represent the elements of the quaternion, and i, j, and k represent the imaginary unit of the quaternion.

[0165] In step 1024, when the data type is quaternion, each element in the attitude angle of the quaternion type is smoothed to obtain the reference attitude angle of the quaternion type.

[0166] As an example, see Figure 8 , Figure 8 This is a schematic diagram illustrating the principle of the parameter processing method for a virtual camera provided in this application embodiment. When the data type is a quaternion, each (individual) element in the quaternion type attitude angle is smoothed to obtain the quaternion type reference attitude angle.

[0167] As an example, when the data type is quaternion, each element (a, b, c, d) in the quaternion type attitude angle ai+bj+ck+d is smoothed to obtain the quaternion type reference attitude angle.

[0168] In step 1025, the reference attitude angle of the quaternion type is converted to Euler angle type, and the Euler angle type reference attitude angle is determined as the target attitude angle.

[0169] As an example, see Figure 8 The reference attitude angle of the quaternion type is converted to Euler angle type, and the Euler angle type reference attitude angle is determined as the target attitude angle.

[0170] Thus, by using attitude angle-based data types, different types of attitude angles can be converted and smoothed, thereby effectively improving the universality of attitude angle smoothing.

[0171] In step 103, a second virtual camera is configured in the virtual scene, and the focus of the second virtual camera corresponds to the focus of the first virtual camera.

[0172] In some embodiments, the correspondence between the focus of the second virtual camera and the focus of the first virtual camera can mean that the distance between the focus of the second virtual camera and the focus of the first virtual camera is less than a focus distance threshold. That is, the distance between the focus of the second virtual camera and the focus of the first virtual camera can be 0. When the distance between the focus of the second virtual camera and the focus of the first virtual camera is 0, the positions of the focus of the second virtual camera and the focus of the first virtual camera coincide.

[0173] In some embodiments, the second virtual camera needs to maintain the same perspective relationship as the first virtual camera. This is achieved by setting the focus of the second virtual camera to be less than the focus distance threshold between it and the focus of the first virtual camera. In other words, the focus positions of the first and second virtual cameras are close to each other, thus ensuring that the second and first virtual cameras maintain the same perspective relationship.

[0174] Thus, by configuring a second virtual camera in the virtual scene whose focal distance from the first virtual camera is less than the focal distance threshold, the second virtual camera always follows the shooting direction of the first virtual camera. This allows the second virtual camera, which has the same shooting function and perspective relationship as the first virtual camera, to replace the first virtual camera in rendering images of the virtual scene including objects based on image data.

[0175] In step 104, the camera parameters of the second virtual camera are adjusted based on the target camera parameters to obtain the adjusted second virtual camera.

[0176] In some embodiments, the adjusted camera parameters of the second virtual camera are the same as the target camera parameters.

[0177] In some embodiments, the target camera parameters include at least one of the target attitude angle parameters, the target field of view parameters, and the target position parameters. Step 104 above can be implemented as follows: based on at least one of the target attitude angle parameters, the target field of view parameters, and the target position parameters, the camera parameters of the second virtual camera are adjusted to obtain the adjusted second virtual camera.

[0178] In some embodiments, step 104 can be implemented as follows: adjust the current camera parameters of the second virtual camera to the target camera parameters to obtain the adjusted second virtual camera.

[0179] In some embodiments, adjusting the current camera parameters of the second virtual camera to the target camera parameters to obtain the adjusted second virtual camera can be achieved as follows: based on the nth target camera parameters, the camera parameters of the second virtual camera at the (n-1)th smooth time are adjusted to the nth target camera parameters to obtain the adjusted second virtual camera.

[0180] In some embodiments, after adjusting to the target camera parameters, the following process may also be performed: in response to the adjustment instruction for the target camera parameters, the target camera parameters are adjusted to obtain the adjusted second virtual camera.

[0181] In some embodiments, the adjusted second virtual camera is used to render an image of a virtual scene including objects based on image data.

[0182] Thus, by acquiring the camera parameters of the first virtual camera bound to the physical camera, smoothing the camera parameters of the first virtual camera to obtain the target camera parameters, and adjusting the camera parameters of the second virtual camera based on the target camera parameters to obtain the adjusted second virtual camera, the image data captured by the physical camera is rendered through the adjusted second virtual camera to obtain an image of a virtual scene including objects. In the rendering process of the combined virtual and real scene, since the first virtual camera is bound to the physical camera in the real scene, the first virtual camera and the physical camera have the same camera parameters. Therefore, smoothing the camera parameters of the first virtual camera is equivalent to smoothing the camera parameters of the physical camera. Furthermore, by configuring the second virtual camera corresponding to the focus of the first virtual camera, the target camera parameters obtained from the smoothing process are transferred to the second virtual camera. In this way, the physical camera in the real scene does not need the assistance of a hardware stabilizer. Even if the physical camera shakes, the camera parameters of the second virtual camera can remain stable, effectively improving the stability performance of the virtual camera and saving the hardware cost of adding a hardware stabilizer to the physical camera, thereby significantly reducing hardware costs.

[0183] In some embodiments, see Figure 9 , Figure 9 This is a flowchart illustrating the parameter processing method for a virtual camera provided in an embodiment of this application, which will be combined with... Figure 9 Steps 201 to 204 are described below. The execution subject of steps 201 to 204 can be a server or a terminal, or it can be implemented by the server and the terminal in collaboration. The following description will take the execution subject as a server as an example.

[0184] In step 201, the camera parameters of the third virtual camera are obtained. The third virtual camera is not bound to the physical camera in the real scene.

[0185] In some embodiments, the camera parameters of the third virtual camera include at least one of attitude angle, field of view angle, and camera position. The first virtual camera is not bound to the physical camera in the real scene. The physical camera is used to acquire image data of objects in the real scene to obtain image data of the objects.

[0186] In some embodiments, a virtual camera is used to acquire image data of virtual objects in a virtual scene to obtain image data of the virtual objects.

[0187] In some embodiments, the camera parameters of the third virtual camera include at least one of attitude angle, field of view angle, and camera position. Attitude angle includes pitch angle, yaw angle, and roll angle. Different rotation sequences of the virtual camera will result in different coordinate transformation matrices. Typically, the spatial rotation of the virtual camera's coordinate system relative to the geographic coordinate system is represented in the order of yaw angle, pitch angle, and roll angle. The size of the field of view angle determines the virtual camera's field of view. The field of view angle is the angle formed by the two edges of the maximum range through which the image of the target object can pass through the lens, with the lens as the vertex.

[0188] In some embodiments, prior to step 201 above, a third virtual camera may be selected by: determining a plurality of configured third virtual cameras in the virtual scene; and in response to a selection operation for the plurality of configured third virtual cameras, determining the selected third virtual camera as the aforementioned third virtual camera.

[0189] In some embodiments, when the camera parameters of the third virtual camera include attitude angles, see [reference needed]. Figure 10 , Figure 10 This is a schematic flowchart of the parameter processing method for a virtual camera provided in an embodiment of this application. Figure 10 Step 201 shown can be achieved by performing steps 2011 to 2013.

[0190] In step 2011, the position parameters of the focus position of the third virtual camera are obtained.

[0191] In some embodiments, the position parameter of the focal position of the third virtual camera is the position coordinate of the third virtual camera in the world coordinate system.

[0192] In some embodiments, the world coordinate system refers to the system's absolute coordinate system. Before a user coordinate system is established, the coordinates of all points on the screen are determined by the origin of this coordinate system.

[0193] In step 2012, the orientation vector of the third virtual camera is determined based on the position parameters of the focal position and the position parameters of the third virtual camera.

[0194] In some embodiments, the orientation vector of the third virtual camera is used to indicate the direction in which the third virtual camera is pointing in the world coordinate system.

[0195] In some embodiments, the direction vector of the third virtual camera is a vector in the world coordinate system that starts at the position of the third virtual camera and ends at the focal position of the third virtual camera.

[0196] In step 2013, the attitude angle of the third virtual camera is determined based on the orientation vector of the third virtual camera.

[0197] In some embodiments, the expression for the direction vector of the third virtual camera can be:

[0198] T = (x1, y1, z1) (9)

[0199] Where T represents the direction vector of the third virtual camera, x1 represents the horizontal component of the direction vector of the third virtual camera, y1 represents the vertical component of the direction vector of the third virtual camera, and z1 represents the vertical component of the direction vector of the third virtual camera.

[0200] In some embodiments, the attitude angles include pitch angle and yaw angle; step 2013 can be implemented as follows: the cosine of the vertical component of the direction vector of the third virtual camera is determined as the pitch angle of the third virtual camera, where the vertical component is the component of the direction vector of the third virtual camera on the vertical axis of the world coordinate system; the ratio of the ordinate component and the abscissa component of the direction vector of the third virtual camera is determined as a reference ratio, where the ordinate component is the component of the target direction vector on the vertical axis of the world coordinate system, and the abscissa component is the component of the target direction vector on the abscissa of the world coordinate system; the tangent of the reference ratio is determined as the yaw angle of the third virtual camera.

[0201] In some embodiments, the attitude angle of the third virtual camera includes a roll angle, the size of which can be 0.

[0202] In some embodiments, the pitch angle of the third virtual camera can be expressed as:

[0203] Q2=b3sin(z1) (10)

[0204] Where b3 represents the pitch angle coefficient, z1 represents the vertical component of the direction vector of the third virtual camera, and Q2 represents the pitch angle of the third virtual camera.

[0205] In some embodiments, the expression for the heading angle of the third virtual camera can be:

[0206] W2=b4tan2(y1,x1) (11)

[0207] Where W2 represents the heading angle of the third virtual camera, b4 represents the heading angle coefficient, x1 represents the horizontal component of the direction vector of the third virtual camera, and y1 represents the vertical component of the direction vector of the third virtual camera.

[0208] Thus, by accurately determining the pitch and yaw angles of the third virtual camera based on its orientation vector, subsequent smoothing processing based on accurate pitch and yaw angles can be performed, effectively improving the accuracy of the pitch and yaw angles.

[0209] In some embodiments, when the camera parameters include the field of view, see [link to relevant documentation]. Figure 11 , Figure 11 This is a schematic flowchart of the parameter processing method for a virtual camera provided in an embodiment of this application. Figure 11 Step 201 shown can be achieved by performing steps 2014 to 2015.

[0210] In step 2014, when a virtual object exists within the field of view of the third virtual camera, the virtual distance is obtained. The virtual distance is the distance between the first position and the second position. The first position is the position of the third virtual camera in the world coordinate system, and the second position is the position of the virtual object in the world coordinate system.

[0211] In some embodiments, when a virtual object exists within the field of view of the third virtual camera, the virtual distance between the position of the third virtual camera in the world coordinate system and the position of the virtual object in the world coordinate system is obtained.

[0212] In step 2015, the field of view of the third virtual camera is determined based on the virtual distance, wherein the value of the virtual distance is proportional to the value of the field of view.

[0213] In some embodiments, the field of view of the third virtual camera is determined by the magnitude of the acquired virtual distance. When the virtual distance decreases, the magnitude of the field of view of the third virtual camera decreases accordingly, and when the virtual distance increases, the magnitude of the field of view of the third virtual camera increases accordingly.

[0214] In this way, by dynamically controlling the value of the virtual distance, the field of view of the third virtual camera is changed accordingly with the change of the virtual distance value, thereby realizing the dynamic adjustment of the field of view of the third virtual camera and realizing the automatic push, pull, pan and tilt of the field of view of the third virtual camera, effectively improving the camera movement effect of the third virtual camera.

[0215] In step 202, the camera parameters of the third virtual camera are smoothed to obtain the target camera parameters of the third virtual camera.

[0216] In some embodiments, the camera parameters of the third virtual camera include at least one of attitude angle, field of view angle, and camera position, wherein the attitude angle includes pitch angle, yaw angle, and roll angle. Step 202 above can be implemented by performing at least one of the following processes: smoothing the attitude angle of the third virtual camera to obtain target attitude angle parameters; smoothing the field of view angle of the third virtual camera to obtain target field of view angle parameters; and smoothing the camera position of the third virtual camera to obtain target camera position parameters.

[0217] In some embodiments, the above-mentioned smoothing of the attitude angles of the third virtual camera to obtain the target attitude angle parameters can be achieved in the following ways: smoothing the pitch angle of the third virtual camera to obtain the target pitch angle parameters; smoothing the yaw angle of the third virtual camera to obtain the target yaw angle parameters; and smoothing the roll angle of the third virtual camera to obtain the target roll angle parameters.

[0218] In some embodiments, smoothing refers to a process that reduces the difference between two adjacent parameters to be smoothed at a smoothing moment in order to achieve parameter smoothing.

[0219] In some embodiments, the third virtual camera has n smoothing moments, where n is a positive integer greater than 1. Step 202 above can be implemented as follows: when the camera parameters of the third virtual camera include the camera parameters at the nth smoothing moment, obtain the smoothing index and the (n-1)th target camera parameters. Based on the smoothing index and the (n-1)th target camera parameters, smooth the camera parameters at the nth smoothing moment to obtain the nth target camera parameters, and use the nth target camera parameters as the target camera parameters.

[0220] In some embodiments, the camera parameters at the nth smoothing time are the camera parameters before the smoothing process is applied to the camera parameters of the third virtual camera at the nth smoothing time.

[0221] In some embodiments, the smoothing index is used to indicate the degree of smoothing of the camera parameters. The (n-1)th target camera parameter is the target camera parameter obtained by smoothing the camera parameters of the third virtual camera at the (n-1)th smoothing time.

[0222] As an example, when n=2, the smoothing index and the first target camera parameters are obtained, where the first target camera parameters are the target camera parameters obtained by smoothing the camera parameters of the third virtual camera at the first smoothing time.

[0223] As an example, when n=3, the smoothing index and the second target camera parameters are obtained, where the second target camera parameters are the target camera parameters obtained by smoothing the camera parameters of the third virtual camera at the second smoothing time.

[0224] As an example, when n=3, the smoothing index and the second target camera parameters are obtained, where the second target camera parameters are the target camera parameters obtained by smoothing the camera parameters of the third virtual camera at the second smoothing time.

[0225] In some embodiments, the smoothing index is between 0 and 1. The smoothing index is used to indicate the degree of smoothness of the camera parameters. The higher the smoothing index, the higher the degree of smoothness of the corresponding camera parameters. The specific setting of the smoothing index can be set according to different application scenarios.

[0226] In some embodiments, the smoothing index is between 0 and 1. The above-mentioned smoothing process of the camera parameters at the nth smoothing time based on the smoothing index and the (n-1)th target camera parameters to obtain the nth target camera parameters, and the use of the nth target camera parameters as the target camera parameters, can be achieved as follows: the product of the camera parameters at the nth smoothing time and the smoothing index is determined as the third reference parameter; the product of the (n-1)th target camera parameters and the supplementary smoothing index is determined as the fourth reference parameter, wherein the supplementary smoothing index is the difference between the smoothing index and 1; the third reference parameter and the fourth reference parameter are added together to obtain the nth target camera parameters, and the nth target camera parameters are used as the target camera parameters.

[0227] In some embodiments, the expression for the third reference parameter can be:

[0228] T3=k3β n (12)

[0229] Where T3 represents the third reference parameter, k3 represents the smoothing exponent, and β n The camera parameters characterize the smoothing time n.

[0230] In some embodiments, the expression for the fourth reference parameter can be:

[0231] T4=(1-k3)α n-1 (13)

[0232] Where T2 represents the fourth reference parameter, k3 represents the smoothing exponent, and α n-1 The parameter represents the (n-1)th target camera parameter, and 1-k3 represents the supplementary smoothing index.

[0233] In some embodiments, the expression for the nth target camera parameter can be:

[0234] α n =T3+T4=k3β n +(1-k3)α n-1 (14)

[0235] Where, α n T3 represents the parameters of the nth target camera, T4 represents the third reference parameter, and T4 represents the fourth reference parameter.

[0236] Thus, by smoothing each of the n smoothing moments of the third virtual camera, the camera parameters of the third virtual camera at each smoothing moment are smoothed, so that the difference between the camera parameters of any two adjacent smoothing moments will not change abruptly, thereby achieving smoothing of the camera parameters.

[0237] In step 203, a fourth virtual camera is configured in the virtual scene, and the focus of the fourth virtual camera corresponds to the focus of the third virtual camera.

[0238] In some embodiments, the correspondence between the focus of the fourth virtual camera and the focus of the third virtual camera can mean that the distance between the focus of the fourth virtual camera and the focus of the third virtual camera is less than a focus distance threshold. That is, the distance between the focus of the fourth virtual camera and the focus of the third virtual camera can be 0. When the distance between the focus of the fourth virtual camera and the focus of the third virtual camera is 0, the positions of the focus of the fourth virtual camera and the focus of the third virtual camera coincide.

[0239] In some embodiments, the fourth virtual camera needs to maintain the same perspective relationship as the third virtual camera. This is achieved by setting the focus of the fourth virtual camera to be less than the focus distance threshold between it and the focus of the third virtual camera. In other words, the focus positions of the third and fourth virtual cameras are relatively close, thus ensuring that the fourth and third virtual cameras maintain the same perspective relationship.

[0240] Thus, by configuring a fourth virtual camera in the virtual scene whose focal distance from the third virtual camera is less than the focal distance threshold, the fourth virtual camera always follows the shooting direction of the third virtual camera. This allows the fourth virtual camera, which has the same shooting function and perspective relationship as the third virtual camera, to replace the third virtual camera in rendering images of the virtual scene, including objects, based on image data.

[0241] In step 204, the camera parameters of the fourth virtual camera are adjusted based on the target camera parameters of the third virtual camera to obtain the adjusted fourth virtual camera.

[0242] In some embodiments, the adjusted fourth virtual camera is used to render images of the virtual scene.

[0243] In some embodiments, the target camera parameters of the third virtual camera include at least one of the target attitude angle parameters, the target field of view parameters, and the target position parameters of the third virtual camera. Step 204 can be implemented as follows: based on at least one of the target attitude angle parameters, the target field of view parameters, and the target position parameters of the third virtual camera, the camera parameters of the fourth virtual camera are adjusted to obtain the adjusted fourth virtual camera.

[0244] In some embodiments, step 204 can be implemented as follows: adjust the current camera parameters of the fourth virtual camera to the target camera parameters to obtain the adjusted fourth virtual camera.

[0245] In some embodiments, adjusting the current camera parameters of the fourth virtual camera to the target camera parameters to obtain the adjusted fourth virtual camera can be achieved as follows: based on the nth target camera parameters, the camera parameters of the fourth virtual camera at the (n-1)th smooth time are adjusted to the nth target camera parameters to obtain the adjusted fourth virtual camera.

[0246] In some embodiments, after adjusting to the target camera parameters, the following process may also be performed: in response to the adjustment instruction for the target camera parameters, the target camera parameters are adjusted to obtain the adjusted fourth virtual camera.

[0247] In some embodiments, the image rendered by the fourth virtual camera can also produce a shaking effect by adding a shaking parameter to the camera parameters of the adjusted fourth virtual camera in response to a shaking parameter addition instruction for the adjusted fourth virtual camera, so that the image rendered by the fourth virtual camera with the added shaking parameter produces a shaking effect.

[0248] Thus, by adding a shake parameter to the adjusted camera parameters of the fourth virtual camera, the reverse application of image stabilization is achieved, which can make the image rendered by the fourth virtual camera produce a shaking effect, simulating the earthquake effect in a real scene, making the image rendered by the adjusted fourth virtual camera more realistic.

[0249] Thus, by acquiring the camera parameters of the first virtual camera bound to the physical camera, smoothing the camera parameters of the first virtual camera to obtain the target camera parameters, and adjusting the camera parameters of the second virtual camera based on the target camera parameters to obtain the adjusted second virtual camera, the image data captured by the physical camera is rendered through the adjusted second virtual camera to obtain an image of a virtual scene including objects. In the rendering process of the combined virtual and real scene, since the first virtual camera is bound to the physical camera in the real scene, the first virtual camera and the physical camera have the same camera parameters. Therefore, smoothing the camera parameters of the first virtual camera is equivalent to smoothing the camera parameters of the physical camera. Furthermore, by configuring the second virtual camera corresponding to the focus of the first virtual camera, the target camera parameters obtained from the smoothing process are transferred to the second virtual camera. In this way, the physical camera in the real scene does not need the assistance of a hardware stabilizer. Even if the physical camera shakes, the camera parameters of the second virtual camera can remain stable, effectively improving the stability performance of the virtual camera and saving the hardware cost of adding a hardware stabilizer to the physical camera, thereby significantly reducing hardware costs.

[0250] The following will describe an exemplary application of the embodiments of this application in a real-world virtual-real fusion application scenario.

[0251] In applications that blend virtual and real worlds, virtual objects and the real world can be displayed in the same line of sight. The parameter processing method for the virtual camera provided in this application can effectively improve the visual appeal of the blended images. For example, in live streaming and video production applications, it allows for free control of the physical camera for shooting while ensuring image stability, thus enhancing the visual experience. Tracking physical objects (i.e., objects in the real scene described above) can achieve smoother camera movements than manual camera movements, improving image quality. In applications such as education, online meetings, and interactive games, it enables high-quality camera movements; for example, a physical object (a person) can stand in the starry sky (a virtual scene).

[0252] This application embodiment smooths the camera parameters of the original virtual-real fusion camera (i.e., the first virtual camera described above) in the virtual scene to obtain smoothed camera parameters, and then configures the smoothed camera parameters to the smoothed virtual camera (i.e., the second virtual camera described above), thereby effectively avoiding the impact of the original virtual-real fusion camera's jitter on the image rendering effect, and thus effectively improving the image rendering effect.

[0253] In this embodiment of the application, a smooth virtual camera can be configured in a virtual scene in a virtual-real fusion application scenario. The focus coordinates of the smooth virtual camera coincide with the focus coordinates of the original virtual-real fusion camera in the virtual scene. In this way, the direction of the smooth virtual camera can be kept consistent with the direction of the original virtual fusion camera in the virtual scene, thereby realizing functions such as automatic tracking and image stabilization.

[0254] It's important to note that if the focus difference between the smooth virtual camera and the original fused virtual camera is only slight, the perspective relationship will still be basically correct. That is, the focus distance between the smooth virtual camera and the original fused virtual camera is less than a distance threshold, the size of which can be specifically set according to the application scenario.

[0255] In some embodiments, see Figure 12 , Figure 12 This is a schematic diagram illustrating the principle of the virtual camera parameter processing method provided in this application embodiment. The original virtual camera in the virtual scene can be a user-selected virtual camera. The original virtual camera can be a virtual camera bound to a physical camera, or it can be a virtual camera not bound to a physical camera. The camera parameters of the original virtual camera include attitude angle, field of view angle, and three-dimensional coordinates. Smoothing the camera parameters of the original virtual camera to obtain smoothed camera parameters can be achieved as follows: smoothing the attitude angle, field of view angle, and three-dimensional coordinates respectively to obtain smoothed attitude angle, smoothed field of view angle, and smoothed three-dimensional coordinates. Below, in conjunction with... Figure 12 The smoothing processes for attitude angle, field of view, and three-dimensional coordinates are explained separately.

[0256] First, the smoothing process for attitude angles will be explained; see [link to documentation]. Figure 12 and Figure 9 , Figure 12 This is a schematic diagram illustrating the principle of the parameter processing method for a virtual camera provided in this application embodiment. In response to a real-time switching operation for the original virtual camera, a virtual camera to be smoothed (i.e., the first virtual camera described above) is determined; the attitude angle smoothing module is invoked to smooth the camera parameters of the virtual camera to be smoothed, thereby obtaining the smoothed attitude angle.

[0257] In some embodiments, the original virtual camera in the virtual scene can be a virtual camera selected by the user. The original virtual camera can be a virtual camera that is bound to a physical camera, or it can be a virtual camera that is not bound to a physical camera.

[0258] As an example, see Figure 5 , Figure 5This is a schematic diagram illustrating the principle of the parameter processing method for a virtual camera provided in this application embodiment. The original virtual cameras in the virtual scene include virtual camera 2, virtual camera 3, and virtual camera 4. Virtual camera 2 can be a virtual camera bound to physical camera 1, where physical camera 1 is a camera that exists in the real world. Virtual camera 3 can be a virtual camera looking at virtual camera 2. Virtual camera 4 can be any virtual camera selected based on a selection operation. Virtual camera 3 and virtual camera 4 are not bound to physical camera 1.

[0259] See Figure 8 In response to the real-time switching operation of the original virtual camera, the virtual camera to be smoothed is determined. When the virtual camera to be smoothed is a virtual camera bound to a physical camera (i.e., the first virtual camera described above), the coordinates of each three-dimensional skeleton point 46 in the world coordinate system are determined based on the virtual fusion object module 42; the spatial position 44 of the virtual camera bound to the physical camera is determined based on the virtual-real fusion camera module 43; and the attitude angle 47 of the virtual camera bound to the physical camera is determined based on the coordinates of each three-dimensional skeleton point and the position of the virtual camera bound to the physical camera. The attitude angle smoothing module is called to smooth the attitude angle 47 of the virtual camera bound to the physical camera to obtain the smoothed attitude angle.

[0260] In some embodiments, the coordinates of each three-dimensional skeleton point in the world coordinate system (i.e., the skeleton point coordinates described above) can be obtained by weighted averaging, and the weighted average can be used as the three-dimensional coordinates of the object in the virtual world.

[0261] In some embodiments, the pose angle of the virtual camera bound to the physical camera can be determined based on the coordinates of each 3D skeleton point and the position of the virtual camera bound to the physical camera as follows: A target vector is determined starting from the position of the virtual camera bound to the physical camera and ending at the 3D coordinates of the object in the virtual world; the pose angle of the virtual camera bound to the physical camera is determined based on the target vector, wherein the expression for the target vector can be:

[0262] v=(x,y,z) (15)

[0263] Where v represents the target vector, x represents the component of the target vector along the horizontal axis of the world coordinate system, y represents the component of the target vector along the vertical axis of the world coordinate system, and z represents the component of the target vector along the vertical axis of the world coordinate system.

[0264] The attitude angles of the virtual camera bound to the physical camera include roll angle, pitch angle, and yaw angle, and the expressions for the attitude angles are as follows:

[0265] G1 = 0 (16)

[0266] G2=asin(z) (17)

[0267] G3=atan2(y,x) (18)

[0268] Wherein, G1 represents the roll angle, G2 represents the pitch angle, and G3 represents the heading angle.

[0269] In some embodiments, the attitude angle smoothing module can be smoothed by calling a filter to smooth the attitude angle and obtaining the smoothed attitude angle.

[0270] The expression for smoothing can be:

[0271] α n =(1-k1)α n-1 +k1β n (19)

[0272] Where, α n α represents the smoothed attitude angle at time n. n-1 β represents the smoothed attitude angle at time n-1. n The attitude angle before smoothing at time n is represented by k1, which represents the attitude angle smoothing exponent. k1∈[0,1] The attitude angle smoothing exponent represents the degree to which the attitude angle is smoothed.

[0273] See Figure 8 In response to the real-time switching operation of the original virtual camera, the virtual camera to be smoothed is determined. When the virtual camera to be smoothed is the virtual camera 49 whose attitude angle is set through mouse or touch operation 48 (i.e. the third virtual camera described above), the set attitude angle is obtained, the attitude angle smoothing module is called, and the obtained attitude angle is smoothed to obtain the smoothed attitude angle.

[0274] See Figure 8 In response to the real-time switching operation of the original virtual camera, the virtual camera to be smoothed is determined. When the virtual camera to be smoothed is a virtual camera looking in the direction of the virtual camera bound to the physical camera, the attitude angle 45 of the virtual camera looking in the direction of the virtual camera bound to the physical camera is obtained, and the attitude angle smoothing module is called to smooth the attitude angle 45 to obtain the smoothed attitude angle.

[0275] See Figure 8The following describes the specific processing steps of the attitude angle smoothing module. First, it determines whether the attitude angle is a quaternion type. If it is, it smooths each element of the quaternion type attitude angle to obtain a smoothed quaternion type attitude angle. Then, it converts the smoothed quaternion type attitude angle to Euler angles. If the attitude angle is an Euler angle type attitude angle, it smooths each element of the Euler angle type attitude angle to obtain a smoothed quaternion type attitude angle.

[0276] Next, the smoothing process for the field of view will be explained; see [link to documentation]. Figure 13 , Figure 13 This is a schematic diagram illustrating the principle of the parameter processing method for a virtual camera provided in this application embodiment. In response to a real-time switching operation on the original virtual camera, a virtual camera to be smoothed is determined. When the virtual camera to be smoothed is a virtual camera bound to a physical camera, the field of view of the virtual camera bound to the physical camera is determined based on the virtual-real fusion real-time calibration module 50. When the virtual camera to be smoothed is not a virtual camera bound to a physical camera, the field of view set through mouse or touch operation is obtained. The real-time field of view smoothing module is invoked to smooth the field of view, obtaining the smoothed attitude angle.

[0277] See Figure 13 The following describes the specific processing procedure for the real-time field-of-view smoothing module. A time-domain filter is invoked to smooth the input field of view, yielding the output of the real-time field-of-view smoothing module.

[0278] In some embodiments, the field of view can be smoothed in the real-time field of view smoothing module by calling a time-domain filter to smooth the input field of view, thereby obtaining the output of the real-time field of view smoothing module.

[0279] The expression for smoothing can be:

[0280] θ n =(1-k2)θ n-1 +k2ε n (20)

[0281] Where, θ n θ represents the smoothed field of view at time n. n-1 ε represents the smoothed field of view at time n-1. n k2 represents the field of view angle before smoothing at time n, and k2 represents the field of view angle smoothing index, k2∈[0,1]. The field of view angle smoothing index represents the degree to which the field of view angle is smoothed.

[0282] Finally, the smoothing process for three-dimensional coordinates is explained; see [link to documentation]. Figure 14 , Figure 14This is a schematic diagram illustrating the principle of the parameter processing method for a virtual camera provided in this application embodiment. In response to a real-time switching operation for the original virtual camera, a virtual camera to be smoothed is determined. When the virtual camera to be smoothed is a virtual camera bound to a physical camera, the three-dimensional coordinates of the virtual camera bound to the physical camera are determined based on the virtual-real fusion real-time calibration module 51. The three-dimensional coordinates are smoothed element by element to obtain the smoothed camera three-dimensional coordinates.

[0283] In some embodiments, the expression for element-wise smoothing can be:

[0284] τ n =(1-k3)τ n-1 +k3ω n (twenty one)

[0285] Where, τ n τ represents the smoothed 3D coordinates of the camera at time n. n-1 ω represents the smoothed 3D coordinates of the camera at time n-1. n The three-dimensional coordinates of the camera after smoothing at time n are represented by k3, which represents the three-dimensional coordinate smoothing exponent. k3∈[0,1] The three-dimensional coordinate smoothing exponent represents the degree to which the field of view is smoothed.

[0286] In some embodiments, the camera movement can be simulated by adding attitude angle jitter to the camera of the virtual stabilizer in post-processing (in reverse, utilizing the image stabilization feature), thereby improving the quality of the camera movement and simulating the camera shaking caused by an earthquake.

[0287] In some embodiments, the smoothing process is not limited to the first-order filter (Infinite ImpulseResponse, IIR) described above; other orders of filters, or filters such as Kalman filters, can also achieve smoothing.

[0288] In some embodiments, for virtual cameras that are not bound to a physical camera, the field of view can be automatically set according to the size of the virtual object projected onto the imaging surface of the virtual camera, thereby enabling automatic push-pull and panning of the virtual camera.

[0289] It is understood that in the embodiments of this application, data related to camera parameters and so on are involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0290] The following description continues to illustrate the exemplary structure of the virtual camera parameter processing device 455 provided in the embodiments of this application as a software module. In some embodiments, such as... Figure 3As shown, the software modules in the virtual camera parameter processing device 455 stored in the memory 450 may include: an acquisition module 4551, used to acquire camera parameters of a first virtual camera, the camera parameters including at least one of attitude angle parameters, field of view parameters, and position parameters, the first virtual camera having a binding relationship with a physical camera in the real scene, the physical camera being used to acquire image data of objects in the real scene to obtain image data of the objects; a smoothing module 4552, used to smooth the camera parameters of the first virtual camera to obtain target camera parameters; a configuration module 4553, used to configure a second virtual camera in the virtual scene, the focus of the second virtual camera corresponding to the focus of the first virtual camera; and an adjustment module 4554, used to adjust the camera parameters of the second virtual camera based on the target camera parameters to obtain an adjusted second virtual camera; wherein, the adjusted second virtual camera is used to render an image of a virtual scene including objects based on image data.

[0291] In some embodiments, when the camera parameters include attitude angles, the acquisition module 4551 is further configured to acquire the target position of the object in the world coordinate system and the position of the first virtual camera; determine the target direction vector based on the target position of the object in the world coordinate system and the position of the first virtual camera, the target direction vector being used to indicate the direction in which the first virtual camera points to the object in the world coordinate system; and determine the attitude angle of the first virtual camera based on the target direction vector.

[0292] In some embodiments, the attitude angle includes pitch angle and yaw angle; the acquisition module 4551 is further configured to determine the cosine value of the vertical component of the target direction vector as the pitch angle of the first virtual camera, where the vertical component is the component of the target direction vector on the vertical axis of the world coordinate system; determine the ratio of the ordinate component and the abscissa component of the target direction vector as a reference ratio, where the ordinate component is the component of the target direction vector on the vertical axis of the world coordinate system and the abscissa component is the component of the target direction vector on the abscissa of the world coordinate system; and determine the tangent value of the reference ratio as the yaw angle of the first virtual camera.

[0293] In some embodiments, when the number of objects is one, the above-mentioned acquisition module 4551 is further used to acquire the coordinates of multiple skeleton points of the object in the world coordinate system; and to perform a weighted summation of the multiple skeleton point coordinates to obtain the target position of the object in the world coordinate system.

[0294] In some embodiments, when the number of objects is at least two, the acquisition module 4551 is further configured to perform the following processing for each object: acquire the coordinates of multiple skeletal points of the object in the world coordinate system; perform a weighted summation of the multiple skeletal point coordinates to obtain the position of the object in the world coordinate system; and determine the target position based on the position of each object in the world coordinate system, wherein the distance between the target position and the position of each object in the world coordinate system is less than a distance threshold.

[0295] In some embodiments, the first virtual camera has n smoothing moments, where n is a positive integer greater than 1; the smoothing module 4552 is further configured to obtain a smoothing index and a (n-1)th target camera parameter when the camera parameters of the first virtual camera include the camera parameters at the nth smoothing moment; wherein, the smoothing index is used to indicate the smoothness of the camera parameters; the (n-1)th target camera parameter is the target camera parameter obtained by smoothing the camera parameters of the first virtual camera at the (n-1)th smoothing moment; based on the smoothing index and the (n-1)th target camera parameter, the camera parameters at the nth smoothing moment are smoothed to obtain the nth target camera parameter, and the nth target camera parameter is used as the target camera parameter.

[0296] In some embodiments, the smoothing index is between 0 and 1. The smoothing module 4552 is further configured to determine the product of the camera parameter at the nth smoothing time and the smoothing index as a first reference parameter; determine the product of the (n-1)th target camera parameter and the supplementary smoothing index as a second reference parameter, wherein the supplementary smoothing index is the difference between the smoothing index and 1; add the first reference parameter and the second reference parameter to obtain the nth target camera parameter, and use the nth target camera parameter as the target camera parameter.

[0297] In some embodiments, when the camera parameters include attitude angles, the attitude angles include pitch angle, yaw angle, and roll angle; the above-mentioned virtual camera parameter processing device further includes: a locking module, configured to lock the target angle in response to a locking command for the target angle in the attitude angles; wherein the target angle includes at least one of pitch angle, yaw angle, and roll angle; the above-mentioned smoothing module is further configured to smooth the portion of the attitude angle other than the target angle to obtain the target camera parameters.

[0298] In some embodiments, the adjustment module 4553 is further configured to adjust the current camera parameters of the second virtual camera to the target camera parameters to obtain the adjusted second virtual camera; the parameter processing device of the virtual camera further includes: an instruction adjustment module, configured to adjust the target camera parameters in response to an adjustment instruction for the target camera parameters to obtain the adjusted second virtual camera.

[0299] In some embodiments, when the camera parameters include attitude angles, the smoothing module 4552 is further configured to obtain the data type of the attitude angles, wherein the data type includes quaternion type and Euler angle type; when the data type is quaternion type, each element in the attitude angle of the quaternion type is smoothed to obtain a reference attitude angle of the quaternion type; the reference attitude angle of the quaternion type is converted to Euler angle type to obtain a reference attitude angle of Euler angle type, and the reference attitude angle of Euler angle type is determined as the target attitude angle.

[0300] In some embodiments, the parameter processing device for the virtual camera further includes: a selection module, configured to determine a plurality of configured virtual cameras in a virtual scene, wherein each configured virtual camera is bound to a different physical camera; and in response to a selection operation for the plurality of configured virtual cameras, to determine the selected virtual camera as the first virtual camera.

[0301] In some embodiments, the parameter processing device for the virtual camera further includes: a second acquisition module for acquiring camera parameters of a third virtual camera, wherein the third virtual camera is not bound to a physical camera in the real scene; a second smoothing module for smoothing the camera parameters of the third virtual camera to obtain target camera parameters of the third virtual camera; a second configuration module for configuring a fourth virtual camera in the virtual scene, wherein the focus of the fourth virtual camera corresponds to the focus of the third virtual camera; and a second adjustment module for adjusting the camera parameters of the fourth virtual camera based on the target camera parameters of the third virtual camera to obtain an adjusted fourth virtual camera; wherein the adjusted fourth virtual camera is used to render an image of the virtual scene.

[0302] In some embodiments, when the camera parameters of the third virtual camera include attitude angles, the second acquisition module is further configured to acquire the position parameters of the focal position of the third virtual camera; determine the direction vector of the third virtual camera based on the position parameters of the focal position and the position parameters of the third virtual camera; and determine the attitude angle of the third virtual camera based on the direction vector of the third virtual camera.

[0303] In some embodiments, when the camera parameters include the field of view, the second acquisition module is further configured to acquire a virtual distance when a virtual object exists within the field of view of the third virtual camera. The virtual distance is the distance between the first position and the second position, where the first position is the position of the third virtual camera in the world coordinate system and the second position is the position of the virtual object in the world coordinate system. Based on the virtual distance, the field of view of the third virtual camera is determined, wherein the value of the virtual distance is proportional to the value of the field of view.

[0304] In some embodiments, the parameter processing apparatus for the virtual camera further includes: a shaking module, configured to add shaking parameters to the camera parameters of the adjusted fourth virtual camera in response to a shaking parameter addition instruction for the adjusted fourth virtual camera, so that the image rendered by the fourth virtual camera with the added shaking parameters produces a shaking effect.

[0305] This application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the virtual camera parameter processing method described above in this application.

[0306] This application provides a computer-readable storage medium storing computer-executable instructions. When these computer-executable instructions are executed by a processor, they cause the processor to execute the parameter processing method for a virtual camera provided in this application. For example, ... Figure 4 The parameter processing method for the virtual camera is shown.

[0307] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of electronic devices including one or any combination of the above-mentioned memories.

[0308] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0309] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0310] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0311] In summary, the embodiments of this application have the following beneficial effects:

[0312] (1) By acquiring the camera parameters of the first virtual camera bound to the physical camera, smoothing the camera parameters of the first virtual camera is performed to obtain the target camera parameters. Based on the target camera parameters, the camera parameters of the second virtual camera are adjusted to obtain the adjusted second virtual camera. The image data collected by the physical camera is rendered through the adjusted second virtual camera to obtain an image of a virtual scene including objects. In the rendering process of the virtual and real combined scene, since the first virtual camera is bound to the physical camera in the real scene, the first virtual camera and the physical camera have the same camera parameters. Therefore, smoothing the camera parameters of the first virtual camera is equivalent to smoothing the camera parameters of the physical camera. Furthermore, by configuring the second virtual camera corresponding to the focus of the first virtual camera, the target camera parameters obtained by the smoothing process are transferred to the second virtual camera. In this way, the physical camera in the real scene does not need the assistance of a hardware stabilizer. Even if the physical camera shakes, the camera parameters of the second virtual camera can remain stable, which effectively improves the stability performance of the virtual camera and saves the hardware cost of adding a hardware stabilizer to the physical camera, thereby significantly reducing the hardware cost.

[0313] (2) By configuring virtual cameras bound to each real camera in the virtual scene and treating the bound virtual cameras as configured virtual cameras; in response to the selection operation for multiple configured virtual cameras, the selected virtual camera is determined as the first virtual camera, thereby realizing the determination of the camera parameters of the virtual camera that needs to be smoothed according to the selection. When there are many physical cameras, the number of virtual cameras bound to the physical cameras will also increase sharply. By selecting the configured virtual cameras, the smoothing object is determined, so that the camera parameters of each virtual camera are not smoothed, but the virtual cameras are selected selectively, and the camera parameters of the selected virtual cameras are smoothed, thereby effectively reducing the number of virtual cameras that need to be smoothed, effectively reducing the amount of computation, and improving the smoothing efficiency.

[0314] (3) By weighted summation of the coordinates of multiple skeleton points, the target position of the object in the world coordinate system is obtained, thereby accurately determining the target position of the object in the world coordinate system, which facilitates the subsequent determination of the attitude angle of the first virtual camera based on the target position, effectively improving the accuracy of the determined attitude angle of the first virtual camera.

[0315] (4) When there are at least two objects, the position of each object in the world coordinate system can be determined separately, and the position where the distance between the positions of each object in the world coordinate system is less than the distance threshold can be determined as the target position. This accurately determines the target position, which facilitates the subsequent determination of the attitude angle of the first virtual camera based on the target position, and effectively improves the accuracy of the determined attitude angle of the first virtual camera.

[0316] (5) By accurately determining the pitch and yaw angles of the first virtual camera based on the target direction vector, it is easier to perform smoothing processing based on the accurate pitch and yaw angles, which effectively improves the accuracy of the pitch and yaw angles.

[0317] (6) By smoothing each of the n smoothing moments of the first virtual camera, the camera parameters of the first virtual camera at each smoothing moment are smoothed, so that the difference between the camera parameters of any two adjacent smoothing moments will not change abruptly, thus achieving the smoothing of the camera parameters.

[0318] (7) By smoothing the part of the attitude angle other than the target angle, the gradual smoothing or partial smoothing of different attitude angles is achieved, thereby ensuring the controllability of attitude angle smoothing and meeting the smoothing requirements of various application scenarios. At the same time, progressive smoothing can be achieved, which reduces the error rate in the smoothing process and improves the accuracy of smoothing.

[0319] (8) By using data types based on attitude angles, different types of attitude angles are converted and smoothed, thereby effectively improving the universality of attitude angle smoothing.

[0320] (9) By configuring a second virtual camera in the virtual scene with a focal distance less than the focal distance threshold between it and the first virtual camera, the second virtual camera always follows the shooting direction of the first virtual camera to shoot. This enables the configuration of a second virtual camera in the virtual scene with the same shooting function and perspective relationship as the first virtual camera, which replaces the first virtual camera in rendering images of the virtual scene including objects based on image data.

[0321] (10) By accurately determining the pitch and yaw angles of the third virtual camera based on the direction vector of the third virtual camera, it is easier to perform smoothing processing based on the accurate pitch and yaw angles, thus effectively improving the accuracy of the pitch and yaw angles.

[0322] (11) By dynamically controlling the field of view of the third virtual camera through the value of the virtual distance, the field of view of the third virtual camera changes accordingly with the value of the virtual distance, thereby realizing the dynamic adjustment of the field of view of the third virtual camera and realizing the automatic push, pull, pan and tilt of the field of view of the third virtual camera, effectively improving the camera movement effect of the third virtual camera.

[0323] (12) By adding a shaking parameter to the camera parameters of the adjusted fourth virtual camera, the reverse application of the anti-shake processing is realized, which can make the image rendered by the fourth virtual camera produce a shaking effect, mimicking the earthquake effect in the real scene, making the image rendered by the adjusted fourth virtual camera more realistic.

[0324] (13) By configuring a fourth virtual camera in the virtual scene whose distance from the focus of the third virtual camera is less than the focus distance threshold, the fourth virtual camera always follows the shooting direction of the third virtual camera to shoot. This enables the configuration of a fourth virtual camera in the virtual scene with the same shooting function and perspective relationship as the third virtual camera, which replaces the third virtual camera to render the image of the virtual scene containing objects based on the image data.

[0325] (14) By adding attitude angle jitter to the camera of the virtual stabilizer in post-production (using the anti-shake feature in reverse), the effect of camera shaking caused by an earthquake and handheld camera shaking is simulated to improve the quality of camera movement.

[0326] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A parameter processing method for a virtual camera, characterized in that, The method includes: Obtain the camera parameters of the first virtual camera, wherein the camera parameters include at least one of attitude angle, field of view angle and camera position; The first virtual camera is bound to a physical camera in the real scene. The physical camera is used to collect image data of objects in the real scene to obtain image data of the objects. The camera parameters of the first virtual camera are smoothed to obtain the target camera parameters; Configure a second virtual camera in the virtual scene, the focus of the second virtual camera corresponding to the focus of the first virtual camera; Based on the target camera parameters, the camera parameters of the second virtual camera are adjusted to obtain the adjusted second virtual camera; The adjusted second virtual camera is used to render an image of a virtual scene including the object based on the image data.

2. The method according to claim 1, characterized in that, When the camera parameters include the attitude angle, obtaining the camera parameters of the first virtual camera includes: Obtain the target position of the object in the world coordinate system and the position of the first virtual camera; Based on the target position of the object in the world coordinate system and the position of the first virtual camera, a target direction vector is determined. The target direction vector is used to indicate the direction in which the first virtual camera points to the object in the world coordinate system. Based on the target direction vector, the attitude angle of the first virtual camera is determined.

3. The method according to claim 2, characterized in that, The attitude angles include pitch angle and yaw angle; determining the attitude angles of the first virtual camera based on the target direction vector includes: The cosine value of the vertical component of the target direction vector is determined as the pitch angle of the first virtual camera, where the vertical component is the component of the target direction vector on the vertical axis of the world coordinate system. The ratio of the vertical component to the horizontal component of the target direction vector is determined as a reference ratio. The vertical component is the component of the target direction vector on the vertical axis of the world coordinate system, and the horizontal component is the component of the target direction vector on the horizontal axis of the world coordinate system. The tangent of the reference ratio is determined as the heading angle of the first virtual camera.

4. The method according to claim 2, characterized in that, When the number of objects is one, obtaining the target position of the object in the world coordinate system includes: Obtain the coordinates of multiple skeletal points of the object in the world coordinate system; The target position of the object in the world coordinate system is obtained by weighted summation of the coordinates of the multiple skeleton points.

5. The method according to claim 2, characterized in that, When the number of objects is at least two, obtaining the target position of the objects in the world coordinate system includes: For each of the objects, the following processing is performed: obtain the coordinates of multiple skeletal points of the object in the world coordinate system; perform a weighted sum of the multiple skeletal point coordinates to obtain the position of the object in the world coordinate system; The target location is determined based on the position of each of the objects in the world coordinate system, wherein the distance between the target location and the position of each of the objects in the world coordinate system is less than a distance threshold.

6. The method according to claim 1, characterized in that, The first virtual camera has n smooth moments, where n is a positive integer greater than 1; The step of smoothing the camera parameters of the first virtual camera to obtain the target camera parameters includes: When the camera parameters of the first virtual camera include the camera parameters at the nth smoothing time, obtain the smoothing index and the (n-1)th target camera parameters; The smoothing index is used to indicate the degree of smoothness of the camera parameters; The (n-1)th target camera parameters are the target camera parameters obtained by smoothing the camera parameters of the first virtual camera at the (n-1)th smoothing time. Based on the smoothing index and the (n-1)th target camera parameters, the camera parameters at the nth smoothing time are smoothed to obtain the nth target camera parameters, and the nth target camera parameters are used as the target camera parameters.

7. The method according to claim 6, characterized in that, The smoothing index is between 0 and 1. The process of smoothing the camera parameters at the nth smoothing time based on the smoothing index and the (n-1)th target camera parameters to obtain the nth target camera parameters, and using the nth target camera parameters as the target camera parameters, includes: The product of the camera parameters at the nth smoothing time and the smoothing exponent is determined as the first reference parameter; The product of the (n-1)th target camera parameter and the supplementary smoothing index is determined as the second reference parameter, wherein the supplementary smoothing index is the difference between the smoothing index and 1. The first reference parameter and the second reference parameter are added together to obtain the nth target camera parameter, and the nth target camera parameter is used as the target camera parameter.

8. The method according to claim 1, characterized in that, When the camera parameters include the attitude angles, the attitude angles include pitch angle, yaw angle, and roll angle; Before smoothing the camera parameters of the first virtual camera to obtain the target camera parameters, the method further includes: In response to a lock command for a target angle among the attitude angles, the target angle is locked; wherein the target angle includes at least one of the pitch angle, yaw angle, and roll angle; The step of smoothing the camera parameters of the first virtual camera to obtain the target camera parameters includes: The portion of the attitude angle other than the target angle is smoothed to obtain the target camera parameters.

9. The method according to claim 1, characterized in that, The step of adjusting the camera parameters of the second virtual camera based on the target camera parameters to obtain the adjusted second virtual camera includes: The current camera parameters of the second virtual camera are adjusted to the target camera parameters to obtain the adjusted second virtual camera; After adjusting to the target camera parameters, the method further includes: In response to the adjustment command for the target camera parameters, the target camera parameters are adjusted to obtain the adjusted second virtual camera.

10. The method according to claim 1, characterized in that, When the camera parameters include the attitude angle, the smoothing process of the camera parameters of the first virtual camera to obtain the target camera parameters includes: Obtain the data type of the attitude angle, wherein the data type includes quaternion type and Euler angle type; When the data type is the quaternion type, each element in the attitude angle of the quaternion type is smoothed to obtain the reference attitude angle of the quaternion type; The reference attitude angle of the quaternion type is converted to a reference attitude angle of Euler angle type, and the reference attitude angle of Euler angle type is determined as the target attitude angle.

11. The method according to claim 1, characterized in that, Before obtaining the camera parameters of the first virtual camera, the method further includes: Multiple configured virtual cameras are identified in the virtual scene, wherein each configured virtual camera is bound to a different physical camera. In response to a selection operation for the plurality of configured virtual cameras, the selected virtual camera is identified as the first virtual camera.

12. The method according to claim 1, characterized in that, The method further includes: Obtain the camera parameters of a third virtual camera, which is not bound to a physical camera in the real scene; The camera parameters of the third virtual camera are smoothed to obtain the target camera parameters of the third virtual camera; A fourth virtual camera is configured in the virtual scene, and the focus of the fourth virtual camera corresponds to the focus of the third virtual camera; Based on the target camera parameters of the third virtual camera, the camera parameters of the fourth virtual camera are adjusted to obtain the adjusted fourth virtual camera; The adjusted fourth virtual camera is used to render images of the virtual scene.

13. The method according to claim 12, characterized in that, When the camera parameters of the third virtual camera include the attitude angle, obtaining the camera parameters of the third virtual camera includes: Obtain the position parameters of the focus position of the third virtual camera; Based on the position parameters of the focal position and the position parameters of the third virtual camera, the orientation vector of the third virtual camera is determined; The attitude angle of the third virtual camera is determined based on the orientation vector of the third virtual camera.

14. The method according to claim 12, characterized in that, When the camera parameters include the field of view, obtaining the camera parameters of the third virtual camera includes: When a virtual object exists within the field of view of the third virtual camera, a virtual distance is obtained. The virtual distance is the distance between a first position and a second position. The first position is the position of the third virtual camera in the world coordinate system, and the second position is the position of the virtual object in the world coordinate system. Based on the virtual distance, the field of view of the third virtual camera is determined, wherein the value of the virtual distance is proportional to the value of the field of view.

15. The method according to claim 12, characterized in that, After adjusting the camera parameters of the fourth virtual camera to obtain the adjusted fourth virtual camera, the method further includes: In response to the instruction to add a jitter parameter to the adjusted fourth virtual camera, the jitter parameter is added to the camera parameters of the adjusted fourth virtual camera so that the image rendered by the fourth virtual camera with the added jitter parameter produces a shaky effect.

16. A parameter processing device for a virtual camera, characterized in that, The device includes: The acquisition module is used to acquire camera parameters of a first virtual camera, the camera parameters including at least one of attitude angle, field of view angle and camera position; wherein, the first virtual camera is bound to a physical camera in the real scene, the physical camera is used to acquire image data of objects in the real scene to obtain image data of the objects; A smoothing module is used to smooth the camera parameters of the first virtual camera to obtain the target camera parameters; A configuration module is used to configure a second virtual camera in a virtual scene, wherein the focus of the second virtual camera corresponds to the focus of the first virtual camera; An adjustment module is used to adjust the camera parameters of the second virtual camera based on the target camera parameters to obtain an adjusted second virtual camera; wherein, the adjusted second virtual camera is used to render an image of a virtual scene including the object based on the image data.

17. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the parameter processing method of the virtual camera according to any one of claims 1 to 15.

18. A computer-readable storage medium storing computer-executable instructions, characterized in that, When the computer-executable instructions are executed by the processor, they implement the parameter processing method of the virtual camera according to any one of claims 1 to 15.

19. A computer program product comprising a computer program or computer-executable instructions, characterized in that, When the computer program or computer-executable instructions are executed by a processor, the parameter processing method of the virtual camera according to any one of claims 1 to 15 is implemented.

Images