Virtual camera-based image acquisition method and related apparatus

Abstract

Embodiments of the present application disclose a camera-based image acquisition method and a related apparatus. The method comprises: constructing a virtual world coordinate system in a three-dimensional virtual photography environment, and importing a three-dimensional model of an object to be photographed; determining target model position information and target model orientation information of the three-dimensional model in the virtual world coordinate system; determining target camera position information and target camera orientation information of each virtual camera in the virtual world coordinate system by means of a target arrangement of the plurality of virtual cameras in the virtual photography environment; and for each virtual camera, capturing a photographed image of the object to be photographed at the viewing angle of said virtual camera by means of the target model position information, the target model orientation information, the target camera position information, and the target camera orientation information. Synchronous acquisition of a plurality of photographed images at a plurality of virtual camera viewing angles can be quickly and accurately carried out without needing to perform steps such as the purchase of real cameras, the physical construction of a hardware-based camera acquisition system, and time synchronization of the real cameras.

Description

Image acquisition method and related device based on virtual camera

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on July 22, 2022, with application number 202210870825.3 and application name “A camera-based image acquisition method and related device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of image acquisition, and in particular to image acquisition based on a virtual camera. Background Art

[0003] At present, with the rapid development of three-dimensional vision technology, in three-dimensional shooting scenarios such as video conferencing, in order to achieve high-precision three-dimensional reconstruction and rendering of the object to be photographed, so as to enhance the realism and immersion of shooting scenes such as video conferencing, it is necessary to use three-dimensional vision technology to collect images shot from multiple camera perspectives, providing a basis for subsequent high-precision three-dimensional reconstruction and rendering.

[0004] In related technologies, collecting images captured from multiple camera perspectives using three-dimensional vision technology means: first, selecting multiple physical cameras, then building multiple physical cameras on-site to form a hardware camera acquisition system, and finally, time-synchronizing the multiple physical cameras so that the multiple physical cameras can synchronously collect images captured from multiple physical camera perspectives.

[0005] However, the various implementation steps in the above method are relatively cumbersome and complicated. Due to the influence of factors such as the environment and hardware, it is difficult to ensure the operational accuracy of each implementation step. It not only consumes a lot of time and energy, but also incurs high hardware costs, greatly reducing the acquisition efficiency and acquisition effect of images taken from multiple camera perspectives.

[0006] Summary of the Invention

[0007] In order to solve the above technical problems, the present application provides an image acquisition method and related devices based on virtual cameras, which can quickly and accurately synchronously acquire images captured from the perspectives of multiple virtual cameras without the need to purchase multiple physical cameras, build a hardware camera acquisition system on site, and synchronize the time of the physical cameras. This saves a lot of time and energy, reduces hardware costs, and improves the acquisition efficiency and acquisition effect of images captured from the perspectives of multiple cameras.

[0008] The embodiments of this application disclose the following technical solutions:

[0009] In one aspect, the present application provides a camera-based image acquisition method, the method comprising:

[0010] In a three-dimensional virtual shooting scene having a virtual world coordinate system, a three-dimensional model of the object to be shot is imported;

[0011] Determining model position information and model posture information of the three-dimensional model in the virtual world coordinate system;

[0012] Determining camera position information and camera posture information of each virtual camera in the virtual world coordinate system according to a layout of multiple virtual cameras in the three-dimensional virtual shooting scene;

[0013] For each virtual camera, an image of the object to be photographed from the perspective of the virtual camera is collected based on the model position information, the model posture information, the camera position information, and the camera posture information.

[0014] On the other hand, the present application provides a camera-based image acquisition device, the device comprising: an import unit, a determination unit, and an acquisition unit;

[0015] The importing unit is used to import the three-dimensional model of the object to be photographed into the three-dimensional virtual shooting scene in which the virtual world coordinate system is constructed;

[0016] The determining unit is configured to determine the model position information and the model posture information of the three-dimensional model in the virtual world coordinate system;

[0017] The determining unit is further configured to determine camera position information and camera posture information of each virtual camera in the virtual world coordinate system according to a layout of multiple virtual cameras in the three-dimensional virtual shooting scene;

[0018] The acquisition unit is configured to acquire, for each virtual camera, an image of the object to be photographed from the perspective of the virtual camera according to the model position information, the model posture information, the camera position information, and the camera posture information.

[0019] In another aspect, the present application provides a camera-based image acquisition device, the device comprising a processor and a memory:

[0020] The memory is used to store a computer program and transmit the computer program to the processor;

[0021] The processor is configured to execute the camera-based image acquisition method described in the above aspect according to the computer program.

[0022] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which is used to store a computer program. When the computer program is executed by a processor, it performs the camera-based image acquisition method described in the above aspect.

[0023] On the other hand, an embodiment of the present application provides a computer program product, which includes a computer program or instructions; when the computer program or instructions are executed by a processor, the camera-based image acquisition method described in the above aspect is executed.

[0024] As can be seen from the above technical solution, a virtual world coordinate system is constructed for a 3D virtual shooting scene, and a 3D model of the object to be photographed is imported into the 3D virtual shooting scene. The model position information and model pose information of the 3D model in the virtual world coordinate system are determined. The camera position information and camera pose information of each virtual camera in the virtual world coordinate system are determined by arranging multiple virtual cameras in the virtual shooting scene. For each virtual camera, the model position information, model pose information, camera position information, and camera pose information are used to capture an image of the object to be photographed from the perspective of the virtual camera. As can be seen, by importing the 3D model of the object to be photographed into the 3D virtual shooting scene to control the position and pose of the 3D model, and arranging multiple virtual cameras to control the position and pose of the multiple virtual cameras, images captured from the perspectives of multiple virtual cameras can be quickly and accurately captured synchronously without the need to purchase multiple physical cameras, build a hardware camera acquisition system on-site, and synchronize the physical cameras. This approach not only saves a significant amount of time and effort, but also reduces hardware costs, improving the efficiency and effectiveness of capturing images captured from the perspectives of multiple cameras. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] FIG1 is a schematic diagram of a hardware camera acquisition system composed of eight physical cameras in a related art;

[0026] FIG2 shows an image captured from the perspective of eight physical cameras in a related art;

[0027] FIG3 is a schematic diagram of an application scenario of an image acquisition method based on a virtual camera provided in an embodiment of the present application;

[0028] FIG4 is a flowchart of an image acquisition method based on a virtual camera provided in an embodiment of the present application;

[0029] FIG5 is a schematic diagram of different layouts of multiple virtual cameras in a three-dimensional virtual shooting scene provided by an embodiment of the present application;

[0030] FIG6 is a schematic diagram of adjusting the posture of a virtual camera according to an embodiment of the present application;

[0031] FIG7 is a schematic diagram of a three-dimensional model of an object to be photographed under different lighting conditions in a three-dimensional virtual shooting scene provided by an embodiment of the present application;

[0032] FIG8 is a schematic diagram of an image of an object to be photographed from the perspective of a virtual camera provided in an embodiment of the present application;

[0033] FIG9 is a flow chart showing the execution steps of an image acquisition method based on a virtual camera according to an embodiment of the present application;

[0034] FIG10 is a schematic diagram of different system architectures of products provided in embodiments of the present application;

[0035] FIG11 is a schematic diagram of an image acquisition device based on a virtual camera provided in an embodiment of the present application;

[0036] FIG12 is a schematic diagram of the structure of a server provided in an embodiment of the present application;

[0037] FIG13 is a schematic structural diagram of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The embodiments of the present application are described below with reference to the accompanying drawings.

[0039] Currently, the acquisition of images captured from multiple camera perspectives using three-dimensional vision technology involves: first, selecting a camera model to purchase multiple corresponding physical cameras; then, constructing multiple physical cameras on-site to form a hardware camera acquisition system, for example, see FIG1 , a schematic diagram of a hardware camera acquisition system composed of eight physical cameras in a related art; finally, time synchronization of the multiple physical cameras is required to enable the synchronous acquisition of images captured from multiple physical camera perspectives by the multiple physical cameras, for example, to obtain images captured from the perspectives of eight physical cameras in a related art as shown in FIG2 .

[0040] However, after research, it was found that the various implementation steps in the above method are rather cumbersome and complicated. Due to the influence of factors such as the environment and hardware, it is difficult to ensure the operational accuracy of each implementation step. It not only consumes a lot of time and energy, but also incurs high hardware costs, greatly reducing the acquisition efficiency and acquisition effect of images taken from multiple camera perspectives.

[0041] In light of this, this application proposes a camera-based image acquisition method and related apparatus. This method introduces a 3D model of the subject to be photographed into a 3D virtual shooting scene to control the 3D model's position and posture, and deploys multiple virtual cameras to control their positions and postures. This method allows for rapid and accurate synchronous acquisition of images from the perspectives of multiple virtual cameras, eliminating the need to purchase multiple physical cameras, build a hardware camera acquisition system, and synchronize the physical cameras. This method saves significant time and effort, reduces hardware costs, and improves the efficiency and effectiveness of capturing images from multiple camera perspectives.

[0042] To facilitate understanding of the technical solution of the present application, the camera-based image acquisition method provided in the embodiment of the present application is introduced below in combination with actual application scenarios.

[0043] See Figure 3, which illustrates an application scenario for a virtual camera-based image acquisition method according to an embodiment of the present application. The application scenario shown in Figure 3 includes a camera acquisition system 301, a 3D reconstruction and rendering system 302, and a 3D display system 303. The camera acquisition system 301 interacts with the 3D reconstruction and rendering system 302, which in turn interacts with the 3D display system 303.

[0044] First, the camera acquisition system 301 pre-creates a 3D virtual shooting scene and constructs a virtual world coordinate system for the 3D virtual shooting scene. Based on this, the camera acquisition system 301 imports a 3D model of the subject to be photographed into the 3D virtual shooting scene constructed in the virtual world coordinate system. The subject to be photographed is a physical object, such as a participant in a virtual video conference. The 3D model of the subject to be photographed is a virtual 3D model that replaces the subject to be photographed in the 3D virtual shooting scene, allowing a virtual camera set up in the 3D virtual shooting scene to capture images of the subject to be photographed.

[0045] As an example, the three-dimensional virtual shooting scene is a virtual video conferencing scene, and the subject to be shot is participant A. In order to capture images of participant A from multiple camera perspectives, the camera acquisition system 301 first needs to import the three-dimensional model of participant A into the virtual video conferencing scene that constructs the virtual world coordinate system.

[0046] Next, the camera acquisition system 301 determines the model position information and model pose information of the 3D model in the virtual world coordinate system. As an example, based on the above example, the camera acquisition system 301 determines the model position information and model pose information of the 3D model in the virtual world coordinate system by controlling the position and pose of the 3D model in the virtual world coordinate system.

[0047] Next, camera acquisition system 301 determines the camera position information and camera pose information of each virtual camera in the virtual world coordinate system based on the layout of the multiple virtual cameras in the 3D virtual shooting scene. As an example, based on the above example, the number of virtual cameras is six. By controlling the layout of the six virtual cameras in the virtual video conferencing scene to be "surround," camera acquisition system 301 determines the camera position information and camera pose information of each virtual camera in the virtual world coordinate system based on the six virtual cameras in the "surround" layout in the virtual video conferencing scene.

[0048] Then, for each virtual camera, the camera acquisition system 301 captures an image of the subject to be captured from the perspective of the virtual camera based on the model position information, model pose information, camera position information, and camera pose information. As an example, based on the above example, the camera acquisition system 301 captures an image of participant A from the perspective of each virtual camera based on the model position information, model pose information, camera position information, and camera pose information, thereby achieving simultaneous capture of images of participant A from the perspectives of multiple virtual cameras.

[0049] Finally, the camera acquisition system 301 sends the images of the subject to be photographed from the perspectives of multiple virtual cameras to the 3D reconstruction and rendering system 302. The 3D reconstruction and rendering system 302 performs 3D reconstruction and rendering on the images of the subject to be photographed from the perspectives of the multiple virtual cameras, obtains a 3D reconstructed and rendered image of the subject to be photographed, and sends it to the 3D display system 303. The 3D display system 303 displays the subject to be photographed in 3D based on the 3D reconstructed and rendered image. As an example, based on the above example, the camera acquisition system 301 sends the images of participant A from the perspectives of multiple virtual cameras to the 3D reconstruction and rendering system 302. The 3D reconstruction and rendering system 302 performs 3D reconstruction and rendering on the images of participant A from the perspectives of multiple virtual cameras, obtains a 3D reconstructed and rendered image of participant A, and sends it to the 3D display system 303. The 3D display system 303 displays participant A in 3D based on the 3D reconstructed and rendered image.

[0050] As can be seen, by importing a 3D model of the subject to be photographed into a 3D virtual shooting scene to control the 3D model's position and posture, and deploying multiple virtual cameras to control their positions and postures, it is possible to quickly and accurately synchronize the images captured from the perspectives of multiple virtual cameras without having to purchase multiple physical cameras, build a hardware camera acquisition system on-site, and synchronize the physical cameras. This approach not only saves considerable time and effort, but also reduces hardware costs, improving the efficiency and quality of image acquisition from multiple camera perspectives.

[0051] The camera-based image acquisition method provided in this application can be applied to camera-based image acquisition devices with data processing capabilities, such as servers and terminal devices. Among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services, etc., but is not limited to these; terminal devices include but are not limited to mobile phones, tablets, computers, computers, smart cameras, smart voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, etc., but are not limited to these. Terminal devices and servers can be directly or indirectly connected through wired or wireless communication, and this application does not impose any restrictions on this.

[0052] The camera-based image acquisition method provided in this application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, in-vehicle scenarios, smart transportation, assisted driving, etc.

[0053] The camera-based image acquisition method provided in this application adopts cloud technology. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and network within a wide area network or local area network to realize data calculation, storage, processing and sharing.

[0054] Cloud computing is a computing model that distributes computing tasks across a resource pool consisting of a large number of computers, enabling various application systems to access computing power, storage space, and information services as needed. The network that provides these resources is called the "cloud." To users, these resources appear infinitely scalable and can be accessed at any time, used on demand, expanded at any time, and paid for on a per-use basis.

[0055] As a provider of cloud computing infrastructure, a cloud computing resource pool (referred to as a cloud platform, generally referred to as an IaaS (Infrastructure as a Service) platform) is established. Various types of virtual resources are deployed in the resource pool for external customers to choose and use. The cloud computing resource pool mainly includes: computing devices (virtualized machines, including operating systems), storage devices, and network devices.

[0056] Cloud conferencing is an efficient, convenient, and low-cost conferencing format based on cloud computing technology. Users can quickly and efficiently share voice, data, and video with teams and clients around the world through a simple, easy-to-use internet interface. The cloud conferencing service provider handles the complex technical aspects of data transmission and processing.

[0057] At present, domestic cloud conferencing mainly focuses on service content based on the SaaS (Software as a Service) model, including telephone, network, video and other service forms. Video conferencing based on cloud computing is called cloud conferencing.

[0058] In the era of cloud conferencing, data transmission, processing, and storage are all handled by the computer resources of video conferencing manufacturers. Users no longer need to purchase expensive hardware or install cumbersome software. They only need to open a browser and log in to the corresponding interface to conduct efficient remote meetings.

[0059] Cloud conferencing systems support dynamic multi-server cluster deployment and offer multiple high-performance servers, significantly improving conference stability, security, and availability. In recent years, video conferencing has gained widespread popularity and adoption across various fields due to its ability to significantly improve communication efficiency, continuously reduce communication costs, and enhance internal management. The integration of cloud computing into video conferencing undoubtedly offers even greater convenience, speed, and ease of use, undoubtedly driving a new surge in video conferencing adoption.

[0060] The following describes in detail the camera-based image acquisition method provided in the embodiments of the present application, taking a terminal device or a server as a camera-based image acquisition device.

[0061] See Figure 4, which is a flowchart of an image acquisition method based on a virtual camera provided in an embodiment of the present application. As shown in Figure 4, the camera-based image acquisition method includes the following steps:

[0062] S401: Importing a three-dimensional model of an object to be photographed into a three-dimensional virtual shooting scene in which a virtual world coordinate system is constructed.

[0063] In the related art, in order to achieve the acquisition of images from multiple camera perspectives using 3D vision technology, the first step is to select and purchase multiple physical cameras, then build multiple physical cameras on-site to form a hardware camera acquisition system, and finally synchronize the multiple physical cameras to achieve the simultaneous acquisition of images from multiple physical camera perspectives. Research has found that the various implementation steps in the above-mentioned related art are relatively cumbersome and complex. Due to the influence of environmental and hardware factors, it is difficult to ensure the accuracy of each implementation step. This not only consumes a lot of time and effort, but also incurs high hardware costs, greatly reducing the efficiency and effectiveness of acquiring images from multiple camera perspectives.

[0064] Therefore, in the embodiments of this application, a virtual simulation of reality is considered, avoiding the implementation steps of the aforementioned related technologies, such as purchasing multiple physical cameras, building a hardware camera acquisition system on-site, and synchronizing the physical cameras. This requires pre-creating a 3D virtual shooting scene to simulate a real 3D shooting scene, and constructing a virtual world coordinate system for the 3D virtual shooting scene, so that spatial points in the 3D virtual shooting scene can be represented by coordinates in the virtual world coordinate system.

[0065] In order to simulate the object to be photographed in a three-dimensional real shooting scene, it is necessary to import a three-dimensional model of the object to be photographed into the three-dimensional virtual shooting scene so that the object to be photographed can be photographed by shooting the three-dimensional model later.

[0066] The 3D model of the object to be photographed must include both geometric and material information. The geometric information represents the spatial shape of the object, while the material information represents the material used. For example, the 3D model of the object to be photographed may be represented by a triangular mesh and a corresponding texture map. The 3D model of the object to be photographed may be in formats such as obj, fbx, or gltf.

[0067] S402: Determine the model position information and model posture information of the three-dimensional model in the virtual world coordinate system.

[0068] In an embodiment of the present application, after executing S401 and importing the three-dimensional model of the object to be photographed into the three-dimensional virtual shooting scene in which the virtual world coordinate system is constructed, the model position information and model posture information of the three-dimensional model in the virtual world coordinate system can be determined. The model position information and model posture information can identify the specific position and posture of the imported three-dimensional model in the three-dimensional virtual shooting scene.

[0069] In order to simulate the position and posture of the object to be photographed in the three-dimensional real shooting scene, it is necessary to set the position and posture of the three-dimensional model of the object to be photographed in the three-dimensional virtual shooting scene, so that the position information and posture information of the three-dimensional model of the object to be photographed in the virtual world coordinate system can be determined as the model position information and model posture information.

[0070] When S402 is specifically implemented, for example, the following two specific implementation methods may be adopted:

[0071] The first specific implementation method: Based on the simulation requirements of the position and posture of the object to be photographed in the three-dimensional real shooting scene, the position and posture of the three-dimensional model of the object to be photographed are directly configured in the three-dimensional virtual shooting scene. Based on this, first, the position configuration information and posture configuration information of the three-dimensional model of the object to be photographed in the virtual world coordinate system can be obtained; then, the model position information of the three-dimensional model of the object to be photographed in the virtual world coordinate system can be determined through the position configuration information, and the model posture information of the three-dimensional model of the object to be photographed in the virtual world coordinate system can be determined through the posture configuration information. Therefore, the present application provides a possible implementation method, and S402 may, for example, include the following S4021-S4022:

[0072] S4021: Obtain position configuration information and posture configuration information of the three-dimensional model in the virtual world coordinate system.

[0073] S4022: Determine model position information and model posture information according to the position configuration information and the posture configuration information.

[0074] The second specific implementation method: In order to make the three-dimensional model of the object to be photographed in the three-dimensional virtual shooting scene more consistent with the movement diversity of the object to be photographed in the three-dimensional real shooting scene, for the original position and posture of the three-dimensional model of the object to be photographed in the three-dimensional virtual shooting scene, the three-dimensional model of the object to be photographed can also be subjected to spatial transformation processing to obtain the three-dimensional model of the object to be photographed after spatial transformation. Based on this, first, the initial model position information, initial model posture information and spatial transformation information of the three-dimensional model of the object to be photographed in the virtual world coordinate system can be obtained; then, on the basis of the initial model position information and initial model posture information, combined with the spatial transformation information, the model position information and model posture information of the three-dimensional model of the object to be photographed in the virtual world coordinate system can be determined. Therefore, the present application provides a possible implementation method, and S402 can, for example, include the following S4023-S4024:

[0075] S4023: Acquire initial model position information, initial model posture information, and space transformation information of the three-dimensional model in the virtual world coordinate system.

[0076] S4024: Determine model position information and model posture information based on the initial model position information, the initial model posture information, and the spatial transformation information.

[0077] In order to simplify the calculation and reduce the amount of calculation, it can be assumed that the object to be photographed is a rigid body, and the three-dimensional model of the object to be photographed is spatially transformed. In fact, the entire three-dimensional model is spatially transformed, such as rotation, translation, scaling and other transformations.

[0078] In addition, in an embodiment of the present application, when the object to be photographed has multiple structural parts, in order to accurately calculate and improve the calculation accuracy, the object to be photographed can also be split into multiple structural parts, and the three-dimensional model of the object to be photographed includes multiple three-dimensional sub-models corresponding to the multiple structural parts of the object to be photographed; based on this, the three-dimensional model of the object to be photographed is spatially transformed, and different spatial transformations can be performed on different three-dimensional sub-models in space. Then, the spatial transformation information of the three-dimensional model of the object to be photographed includes multiple spatial transformation sub-information corresponding to the multiple three-dimensional sub-models.

[0079] As an example, the subject to be photographed is participant A, and participant A can be divided into three structural parts: head, torso, and limbs. The three-dimensional model of participant A includes a three-dimensional sub-model corresponding to the head, a three-dimensional sub-model corresponding to the torso, and a three-dimensional sub-model corresponding to the limbs. Simulating the twisting of the head, rotation of the torso, and swinging of the limbs of participant A in a three-dimensional real shooting scene, different spatial transformation processing is performed on the three-dimensional sub-model corresponding to the head, the three-dimensional sub-model corresponding to the torso, and the three-dimensional sub-model corresponding to the limbs. The spatial transformation information of the three-dimensional model of participant A includes multiple spatial transformation sub-information corresponding to the three-dimensional sub-model corresponding to the head, the three-dimensional sub-model corresponding to the torso, and the three-dimensional sub-model corresponding to the limbs.

[0080] S403: Determine camera position information and camera posture information of each virtual camera in the virtual world coordinate system according to the layout of multiple virtual cameras in the three-dimensional virtual shooting scene.

[0081] In the embodiment of the present application, after executing S402 to determine the model position information and model posture information of the three-dimensional model in the virtual world coordinate system, in order to subsequently capture images captured from multiple camera perspectives using three-dimensional vision technology, multiple physical cameras built on-site in a three-dimensional real shooting scene are also simulated; it is necessary to arrange multiple virtual cameras in the three-dimensional virtual shooting scene. The arrangement of the multiple virtual cameras determines the position and posture of each virtual camera in the virtual world coordinate system. Therefore, the camera position information and camera posture information of each virtual camera in the virtual world coordinate system can be determined by the arrangement of the multiple virtual cameras in the three-dimensional virtual shooting scene. Among them, the virtual camera can be either a monocular camera or a binocular camera, which is not specifically limited in the embodiment of the present application.

[0082] When S403 is specifically implemented, for each virtual camera, first, the positions of multiple virtual cameras need to be set according to the layout method in the three-dimensional virtual shooting scene. Then, according to the layout method of multiple virtual cameras in the three-dimensional virtual shooting scene, the camera position information of the virtual camera in the virtual world coordinate system can be determined. Then, in order to make the three-dimensional model of the object to be photographed in the center of the virtual camera's screen, it is necessary to determine the positional relationship between the three-dimensional model and the virtual camera through the model position information of the three-dimensional model of the object to be photographed and the camera position information of the virtual camera; the posture of the virtual camera is set according to the positional relationship in the three-dimensional virtual shooting scene, and the camera posture information of the virtual camera in the virtual world coordinate system can be determined. Therefore, the present application provides a possible implementation method, and S403 may, for example, include the following S4031-S4033:

[0083] S4031: Determine camera position information according to the layout method.

[0084] The layout mode can be determined in the following two specific implementation modes:

[0085] The first specific implementation method: Based on the layout requirements for multiple virtual cameras, the layout of multiple virtual cameras is directly configured in the three-dimensional virtual shooting scene. Based on this, the layout configuration information of the multiple virtual cameras in the virtual world coordinate system can be first obtained; then, the layout configuration information can be used to determine the layout of the multiple virtual cameras in the virtual world coordinate system. Therefore, this application provides a possible implementation method. The layout determination steps can include the following steps S1-S2:

[0086] S1: Obtaining layout configuration information of multiple virtual cameras in a three-dimensional virtual shooting scene.

[0087] S2: Determine the layout mode according to the layout configuration information.

[0088] The second specific implementation method: In order to make the multiple virtual cameras in the three-dimensional virtual shooting scene more consistent with the layout diversification of the multiple physical cameras in the three-dimensional real shooting scene, for the original layout method of the multiple virtual cameras in the three-dimensional virtual shooting scene, the layout of the multiple virtual cameras can also be adjusted to obtain the multiple virtual cameras after the layout adjustment. Based on this, first, the initial layout method and layout adjustment information of the multiple virtual cameras in the virtual world coordinate system can be obtained; then, based on the initial layout method, combined with the layout adjustment information, the layout method of the multiple virtual cameras in the virtual world coordinate system can be determined. Therefore, the present application provides a possible implementation method, and the layout determination step can, for example, include the following S3-S4:

[0089] S3: Obtaining initial layout mode and layout adjustment information of multiple virtual cameras in the three-dimensional virtual shooting scene.

[0090] S4: Determine the layout mode according to the initial layout mode and the layout adjustment information.

[0091] In the related art, to achieve different layouts of multiple physical cameras, such as "surround," "up-and-down," and "four-corner," it is necessary to disassemble and reassemble multiple physical cameras in the on-site hardware camera acquisition system. This method is limited by hardware debugging, and not only is the implementation process cumbersome and complicated, but it also consumes a lot of time, making it difficult to verify the differences and impacts of the different layouts of multiple virtual cameras on the images captured from the perspectives of multiple cameras. However, the implementation methods S3-S4 in the embodiments of the present application directly adjust the original layouts of multiple virtual cameras in the three-dimensional virtual shooting scene. Without hardware debugging, different layouts of multiple virtual cameras can be quickly, easily, and low-costly implemented, and images captured from the perspectives of multiple virtual cameras under different layouts of multiple virtual cameras can be captured efficiently and at low cost, thereby facilitating verification of the differences and impacts of different layouts of multiple virtual cameras on the images captured from the perspectives of multiple cameras.

[0092] As an example, see Figure 5, a schematic diagram illustrating different layouts of multiple virtual cameras in a three-dimensional virtual filming scene. The three-dimensional virtual filming scene is a virtual video conferencing scene, the subject to be filmed is a table, and there are six virtual cameras, represented by small squares in the figure. Figure 5 (a) illustrates a "top-and-bottom" layout of the six virtual cameras in the virtual video conferencing scene, arranged around a screen in front of the table and facing the table. Figure 5 (b) illustrates a "surround" layout of the six virtual cameras in the virtual video conferencing scene, arranged around a screen in front of the table and facing the table.

[0093] S4032: Determine the positional relationship between the three-dimensional model and the virtual camera based on the model position information and the camera position information.

[0094] S4033: Adjust the posture of the virtual camera according to the position relationship to determine the camera posture information.

[0095] As an example, see FIG6 , a schematic diagram of adjusting the posture of a virtual camera. Based on FIG5 (b) above, for the virtual camera located on the left side of the screen, in order to center the three-dimensional model of the table in the virtual camera's image, the positional relationship between the three-dimensional model of the table and the virtual camera is determined based on the model position information of the three-dimensional model of the table and the camera position information of the virtual camera. Based on this positional relationship, the virtual camera is rotated about the Z axis toward the table by an angle α, which is calculated using the trigonometric function relationship indicated in the figure. Similarly, the virtual camera located at the top of the screen needs to be rotated about the Y axis toward the table by a certain angle.

[0096] S404: For each virtual camera, based on the model position information, the model posture information, the camera position information, and the camera posture information, an image of the object to be photographed is collected from the perspective of the virtual camera.

[0097] In the embodiment of the present application, after executing S402-S403 to determine the model position information and model posture information of the three-dimensional model of the object to be photographed in the virtual world coordinate system, as well as the camera position information and camera posture information of each virtual camera, for each virtual camera, the image acquisition device can accurately determine the relative position and posture relationship between the three-dimensional model and the virtual camera in the three-dimensional virtual shooting scene through the model position information, model posture information, camera position information and camera posture information, so that the image of the three-dimensional model relative to the virtual camera in the three-dimensional virtual shooting scene can be captured by the virtual camera, thereby efficiently, conveniently and highly accurately capturing the captured image of the object to be photographed from the perspective of the virtual camera, providing a basis for the subsequent high-precision three-dimensional reconstruction and rendering of the object to be photographed.

[0098] When S404 is specifically implemented, for each virtual camera, first, the distance of the three-dimensional model of the object to be photographed relative to the virtual camera can be determined through the model position information and model posture information of the three-dimensional model of the object to be photographed in the virtual world coordinate system, that is, the depth of the three-dimensional model of the object to be photographed relative to the virtual camera; then, the external parameters of the virtual camera can be determined through the camera position information and camera posture information of the virtual camera in the virtual world coordinate system; finally, based on the above-mentioned distance of the three-dimensional model of the object to be photographed relative to the virtual camera and the external parameters of the virtual camera, combined with the preset internal parameters of the virtual camera, the image under the visual sense of the virtual camera can be rendered to achieve the acquisition of the captured image under the visual sense of the virtual camera. Therefore, the present application provides a possible implementation method, and S404 may, for example, include the following S4041-S4043:

[0099] S4041: For each virtual camera, determine the distance between the three-dimensional model and the virtual camera based on the model position information, the model posture information, the camera position information, and the camera posture information.

[0100] In related art, the depth of the object to be photographed relative to the virtual camera is estimated based on the time-of-flight principle. This is subject to multipath interference, resulting in low measurement accuracy and inaccurate depth estimation. However, in the implementation method of S4041 in the embodiment of the present application, the model position information and model pose information of the three-dimensional model of the object to be photographed in the virtual world coordinate system, as well as the camera position information and camera pose information of the virtual camera, are all known information. This allows the depth of the three-dimensional model of the object to be photographed relative to the virtual camera to be accurately calculated, thereby improving depth accuracy.

[0101] S4042: Determine the external parameters of the virtual camera based on the camera position information and the camera attitude information.

[0102] As an example, the following formula is used to determine the external parameters of the virtual camera:

[0103] Among them, R represents the posture information of the coordinate axis in the virtual world coordinate system in the virtual camera coordinate system of the virtual camera, t represents the position information of the coordinate origin in the virtual world coordinate system in the virtual camera coordinate system of the virtual camera, C represents the camera position information of the virtual camera in the virtual world coordinate system, R C Represents the camera pose information of the virtual camera in the virtual world coordinate system, and I represents the unit matrix.

[0104] In related technologies, to achieve extrinsic parameter calibration of a physical camera, it is necessary to first extract image features based on Structure from Motion (SfM), estimate the initial extrinsic parameters of the physical camera through feature matching, and then align the point cloud based on the Iterative Closest Point (ICP) algorithm to calculate the final extrinsic parameters of the physical camera. This method will result in inaccurate feature matching estimation when the object to be photographed has no texture or repeated texture, resulting in estimation errors in the camera's extrinsic parameters. However, in the implementation method of S4042 in the embodiment of the present application, the camera position information and camera posture information of the virtual camera in the virtual world coordinate system are both known information, which can accurately calculate the extrinsic parameters of the virtual camera, thereby avoiding estimation errors in the camera's extrinsic parameters.

[0105] S4043: Capture images based on the distance, external parameters, and preset internal parameters of the virtual camera.

[0106] Among them, the preset internal parameters of the virtual camera are the inherent properties of the known virtual camera, which are related to the image sensor characteristics in the virtual camera and the errors of the lens itself, and usually include focal length, principal point offset, lens distortion, etc.; in order to simulate different physical cameras in a diversified way, in the three-dimensional virtual shooting scene, the preset internal parameters of the virtual camera can also be adjusted arbitrarily, then the internal parameter adjustment information of the preset internal parameters can be obtained, and the preset internal parameters can be adjusted to internal parameters through the internal parameter adjustment information of the preset internal parameters, so that based on the distance of the three-dimensional model of the object to be photographed relative to the virtual camera and the external parameters of the virtual camera, combined with the internal parameters of the virtual camera, the image under the visual sense of the virtual camera can be rendered, and the captured image under the visual sense of the virtual camera can be acquired. Therefore, the present application provides a possible implementation method, and the method can also include the following S5-S6:

[0107] S5: Obtain internal parameter adjustment information of the preset internal parameter.

[0108] S6: Adjust the preset internal reference to the internal reference according to the internal reference adjustment information of the preset internal reference.

[0109] Correspondingly, S4043 may include, for example: acquiring and shooting images according to the distance, external parameters, and internal parameters.

[0110] In addition, in an embodiment of the present application, in order to make the captured images of the object to be photographed from the perspective of the virtual camera more diverse, the lighting conditions of the three-dimensional virtual shooting scene can also be configured, for example, the light source and light intensity can be configured to simulate the three-dimensional real shooting scene under different lighting conditions. Based on this, first, the lighting configuration information of the three-dimensional virtual shooting scene can be obtained; then, the lighting conditions of the three-dimensional virtual shooting scene can be determined through the lighting configuration information, so that when executing S404, the captured images of the object to be photographed from the perspective of the virtual camera that meet the lighting conditions can be collected in combination with the lighting conditions. Therefore, the present application provides a possible implementation method, and the method can also include the following S7-S8:

[0111] S7: Obtain lighting configuration information of the three-dimensional virtual shooting scene.

[0112] S8: Determine the lighting conditions of the three-dimensional virtual shooting scene according to the lighting configuration information.

[0113] Correspondingly, S404 may include, for example: for each virtual camera, collecting a captured image that meets the lighting conditions according to the model position information, the model posture information, the camera position information, the camera posture information and the lighting conditions.

[0114] As an example, referring to Figures 5 and 6 above, Figure 7 shows a schematic diagram of a 3D model of an object to be photographed under different lighting conditions in a 3D virtual shooting scene. Figure 7 (a) shows a 3D model of a table with one light source added to the virtual video conferencing scene; Figure 7 (b) shows a 3D model of a table with two light sources added to the virtual video conferencing scene.

[0115] In addition, since the related art requires multiple sampling integrations when detecting phase shift based on the time-of-flight principle, resource consumption is large, resulting in a low image resolution of the captured image of the object to be photographed, which is not conducive to subsequent high-precision three-dimensional reconstruction and rendering. Therefore, in an embodiment of the present application, in order to improve the image properties of the captured image of the object to be photographed, the image properties of the captured image can also be configured to facilitate subsequent high-precision three-dimensional reconstruction and rendering. Based on this, first, the image property configuration information of the captured image can be obtained; then, the image property information of the captured image can be determined through the image property configuration information, so that when executing S404, the image property information can be combined to capture the captured image of the object to be photographed under the perspective of the virtual camera and in accordance with the image property information. Therefore, the present application provides a possible implementation method, and the method can also include the following S9-S10, for example:

[0116] S9: Acquire image attribute configuration information of the captured image.

[0117] Among them, since the captured images of the object to be photographed are mainly used for subsequent high-precision three-dimensional reconstruction and rendering, it is necessary to ensure the image properties such as image resolution or image frame rate of the captured images; therefore, the image attribute configuration information may, for example, include image resolution configuration information or image frame rate configuration information.

[0118] S10: Determine image attribute information of the captured image according to the image attribute configuration information.

[0119] Correspondingly, S404 may include, for example: for each virtual camera, collecting a captured image that meets the image attribute information according to the model position information, the model posture information, the camera position information, the camera posture information and the image attribute information.

[0120] In addition, in an embodiment of the present application, in order to diversify the background image of the captured image of the subject to be photographed, the background image of the captured image can be further rendered to obtain captured images of the subject to be photographed under different backgrounds. Based on this, first, background rendering information of the captured image can be obtained; then, based on the background rendering information, a preset background image of the captured image can be rendered as the background image. Therefore, the present application provides a possible implementation method, and the method can also include S11-12:

[0121] S11: Obtain background rendering information of the captured image.

[0122] S12: Rendering a preset background image of the captured image as a background image according to the background rendering information.

[0123] Based on the above description, see Figure 8, a schematic diagram of an image of an object to be photographed from the perspective of a virtual camera. The object to be photographed is an open-source monkey head model (Suzanne Monkey). Figure 8 (a) shows a color image of the open-source monkey head model from the perspective of the virtual camera, and Figure 8 (b) shows a depth image of the open-source monkey head model from the perspective of the virtual camera.

[0124] The camera-based image acquisition method provided in the above embodiment constructs a virtual world coordinate system for a three-dimensional virtual shooting scene, imports a three-dimensional model of the object to be photographed into the three-dimensional virtual shooting scene, determines the model position information and model pose information of the three-dimensional model in the virtual world coordinate system, and determines the camera position information and camera pose information of each virtual camera in the virtual world coordinate system by arranging multiple virtual cameras in the virtual shooting scene. For each virtual camera, the model position information, model pose information, camera position information, and camera pose information are used to capture an image of the object to be photographed from the perspective of the virtual camera. As can be seen, by importing the three-dimensional model of the object to be photographed into the three-dimensional virtual shooting scene to control the position and pose of the three-dimensional model, and by arranging multiple virtual cameras to control the position and pose of the multiple virtual cameras, images captured from the perspectives of multiple virtual cameras can be quickly and accurately captured synchronously without the need to purchase multiple physical cameras, build a hardware camera acquisition system on-site, and synchronize the physical cameras. This method not only saves a considerable amount of time and effort, but also reduces hardware costs, and improves the efficiency and effectiveness of capturing images captured from the perspectives of multiple cameras.

[0125] Corresponding to the above-mentioned camera-based image acquisition method, see FIG9 for a flowchart of the execution steps of the virtual camera-based image acquisition method. The execution steps are as follows:

[0126] Step 1: In the three-dimensional virtual shooting scene constructed in the virtual world coordinate system, import the three-dimensional model of the object to be photographed.

[0127] Step 2: Set the position and posture of the 3D model in the virtual world coordinate system.

[0128] Step 3: Set the positions and postures of multiple virtual cameras in the virtual world coordinate system.

[0129] Step 4: Set the lighting conditions of the 3D virtual shooting scene.

[0130] Step 5: Render the image of the object to be photographed from the perspective of each virtual camera.

[0131] Furthermore, the aforementioned camera-based image acquisition method is applied to the camera acquisition system within the product's system architecture. See Figure 10 for a schematic diagram of different system architectures for the product. The product comprises a camera acquisition system, a 3D reconstruction and rendering system, and a 3D display system. The camera acquisition system is deployed at the transmitter, the 3D reconstruction and rendering system can be deployed at either the transmitter or the receiver, and the 3D display system is deployed at the receiver. Figure 10 (a) shows the 3D reconstruction and rendering system deployed at the transmitter, while Figure 10 (b) shows the 3D reconstruction and rendering system deployed at the receiver.

[0132] Among them, the camera acquisition system sends the images of the object to be photographed from the perspective of multiple virtual cameras to the three-dimensional reconstruction and rendering system; the three-dimensional reconstruction and rendering system performs three-dimensional reconstruction and rendering on the images of the object to be photographed from the perspective of multiple virtual cameras, obtains the three-dimensional reconstructed rendering image of the object to be photographed and sends it to the three-dimensional display system; the three-dimensional display system displays the object to be photographed in three dimensions based on the three-dimensional reconstructed rendering image of the object to be photographed.

[0133] In response to the camera-based image acquisition method provided in the above embodiment, an embodiment of the present application also provides a camera-based image acquisition device.

[0134] Refer to Figure 11, which is a schematic diagram of an image acquisition device based on a virtual camera provided in an embodiment of the present application. As shown in Figure 11, the camera-based image acquisition device 1100 includes: an import unit 1101, a determination unit 1102, and a collection unit 1103;

[0135] The importing unit 1101 is used to import a 3D model of an object to be photographed into a 3D virtual shooting scene constructed in a virtual world coordinate system;

[0136] A determining unit 1102 is configured to determine model position information and model posture information of the three-dimensional model in the virtual world coordinate system;

[0137] The determining unit 1102 is further configured to determine the camera position information and camera posture information of each virtual camera in the virtual world coordinate system according to the layout of the multiple virtual cameras in the three-dimensional virtual shooting scene;

[0138] The acquisition unit 1103 is configured to acquire, for each virtual camera, an image of the object to be photographed from the perspective of the virtual camera according to the model position information, the model posture information, the camera position information, and the camera posture information.

[0139] As a possible implementation manner, the determining unit 1102 is configured to:

[0140] Determine the camera position information according to the layout method;

[0141] Determine the positional relationship between the three-dimensional model and the virtual camera according to the model position information and the camera position information;

[0142] The virtual camera is adjusted in attitude according to the position relationship to determine the camera attitude information.

[0143] As a possible implementation, the collection unit 1103 is configured to:

[0144] For each virtual camera, determine the distance between the 3D model and the virtual camera based on the model position information, the model attitude information, the camera position information, and the camera attitude information;

[0145] Determine the external parameters of the virtual camera based on the camera position information and camera attitude information;

[0146] The captured image is acquired based on the distance, external parameters and preset internal parameters of the virtual camera.

[0147] As a possible implementation manner, the determining unit 1102 is further configured to:

[0148] Obtaining layout configuration information of multiple virtual cameras in a three-dimensional virtual shooting scene;

[0149] Determine the layout mode based on the layout configuration information.

[0150] As a possible implementation manner, the determining unit 1102 is further configured to:

[0151] Obtaining initial layout mode and layout adjustment information of multiple virtual cameras in a three-dimensional virtual shooting scene;

[0152] Determine the layout mode according to the initial layout mode and layout adjustment information.

[0153] As a possible implementation manner, the determining unit 1102 is configured to:

[0154] Obtaining position configuration information and posture configuration information of the three-dimensional model in the virtual world coordinate system;

[0155] The model position information and the model attitude information are determined according to the position configuration information and the attitude configuration information.

[0156] As a possible implementation manner, the determining unit 1102 is configured to:

[0157] Obtaining initial model position information, initial model posture information, and spatial transformation information of the three-dimensional model in the virtual world coordinate system;

[0158] The model position information and the model posture information are determined according to the initial model position information, the initial model posture information and the spatial transformation information.

[0159] As a possible implementation, when the object to be photographed includes multiple structural parts, the three-dimensional model includes multiple three-dimensional sub-models corresponding to the multiple structural parts, and the spatial transformation information includes multiple spatial transformation sub-information corresponding to the multiple three-dimensional sub-models.

[0160] As a possible implementation, the apparatus further includes: a first acquiring unit;

[0161] A first acquisition unit is used to acquire lighting configuration information of a three-dimensional virtual shooting scene;

[0162] The determining unit 1102 is further configured to determine the lighting conditions of the 3D virtual shooting scene according to the lighting configuration information;

[0163] The acquisition unit 1103 is configured to acquire, for each virtual camera, images that meet the lighting conditions based on the model position information, the model posture information, the camera position information, the camera posture information, and the lighting conditions.

[0164] As a possible implementation, the method further includes: a second acquiring unit;

[0165] A second acquiring unit, configured to acquire image attribute configuration information of the captured image;

[0166] The determining unit 1102 is further configured to determine image attribute information of the captured image according to the image attribute configuration information;

[0167] The acquisition unit 1103 is configured to acquire, for each virtual camera, a captured image that meets the image attribute information based on the model position information, the model posture information, the camera position information, the camera posture information, and the image attribute information.

[0168] As a possible implementation, the method further includes: a third acquisition unit and a rendering unit;

[0169] a third acquiring unit, configured to acquire background rendering information of the captured image;

[0170] The rendering unit is used to render the preset background image of the captured image as the background image according to the background rendering information.

[0171] The camera-based image acquisition device provided in the above embodiment constructs a virtual world coordinate system for a three-dimensional virtual shooting scene, imports a three-dimensional model of the object to be photographed into the three-dimensional virtual shooting scene, determines the model position information and model pose information of the three-dimensional model in the virtual world coordinate system, and determines the camera position information and camera pose information of each virtual camera in the virtual world coordinate system by arranging multiple virtual cameras in the virtual shooting scene. For each virtual camera, the model position information, model pose information, camera position information, and camera pose information are used to capture an image of the object to be photographed from the perspective of the virtual camera. As can be seen, by importing the three-dimensional model of the object to be photographed into the three-dimensional virtual shooting scene to control the position and pose of the three-dimensional model, and by arranging multiple virtual cameras to control the position and pose of the multiple virtual cameras, images captured from the perspectives of multiple virtual cameras can be quickly and accurately captured synchronously without the need to purchase multiple physical cameras, build a hardware camera acquisition system on-site, and synchronize the physical cameras. This method not only saves a considerable amount of time and effort, but also reduces hardware costs, and improves the efficiency and effectiveness of capturing images captured from the perspectives of multiple cameras.

[0172] In response to the camera-based image acquisition method described above, an embodiment of the present application also provides a camera-based image acquisition device to enable the above-mentioned camera-based image acquisition method to be implemented and applied in practice. The following will introduce the computer device provided in the embodiment of the present application from the perspective of hardware instantiation.

[0173] Referring to Figure 12, Figure 12 is a schematic diagram of a server structure provided in an embodiment of the present application. The server 1200 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPUs) 1222 (for example, one or more processors) and a memory 1232, and one or more storage media 1230 (for example, one or more mass storage devices) storing application programs 1242 or data 1244. Among them, the memory 1232 and the storage medium 1230 can be temporary storage or permanent storage. The program stored in the storage medium 1230 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 1222 can be configured to communicate with the storage medium 1230 to execute a series of instruction operations in the storage medium 1230 on the server 1200.

[0174] The server 1200 may also include one or more power supplies 1226, one or more wired or wireless network interfaces 1250, one or more input and output interfaces 1258, and / or one or more operating systems 1241, such as Windows Server 200. TM , Mac OS X TM , Unix TM , Linux TM , FreeBSD TM etc.

[0175] The steps executed by the server in the above embodiment may be based on the server structure shown in FIG12 .

[0176] The CPU 1222 is configured to execute the following steps:

[0177] In constructing a three-dimensional virtual shooting scene in a virtual world coordinate system, a three-dimensional model of the object to be shot is imported;

[0178] Determining model position information and model posture information of the three-dimensional model in the virtual world coordinate system;

[0179] Determining camera position information and camera posture information of each virtual camera in a virtual world coordinate system according to a layout of multiple virtual cameras in a three-dimensional virtual shooting scene;

[0180] For each virtual camera, an image of the object to be photographed from the perspective of the virtual camera is collected based on the model position information, the model posture information, the camera position information, and the camera posture information.

[0181] Optionally, the CPU 1222 may also execute the method steps of any specific implementation of the camera-based image acquisition method in the embodiments of the present application.

[0182] See Figure 13, which is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application. For ease of explanation, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method section of the embodiment of the present application. The terminal device can be any terminal device including a mobile phone, tablet computer, PDA, etc. Taking the terminal device as a mobile phone as an example:

[0183] FIG13 is a block diagram showing a partial structure of a mobile phone related to a terminal device provided in an embodiment of the present application. Referring to FIG13 , the mobile phone includes components such as a radio frequency (RF) circuit 1310, a memory 1320, an input unit 1330, a display unit 1340, a sensor 1350, an audio circuit 1360, a wireless fidelity (WiFi) module 1370, a processor 1380, and a power supply 1390. Those skilled in the art will appreciate that the mobile phone structure shown in FIG13 does not limit the mobile phone and may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0184] The following is a detailed introduction to the various components of the mobile phone in conjunction with Figure 13:

[0185] The RF circuit 1310 may be used for receiving and sending signals during information transmission or communication, and in particular, for receiving downlink information from a base station and sending it to the processor 1380 for processing.

[0186] The memory 1320 may be used to store software programs and modules. The processor 1380 implements various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 1320 .

[0187] The input unit 1330 may be configured to receive input digital or character information and generate key signal input related to user settings and function control of the mobile phone. Specifically, the input unit 1330 may include a touch panel 1331 and other input devices 1332 .

[0188] The display unit 1340 may be configured to display information input by the user or information provided to the user, as well as various menus of the mobile phone. The display unit 1340 may include a display panel 1341 .

[0189] The mobile phone may further include at least one sensor 1350 .

[0190] The audio circuit 1360 , the speaker 1361 , and the microphone 1362 can provide an audio interface between the user and the mobile phone.

[0191] The processor 1380 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. By running or executing software programs and / or modules stored in the memory 1320 and calling data stored in the memory 1320, it performs various functions of the mobile phone and processes data, thereby controlling the mobile phone as a whole.

[0192] Although not shown, the mobile phone may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0193] In the embodiment of the present application, the memory 1320 included in the mobile phone can store program codes and transmit the program codes to the processor.

[0194] The processor 1380 included in the mobile phone can execute the camera-based image acquisition method provided in the above embodiment according to the instructions in the program code.

[0195] An embodiment of the present application further provides a computer-readable storage medium for storing a computer program, which is used to execute the camera-based image acquisition method provided in the above embodiment.

[0196] The present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the camera-based image acquisition method provided in various optional implementations of the above aspects.

[0197] A person skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the above-mentioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the above-mentioned storage medium can be at least one of the following media: read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc., various media that can store program codes.

[0198] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments. The device and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0199] The above is merely one specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A camera-based image acquisition method, the method being performed by an image acquisition device, the method comprising: In a three-dimensional virtual shooting scene having a virtual world coordinate system, a three-dimensional model of the object to be shot is imported; Determining model position information and model posture information of the three-dimensional model in the virtual world coordinate system; Determining camera position information and camera posture information of each virtual camera in the virtual world coordinate system according to a layout of multiple virtual cameras in the three-dimensional virtual shooting scene; For each virtual camera, an image of the object to be photographed from the perspective of the virtual camera is collected based on the model position information, the model posture information, the camera position information, and the camera posture information.

2. The method according to claim 1, wherein determining the camera position information and camera posture information of each virtual camera in the virtual world coordinate system according to the layout of multiple virtual cameras in the three-dimensional virtual shooting scene comprises: Determining the camera position information according to the layout mode; Determining a positional relationship between the three-dimensional model and the virtual camera according to the model position information and the camera position information; The virtual camera is adjusted in posture according to the positional relationship to determine the camera posture information.

3. The method according to claim 1 or 2, wherein for each virtual camera, based on the model position information, the model posture information, the camera position information, and the camera posture information, collecting an image of the object to be photographed from the perspective of the virtual camera, comprises: For each virtual camera, determining a distance between the three-dimensional model and the virtual camera according to the model position information, the model pose information, the camera position information, and the camera pose information; Determining the external parameters of the virtual camera according to the camera position information and the camera posture information; The captured image is acquired according to the distance, the external parameter, and a preset internal parameter of the virtual camera.

4. The method according to any one of claims 1 to 3, wherein the step of determining the layout mode comprises: Acquiring layout configuration information of a plurality of virtual cameras in the three-dimensional virtual shooting scene; The layout mode is determined according to the layout configuration information.

5. The method according to any one of claims 1 to 4, wherein the step of determining the layout comprises: Acquiring initial layout mode and layout adjustment information of a plurality of virtual cameras in the three-dimensional virtual shooting scene; The layout mode is determined according to the initial layout mode and the layout adjustment information.

6. The method according to any one of claims 1 to 5, wherein determining the model position information and model posture information of the three-dimensional model in the virtual world coordinate system comprises: Acquiring position configuration information and posture configuration information of the three-dimensional model in the virtual world coordinate system; The model position information and the model posture information are determined according to the position configuration information and the posture configuration information.

7. The method according to any one of claims 1 to 6, wherein determining the model position information and model posture information of the three-dimensional model in the virtual world coordinate system comprises: Acquiring initial model position information, initial model posture information, and spatial transformation information of the three-dimensional model in the virtual world coordinate system; The model position information and the model posture information are determined according to the initial model position information, the initial model posture information and the spatial transformation information.

8. According to the method of claim 7, when the object to be photographed includes multiple structural parts, the three-dimensional model includes multiple three-dimensional sub-models corresponding to the multiple structural parts, and the spatial transformation information includes multiple spatial transformation sub-information corresponding to the multiple three-dimensional sub-models.

9. The method according to any one of claims 1 to 8, further comprising: Acquiring lighting configuration information of the three-dimensional virtual shooting scene; determining the lighting conditions of the three-dimensional virtual shooting scene according to the lighting configuration information; The step of collecting, for each virtual camera, an image of the object to be photographed from the perspective of the virtual camera according to the model position information, the model posture information, the camera position information, and the camera posture information includes: For each virtual camera, a captured image that meets the lighting condition is collected based on the model position information, the model posture information, the camera position information, the camera posture information and the lighting condition.

10. The method according to any one of claims 1 to 9, further comprising: Acquiring image attribute configuration information of the captured image; determining image attribute information of the captured image according to the image attribute configuration information; The step of collecting, for each virtual camera, an image of the object to be photographed from the perspective of the virtual camera according to the model position information, the model posture information, the camera position information, and the camera posture information includes: For each virtual camera, a captured image that meets the image attribute information is collected based on the model position information, the model posture information, the camera position information, the camera posture information and the image attribute information.

11. The method according to any one of claims 1 to 10, further comprising: Obtaining background rendering information of the captured image; According to the background rendering information, a preset background image of the captured image is rendered as a background image.

12. A camera-based image acquisition device, comprising: Import unit, determination unit and acquisition unit; The importing unit is used to import the three-dimensional model of the object to be photographed into the three-dimensional virtual shooting scene in which the virtual world coordinate system is constructed; The determining unit is configured to determine the model position information and the model posture information of the three-dimensional model in the virtual world coordinate system; The determining unit is further configured to determine camera position information and camera posture information of each virtual camera in the virtual world coordinate system according to a layout of multiple virtual cameras in the three-dimensional virtual shooting scene; The acquisition unit is configured to acquire, for each virtual camera, an image of the object to be photographed from the perspective of the virtual camera according to the model position information, the model posture information, the camera position information, and the camera posture information.

13. A computer device comprising a processor and a memory: The memory is used to store a computer program and transmit the computer program to the processor; The processor is configured to execute the camera-based control method according to any one of claims 1 to 11 according to the computer program. Image acquisition method.

14. A computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, wherein when the computer program is executed by a processor, the camera-based image acquisition method according to any one of claims 1 to 11 is executed.

15. A computer program product, comprising a computer program or instructions; when the computer program or instructions are executed by a processor, the camera-based image acquisition method according to any one of claims 1 to 11 is executed.

Images