A camera-based image acquisition method and related apparatus

Abstract

Embodiments of the present application disclose a camera-based image acquisition method and related device, the method comprising: constructing a virtual world coordinate system in a three-dimensional virtual shooting scene, and importing a three-dimensional model of a to-be-shot object; determining target model position information and target model posture information of the three-dimensional model in the virtual world coordinate system; determining target camera position information and target camera posture information of each virtual camera in the virtual world coordinate system through a target layout mode of a plurality of virtual cameras in the virtual shooting scene; and for each virtual camera, acquiring a shooting image of the to-be-shot object under the perspective of the virtual camera through the target model position information, the target model posture information, the target camera position information, and the target camera posture information. The shooting image under the perspective of the plurality of virtual cameras can be quickly and accurately acquired synchronously without implementing steps such as purchasing a plurality of entity cameras, building a hardware camera acquisition system on site, and time synchronization of the entity cameras.

Description

Technical Field

[0001] This application relates to the field of image acquisition, and in particular to a camera-based image acquisition method and related apparatus. Background Technology

[0002] At present, with the rapid development of 3D vision technology, in order to achieve high-precision 3D reconstruction and rendering of the object to be photographed in 3D shooting scenarios such as video conferencing, and to enhance the realism and immersion of shooting scenarios such as video conferencing, it is necessary to acquire images from multiple camera perspectives through 3D vision technology, so as to provide a foundation for subsequent high-precision 3D reconstruction and rendering.

[0003] In related technologies, acquiring images from multiple camera perspectives using 3D vision technology refers to: first, selecting and purchasing multiple physical cameras; then, setting up a hardware camera acquisition system on-site using multiple physical cameras; and finally, synchronizing the multiple physical cameras in time to enable simultaneous acquisition of images from multiple camera perspectives.

[0004] However, the implementation steps in the above methods are quite cumbersome and complex. Due to the influence of environmental and hardware factors, it is difficult to guarantee the accuracy of each implementation step. This not only consumes a lot of time and effort, but also incurs high hardware costs, greatly reducing the acquisition efficiency and effect of images captured from multiple camera perspectives. Summary of the Invention

[0005] To address the aforementioned technical issues, this application provides a camera-based image acquisition method and related apparatus. This method eliminates the need for purchasing multiple physical cameras, setting up a hardware camera acquisition system on-site, and synchronizing the physical cameras with the time. It enables the rapid and accurate synchronous acquisition of images captured from the perspectives of multiple virtual cameras. This saves significant time and effort, reduces hardware costs, and improves the efficiency and effectiveness of image acquisition from multiple camera perspectives.

[0006] The embodiments of this application disclose the following technical solutions:

[0007] On one hand, this application provides a camera-based image acquisition method, the method comprising:

[0008] Import the 3D model of the object to be photographed into the 3D virtual shooting scene that constructs a virtual world coordinate system;

[0009] Determine the target model's position information and target model's pose information in the virtual world coordinate system;

[0010] Based on the target layout of multiple virtual cameras in the three-dimensional virtual shooting scene, determine the target camera position information and target camera pose information of each virtual camera in the virtual world coordinate system;

[0011] For each virtual camera, an image of the object to be photographed is acquired from the perspective of the virtual camera, based on the target model position information, the target model pose information, the target camera position information, and the target camera pose information.

[0012] On the other hand, this application provides a camera-based image acquisition device, the device comprising: an import unit, a determination unit, and an acquisition unit;

[0013] The import unit is used to import the three-dimensional model of the object to be photographed into a three-dimensional virtual shooting scene that constructs a virtual world coordinate system.

[0014] The determining unit is used to determine the target model position information and target model pose information of the three-dimensional model in the virtual world coordinate system;

[0015] The determining unit is further configured to determine the target camera position information and target camera pose information of each virtual camera in the virtual world coordinate system based on the target layout of multiple virtual cameras in the three-dimensional virtual shooting scene.

[0016] The acquisition unit is used to acquire images of the object to be photographed from the perspective of the virtual camera for each virtual camera, based on the target model position information, the target model pose information, the target camera position information, and the target camera pose information.

[0017] On the other hand, this application provides a camera-based image acquisition device, the device including a processor and a memory:

[0018] The memory is used to store program code and transmit the program code to the processor;

[0019] The processor is used to execute the camera-based image acquisition method described above according to the instructions in the program code.

[0020] On the other hand, embodiments of this application provide a computer-readable storage medium for storing a computer program that, when executed by a processor, performs the camera-based image acquisition method described above.

[0021] On the other hand, embodiments of this application provide a computer program product, which includes a computer program or instructions; when the computer program or instructions are executed by a processor, the camera-based image acquisition method described above is performed.

[0022] As can be seen from the above technical solution, a virtual world coordinate system is constructed for the 3D virtual shooting scene, and a 3D model of the object to be photographed is imported into the 3D virtual shooting scene. The target model's position and pose information are determined in the virtual world coordinate system. The target camera's position and pose information are determined by the target layout of multiple virtual cameras in the virtual shooting scene. For each virtual camera, images of the object to be photographed from the virtual camera's perspective are acquired using the target model's position and pose information, the target camera's position information, and the target camera's pose information. Therefore, importing the 3D model of the object to be photographed into the 3D virtual shooting scene to control its position and pose, and deploying multiple virtual cameras to control their positions and poses, eliminates the need to purchase multiple physical cameras, build a physical hardware camera acquisition system, and synchronize the physical cameras. This allows for the rapid and accurate synchronous acquisition of images from multiple virtual camera perspectives. In other words, this method saves significant time and effort, reduces hardware costs, and improves the efficiency and quality of image acquisition from multiple camera perspectives. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of a hardware camera acquisition system consisting of eight physical cameras in a related technology.

[0025] Figure 2 Images captured from the perspective of eight physical cameras in a related technology;

[0026] Figure 3 This is a schematic diagram illustrating an application scenario of a camera-based image acquisition method provided in an embodiment of this application.

[0027] Figure 4 A flowchart illustrating a camera-based image acquisition method provided in this application embodiment;

[0028] Figure 5A schematic diagram illustrating different target layouts of multiple virtual cameras in a three-dimensional virtual shooting scene provided in the embodiments of this application;

[0029] Figure 6 This is a schematic diagram illustrating the posture adjustment of a virtual camera as provided in an embodiment of this application.

[0030] Figure 7 A schematic diagram of a 3D model of an object to be photographed under different lighting conditions in a 3D virtual shooting scene provided in this application embodiment;

[0031] Figure 8 A schematic diagram of an image of a subject captured from the perspective of a virtual camera, as provided in an embodiment of this application;

[0032] Figure 9 A flowchart illustrating the execution steps of a camera-based image acquisition method provided in this application embodiment;

[0033] Figure 10 Schematic diagrams of different system architectures of the products provided in the embodiments of this application;

[0034] Figure 11 A schematic diagram of a camera-based image acquisition device provided in an embodiment of this application;

[0035] Figure 12 This application provides a schematic diagram of the structure of a server according to an embodiment of the present application.

[0036] Figure 13 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation

[0037] The embodiments of this application will now be described with reference to the accompanying drawings.

[0038] Currently, acquiring images from multiple camera perspectives using 3D vision technology involves: first, selecting and purchasing multiple physical cameras; then, setting up a hardware camera acquisition system on-site using these physical cameras, for example, see [link to example]. Figure 1 The diagram illustrates a hardware camera acquisition system consisting of eight physical cameras in a related technology. Finally, time synchronization of the multiple physical cameras is required to enable simultaneous acquisition of images from multiple camera perspectives. For example, see [reference needed]. Figure 2 The image shown is captured from the perspective of eight physical cameras in a related technology.

[0039] However, research has revealed that the implementation steps in the above method are quite cumbersome and complex. Due to the influence of environmental and hardware factors, it is difficult to guarantee the accuracy of each implementation step. This not only consumes a lot of time and effort but also incurs high hardware costs, greatly reducing the acquisition efficiency and effect of images captured from multiple camera perspectives.

[0040] In view of this, this application proposes a camera-based image acquisition method and related apparatus. The method imports a 3D model of the object to be photographed into a 3D virtual shooting scene to control the position and orientation of the 3D model, and deploys multiple virtual cameras to control their positions and orientations. This eliminates the need for purchasing multiple physical cameras, building a physical hardware camera acquisition system, and synchronizing the physical cameras with the physical cameras. It enables rapid and accurate synchronous acquisition of images from the perspectives of multiple virtual cameras. In other words, this method saves significant time and effort, reduces hardware costs, and improves the efficiency and quality of image acquisition from multiple camera perspectives.

[0041] To facilitate understanding of the technical solution of this application, the camera-based image acquisition method provided in the embodiments of this application will be introduced below in conjunction with actual application scenarios.

[0042] See Figure 3 , Figure 3 This is a schematic diagram illustrating an application scenario of a camera-based image acquisition method provided in an embodiment of this application. Figure 3 The application scenario shown includes a camera acquisition system 301, a 3D reconstruction and rendering system 302, and a 3D display system 303; wherein, the camera acquisition system 301 interacts with the 3D reconstruction and rendering system 302, and the 3D reconstruction and rendering system 302 interacts with the 3D display system 303.

[0043] First, the camera acquisition system 301 pre-creates a 3D virtual shooting scene and constructs a virtual world coordinate system for the 3D virtual shooting scene. Based on this, the camera acquisition system 301 imports the 3D model of the object to be photographed into the 3D virtual shooting scene with the constructed virtual world coordinate system. As an example, the 3D virtual shooting scene is a virtual video conference scene, and the object to be photographed is participant A. In order to acquire images of participant A from multiple camera perspectives, the camera acquisition system 301 first needs to import the 3D model of participant A into the virtual video conference scene with the constructed virtual world coordinate system.

[0044] Secondly, the camera acquisition system 301 determines the target model's position and pose information in the virtual world coordinate system. As an example, based on the above example, by controlling the position and pose of the 3D model in the virtual world coordinate system, the camera acquisition system 301 determines the target model's position and pose information in the virtual world coordinate system.

[0045] Next, the camera acquisition system 301 determines the target camera position information and target camera attitude information of each virtual camera in the virtual world coordinate system based on the target layout of multiple virtual cameras in the 3D virtual shooting scene. As an example, based on the above example, the number of multiple virtual cameras is 6. By controlling the target layout of the 6 virtual cameras in the virtual video conferencing scene to be "surround", the camera acquisition system 301 determines the target camera position information and target camera attitude information of each virtual camera in the virtual world coordinate system based on the "surround" layout of the 6 virtual cameras in the virtual video conferencing scene.

[0046] Then, for each virtual camera, the camera acquisition system 301 acquires images of the object to be photographed from the perspective of the virtual camera, based on the target model's position information, target model's pose information, target camera's position information, and target camera's pose information. As an example, based on the above example, the camera acquisition system 301 acquires images of participant A from the perspective of each virtual camera, based on the target model's position information, target model's pose information, target camera's position information, and target camera's pose information, thus achieving simultaneous acquisition of images of participant A from the perspectives of multiple virtual cameras.

[0047] Finally, the camera acquisition system 301 sends the images of the object to be photographed from the perspectives of multiple virtual cameras to the 3D reconstruction and rendering system 302; the 3D reconstruction and rendering system 302 performs 3D reconstruction and rendering on the images of the object to be photographed from the perspectives of multiple virtual cameras, obtains a 3D reconstructed and rendered image of the object to be photographed, and sends it to the 3D display system 303; the 3D display system 303 displays the object to be photographed in 3D based on the 3D reconstructed and rendered image. As an example, based on the above example, the camera acquisition system 301 sends the images of participant A from the perspectives of multiple virtual cameras to the 3D reconstruction and rendering system 302; the 3D reconstruction and rendering system 302 performs 3D reconstruction and rendering on the images of participant A from the perspectives of multiple virtual cameras, obtains a 3D reconstructed and rendered image of participant A, and sends it to the 3D display system 303; the 3D display system 303 displays participant A in 3D based on the 3D reconstructed and rendered image.

[0048] As can be seen, by importing a 3D model of the object to be photographed into a 3D virtual shooting scene to control the position and posture of the 3D model, and deploying multiple virtual cameras to control the position and posture of the multiple virtual cameras, the steps of purchasing multiple physical cameras, setting up a hardware camera acquisition system on-site, and synchronizing the time of the physical cameras can be eliminated. This allows for the rapid and accurate synchronous acquisition of images from the perspectives of multiple virtual cameras. In other words, this method saves significant time and effort, reduces hardware costs, and improves the efficiency and quality of image acquisition from multiple camera perspectives.

[0049] The camera-based image acquisition method provided in this application can be applied to camera-based image acquisition devices with data processing capabilities, such as servers and terminal devices. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services, but is not limited to these. Terminal devices include, but are not limited to, mobile phones, tablets, computers, smart cameras, smart voice interaction devices, smart home appliances, vehicle terminals, and aircraft, but are not limited to these. The terminal devices and the server can be directly or indirectly connected via wired or wireless communication, which is not restricted herein.

[0050] The camera-based image acquisition method provided in this application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, vehicle scenarios, intelligent transportation, and assisted driving.

[0051] The camera-based image acquisition method provided in this application adopts cloud technology. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or local area network to realize data computation, storage, processing, and sharing.

[0052] Cloud computing is a computing model that distributes computing tasks across a large pool of computers, enabling various application systems to access computing power, storage space, and information services as needed. The network providing these resources is called the "cloud." From the user's perspective, resources in the "cloud" appear infinitely scalable, readily available, on-demand, and expandable, with payment based on usage.

[0053] As a provider of fundamental cloud computing capabilities, a cloud resource pool (referred to as a cloud platform, generally called an IaaS (Infrastructure as a Service) platform) is established. Various types of virtual resources are deployed in the resource pool for external customers to choose from. The cloud resource pool mainly includes: computing devices (virtualized machines containing operating systems), storage devices, and network devices.

[0054] Cloud conferencing is an efficient, convenient, and low-cost form of meeting based on cloud computing technology. Users only need to use an internet interface to quickly and efficiently share voice, data files, and video with teams and clients around the world. The complex technologies such as data transmission and processing during the meeting are handled by the cloud conferencing service provider.

[0055] Currently, domestic cloud conferencing mainly focuses on services based on the SaaS (Software as a Service) model, including telephone, internet, and video services. Video conferencing based on cloud computing is called cloud conferencing.

[0056] In the era of cloud conferencing, data transmission, processing, and storage are all handled by the computer resources of the video conferencing vendors. Users no longer need to purchase expensive hardware or install cumbersome software. They can simply open a browser, log in to the corresponding interface, and conduct efficient remote meetings.

[0057] Cloud conferencing systems support dynamic multi-server cluster deployment and provide multiple high-performance servers, significantly improving meeting stability, security, and availability. In recent years, video conferencing has gained popularity across various fields due to its ability to greatly improve communication efficiency, continuously reduce communication costs, and upgrade internal management. Undoubtedly, with the application of cloud computing, video conferencing will be even more attractive in terms of convenience, speed, and ease of use, inevitably ushering in a new wave of video conferencing applications.

[0058] The following section uses a terminal device or server as a camera-based image acquisition device to specifically describe the camera-based image acquisition method provided in the embodiments of this application.

[0059] See Figure 4 This figure is a flowchart of a camera-based image acquisition method provided in an embodiment of this application. Figure 4 As shown, this camera-based image acquisition method includes the following steps:

[0060] S401: Import the 3D model of the object to be photographed into a 3D virtual shooting scene that constructs a virtual world coordinate system.

[0061] In related technologies, to acquire images from multiple camera perspectives using 3D vision technology, the process involves several steps: first, selecting and purchasing multiple physical cameras; second, constructing a hardware camera acquisition system on-site; and finally, synchronizing the multiple cameras to simultaneously acquire images from their respective perspectives. Research has revealed that these steps are cumbersome and complex, and are limited by environmental and hardware factors, making it difficult to guarantee accuracy. This not only consumes significant time and effort but also incurs high hardware costs, greatly reducing the efficiency and quality of image acquisition from multiple camera perspectives.

[0062] Therefore, in this embodiment, a virtual simulation of reality is considered to avoid the steps of purchasing multiple physical cameras, setting up a hardware camera acquisition system on-site, and synchronizing the time of the physical cameras as described in the related technologies. This requires pre-creating a 3D virtual shooting scene to simulate a real 3D shooting scenario, and constructing a virtual world coordinate system for the 3D virtual shooting scene. To simulate the object to be photographed in the real 3D shooting scene, a 3D model of the object to be photographed needs to be imported into the 3D virtual shooting scene so that the object can be photographed subsequently by shooting the 3D model.

[0063] The 3D model of the object to be photographed needs to include its geometric and material information. The geometric information represents the object's spatial shape, while the material information represents the materials used. For example, the 3D model of the object can be represented by a triangular mesh and corresponding texture maps. The model format can be obj, fbx, or gltf, etc.

[0064] S402: Determine the target model's position and pose information in the virtual world coordinate system.

[0065] In this embodiment of the application, after importing the 3D model of the object to be photographed into the 3D virtual shooting scene that constructs the virtual world coordinate system in S401, in order to simulate the position and posture of the object to be photographed in the 3D real shooting scene, it is necessary to set the position and posture of the 3D model of the object to be photographed in the 3D virtual shooting scene, so as to determine the position information and posture information of the 3D model of the object to be photographed in the virtual world coordinate system, which can be used as the target model position information and target model posture information.

[0066] In the specific implementation of S402, the following two specific implementation methods can be adopted:

[0067] The first specific implementation method: Based on the simulation requirements of the position and posture of the object to be photographed in the 3D real shooting scene, the position and posture of the 3D model of the object to be photographed are directly configured in the 3D virtual shooting scene. Based on this, firstly, the position configuration information and posture configuration information of the 3D model of the object to be photographed in the virtual world coordinate system can be obtained; then, through the position configuration information, the target model position information of the 3D model of the object to be photographed in the virtual world coordinate system can be determined, and through the posture configuration information, the target model posture information of the 3D model of the object to be photographed in the virtual world coordinate system can be determined. Therefore, this application provides a possible implementation method, and S402 may include, for example, the following S4021-S4022:

[0068] S4021: Obtain the position and orientation configuration information of the 3D model in the virtual world coordinate system.

[0069] S4022: Determine the target model's position information and target model's attitude information based on the position configuration information and attitude configuration information.

[0070] The second specific implementation method: In order to make the 3D model of the object to be photographed in the 3D virtual shooting scene more consistent with the diverse motion of the object to be photographed in the 3D real shooting scene, the 3D model of the object to be photographed in the 3D virtual shooting scene can also be spatially transformed to obtain the spatially transformed 3D model of the object to be photographed. Based on this, firstly, the initial model position information, initial model posture information, and spatial transformation information of the 3D model of the object to be photographed in the virtual world coordinate system can be obtained; then, based on the initial model position information and initial model posture information, combined with the spatial transformation information, the target model position information and target model posture information of the 3D model of the object to be photographed in the virtual world coordinate system can be determined. Therefore, this application provides a possible implementation method, and S402 may include, for example, the following S4023-S4024:

[0071] S4023: Obtain the initial model position information, initial model pose information, and spatial transformation information of the 3D model in the virtual world coordinate system.

[0072] S4024: Determine the target model's position information and target model's attitude information based on the initial model's position information, initial model's attitude information, and spatial transformation information.

[0073] In order to simplify calculations and reduce the amount of computation, we can assume that the object to be photographed is a rigid body and perform spatial transformation processing on the three-dimensional model of the object to be photographed. In fact, we perform similar transformation processing on the overall three-dimensional model in space, such as rotation, translation, scaling and other transformation processing.

[0074] Furthermore, in this embodiment of the application, when the object to be photographed has multiple structural parts, in order to accurately calculate and improve the accuracy of the calculation, the object to be photographed can be divided into multiple structural parts. Then, the three-dimensional model of the object to be photographed includes multiple three-dimensional sub-models corresponding to the multiple structural parts of the object to be photographed. Based on this, the three-dimensional model of the object to be photographed can be subjected to spatial transformation processing, which can be different spatial transformation processing on different three-dimensional sub-models. Then, the spatial transformation information of the three-dimensional model of the object to be photographed includes multiple spatial transformation sub-information corresponding to multiple three-dimensional sub-models.

[0075] As an example, the subject to be photographed is participant A. Participant A can be divided into three structural parts: head, torso, and limbs. The 3D model of participant A includes a 3D sub-model corresponding to the head, a 3D sub-model corresponding to the torso, and a 3D sub-model corresponding to the limbs. To simulate the head twisting, torso rotation, and limb swinging of participant A in a realistic 3D shooting scene, different spatial transformations are performed on the 3D sub-models corresponding to the head, torso, and limbs. The spatial transformation information of participant A's 3D model includes multiple spatial transformation sub-information corresponding to the 3D sub-models corresponding to the head, torso, and limbs.

[0076] S403: Based on the target layout of multiple virtual cameras in the 3D virtual shooting scene, determine the target camera position information and target camera attitude information of each virtual camera in the virtual world coordinate system.

[0077] In this embodiment, after executing S402 to determine the target model position information and target model pose information of the 3D model in the virtual world coordinate system, in order to subsequently acquire images from multiple camera perspectives using 3D vision technology, multiple physical cameras are simulated to be built in a real 3D shooting scene. Therefore, multiple virtual cameras need to be deployed in the 3D virtual shooting scene. The deployment method of the multiple virtual cameras determines the position and pose of each virtual camera in the virtual world coordinate system. Thus, by determining the target layout method of the multiple virtual cameras in the 3D virtual shooting scene, the target camera position information and target camera pose information of each virtual camera in the virtual world coordinate system can be determined. The virtual camera can be either a monocular camera or a binocular camera; this embodiment does not impose a specific limitation.

[0078] In the specific implementation of S403, for each virtual camera, firstly, in the 3D virtual shooting scene, the positions of multiple virtual cameras need to be set according to the target layout. Based on the target layout of the multiple virtual cameras in the 3D virtual shooting scene, the target camera position information of the virtual camera in the virtual world coordinate system can be determined. Then, to ensure that the 3D model of the object to be photographed is centered in the virtual camera's view, the positional relationship between the 3D model and the virtual camera needs to be determined using the target model position information of the 3D model of the object to be photographed and the target camera position information of the virtual camera. By setting the pose of the virtual camera according to the positional relationship in the 3D virtual shooting scene, the target camera pose information of the virtual camera in the virtual world coordinate system can be determined. Therefore, this application provides a possible implementation, and S403 may include, for example, the following S4031-S4033:

[0079] S4031: Determine the target camera position information based on the target layout.

[0080] The target layout method can be determined in the following two specific ways:

[0081] The first specific implementation method: Based on the layout requirements for multiple virtual cameras, the layout of multiple virtual cameras is directly configured in the 3D virtual shooting scene. Based on this, firstly, the layout configuration information of multiple virtual cameras in the virtual world coordinate system can be obtained; then, through the layout configuration information, the target layout method of the multiple virtual cameras in the virtual world coordinate system can be determined. Therefore, this application provides a possible implementation method, and the steps for determining the target layout method may include, for example, the following S1-S2:

[0082] S1: Obtain the layout configuration information of multiple virtual cameras in a 3D virtual shooting scene.

[0083] S2: Determine the target layout method based on the layout configuration information.

[0084] The second specific implementation method: To make the layout of multiple virtual cameras in a 3D virtual shooting scene more consistent with the diversified layout of multiple physical cameras in a 3D real shooting scene, the layout of multiple virtual cameras in the 3D virtual shooting scene can be adjusted to obtain multiple virtual cameras with adjusted layout. Based on this, firstly, the initial layout and layout adjustment information of multiple virtual cameras in the virtual world coordinate system can be obtained; then, based on the initial layout and combined with the layout adjustment information, the target layout of multiple virtual cameras in the virtual world coordinate system can be determined. Therefore, this application provides a possible implementation method, and the steps for determining the target layout may include, for example, the following S3-S4:

[0085] S3: Obtain the initial layout and layout adjustment information of multiple virtual cameras in a 3D virtual shooting scene.

[0086] S4: Determine the target layout method based on the initial layout method and layout adjustment information.

[0087] In related technologies, implementing different layouts of multiple physical cameras, such as "surround," "top and bottom," and "corner," requires disassembling and reassembling multiple physical cameras in a physically constructed hardware camera acquisition system. This method is limited by hardware debugging, making the implementation steps cumbersome and complex, and consuming a lot of time. It is also inconvenient to verify the differences and impacts of different layouts of multiple virtual cameras on the captured images from multiple camera perspectives. However, the implementation method of S3-S4 in this application directly adjusts the original layout of multiple virtual cameras in the 3D virtual shooting scene. Without hardware debugging, it can quickly and easily implement different layouts of multiple virtual cameras, efficiently and cost-effectively acquiring captured images from multiple virtual camera perspectives under different layouts of multiple virtual cameras. This facilitates the verification of the differences and impacts of different layouts of multiple virtual cameras on the captured images from multiple camera perspectives.

[0088] As an example, see Figure 5 The diagram illustrates different target layouts of multiple virtual cameras in a 3D virtual shooting scene. The 3D virtual shooting scene is a virtual video conferencing scene, the object to be filmed is a table, and there are six virtual cameras, represented by small squares in the diagram. Figure 5 In the middle (a), it represents a virtual video conferencing scenario where six virtual cameras are arranged in a "top-to-bottom" layout around the screen in front of the table. Figure 5 (b) indicates a "surround" layout of six virtual cameras facing the table around the screen in front of the table in a virtual video conferencing scenario.

[0089] S4032: Determine the positional relationship between the 3D model and the virtual camera based on the target model position information and the target camera position information.

[0090] S4033: Adjust the virtual camera's attitude based on its positional relationship to determine the target camera's attitude information.

[0091] As an example, see Figure 6 This diagram illustrates a method for adjusting the pose of a virtual camera. In the aforementioned... Figure 5Building upon (b), for the virtual camera located on the left side of the screen, to center the 3D model of the table within its view, the positional relationship between the table's 3D model and the virtual camera is determined based on their respective target model position information and the virtual camera's target camera position information. The virtual camera is then rotated by an angle α around the Z-axis towards the table, calculated using the trigonometric functions shown in the diagram. Similarly, the virtual camera located above the screen needs to be rotated by a certain angle around the Y-axis towards the table.

[0092] S404: For each virtual camera, based on the target model position information, target model pose information, target camera position information, and target camera pose information, acquire images of the object to be photographed from the perspective of the virtual camera.

[0093] In this embodiment, after executing S402-S403 to determine the target model position information and target model pose information of the 3D model of the object to be photographed in the virtual world coordinate system, as well as the target camera position information and target camera pose information of each virtual camera, for each virtual camera, the target model position information, target model pose information, target camera position information, and target camera pose information can be used to efficiently, conveniently, and accurately acquire the image of the object to be photographed from the perspective of the virtual camera, providing a foundation for the subsequent high-precision 3D reconstruction and rendering of the object to be photographed.

[0094] In the specific implementation of S404, for each virtual camera, firstly, by using the target model position information and target model pose information of the 3D model of the object to be photographed in the virtual world coordinate system, as well as the target camera position information and target camera pose information of the virtual camera, the distance of the 3D model of the object to be photographed relative to the virtual camera, i.e., the depth of the 3D model of the object to be photographed relative to the virtual camera, can be determined using the target camera position information and target camera pose information of the virtual camera in the virtual world coordinate system. Finally, based on the distance of the 3D model of the object to be photographed relative to the virtual camera and the extrinsic parameters of the virtual camera, combined with the preset intrinsic parameters of the virtual camera, the image under the vision of the virtual camera can be rendered, thereby realizing the acquisition of the captured image under the vision of the virtual camera. Therefore, this application provides a possible implementation method, and S404 may include, for example, the following S4041-S4043:

[0095] S4041: For each virtual camera, determine the distance between the 3D model and the virtual camera based on the target model position information, target model pose information, target camera position information, and target camera pose information.

[0096] In related technologies, the depth of the object to be photographed relative to the virtual camera is estimated based on the time-of-flight principle. However, this method suffers from low measurement accuracy and insufficient depth estimation due to multipath interference. In contrast, the implementation of S4041 in this application uses known information regarding the target model position and pose of the 3D model of the object to be photographed in the virtual world coordinate system, as well as the target camera position and pose of the virtual camera. This allows for precise calculation of the depth of the 3D model of the object relative to the virtual camera, thereby improving depth accuracy.

[0097] S4042: Determine the extrinsic parameters of the virtual camera based on the target camera position information and target camera attitude information.

[0098] As an example, the extrinsic parameters of a virtual camera are determined using the following formula:

[0099]

[0100] Where R represents the pose information of the coordinate axes in the virtual world coordinate system within the virtual camera coordinate system, t represents the position information of the origin of the virtual world coordinate system within the virtual camera coordinate system, and C represents the target camera position information of the virtual camera in the virtual world coordinate system. C This represents the target camera pose information of the virtual camera in the virtual world coordinate system, where I represents the identity matrix.

[0101] In related technologies, extrinsic parameter calibration of a physical camera requires first extracting image features based on Structure from Motion (SfM), estimating the initial extrinsic parameters of the physical camera using feature matching, and then registering the point cloud using the Iterative Closest Point (ICP) algorithm to calculate the final extrinsic parameters of the physical camera. This method suffers from inaccurate feature matching estimation when the object to be photographed has no texture or repetitive texture, leading to estimation errors in the camera's extrinsic parameters. However, in the implementation of S4042 in this application, the target camera position information and target camera pose information in the virtual world coordinate system are both known information, allowing for accurate calculation of the virtual camera's extrinsic parameters and thus avoiding estimation errors in the camera's extrinsic parameters.

[0102] S4043: Captures images based on distance, extrinsic parameters, and preset intrinsic parameters of the virtual camera.

[0103] The preset intrinsic parameters of the virtual camera are known inherent properties of the virtual camera, related to the characteristics of the image sensor and the lens's inherent errors, typically including focal length, principal point offset, and lens distortion. To simulate different physical cameras in a diverse manner, the preset intrinsic parameters of the virtual camera can be arbitrarily adjusted in a 3D virtual shooting scene. This allows for the acquisition of the preset intrinsic parameter adjustment information. By adjusting the preset intrinsic parameters to target intrinsic parameters, and based on the distance between the 3D model of the object to be photographed and the virtual camera's extrinsic parameters, combined with the virtual camera's target intrinsic parameters, the image under the virtual camera's visual perspective can be rendered, thus achieving the acquisition of the captured image from the virtual camera's perspective. Therefore, this application provides a possible implementation method, which may further include the following steps S5-S6:

[0104] S5: Obtain the preset intrinsic parameter adjustment information.

[0105] S6: Adjust the preset intrinsic parameters to the target intrinsic parameters based on the intrinsic parameter adjustment information of the preset intrinsic parameters.

[0106] Correspondingly, S4043 may include, for example, acquiring images based on distance, extrinsic parameters, and target intrinsic parameters.

[0107] Furthermore, in this embodiment of the application, in order to make the captured images of the object to be photographed from the perspective of the virtual camera more diverse, the lighting conditions of the three-dimensional virtual shooting scene can also be configured, for example, configuring the light source and light intensity, to simulate the three-dimensional real shooting scene under different lighting conditions. Based on this, firstly, the lighting configuration information of the three-dimensional virtual shooting scene can be obtained; then, through the lighting configuration information, the target lighting conditions of the three-dimensional virtual shooting scene can be determined so that when executing S404, combined with the target lighting conditions, the captured images of the object to be photographed from the perspective of the virtual camera that meet the target lighting conditions can be acquired. Therefore, this application provides a possible implementation method, and the method may further include the following S7-S8:

[0108] S7: Obtain the lighting configuration information of the 3D virtual shooting scene.

[0109] S8: Determine the target lighting conditions for the 3D virtual shooting scene based on the lighting configuration information.

[0110] Correspondingly, S404 may include, for example, acquiring images that meet the target lighting conditions for each virtual camera, based on the target model position information, target model pose information, target camera position information, target camera pose information, and target lighting conditions.

[0111] As an example, in the above Figures 5-6 Based on this, see Figure 7This diagram illustrates a 3D model of an object to be photographed under different lighting conditions in a 3D virtual shooting scene. Figure 7 (a) represents a 3D model of a table with an added light source in a virtual video conferencing scenario; where, Figure 7 (b) represents a 3D model of a table with two additional light sources in a virtual video conferencing scenario.

[0112] Furthermore, since the phase shift detection based on the time-of-flight principle in related technologies requires multiple sampling integrations, resulting in high resource consumption and low image resolution of the captured images of the object to be photographed, it is not conducive to subsequent high-precision 3D reconstruction and rendering. Therefore, in this embodiment, in order to improve the image attributes of the captured images of the object to be photographed, the image attributes of the captured images can also be configured to facilitate subsequent high-precision 3D reconstruction and rendering. Based on this, firstly, image attribute configuration information of the captured images can be obtained; then, through the image attribute configuration information, the target image attribute information of the captured images can be determined so that when executing S404, combined with the target image attribute information, the captured images of the object to be photographed from the perspective of the virtual camera that conform to the target image attribute information can be acquired. Therefore, this application provides a possible implementation method, which may further include the following S9-S10:

[0113] S9: Obtain image attribute configuration information of the captured image.

[0114] Since the captured images of the object to be photographed are mainly used for subsequent high-precision 3D reconstruction and rendering, it is necessary to ensure the image attributes such as image resolution or image frame rate of the captured images; therefore, the image attribute configuration information may include, for example, image resolution configuration information or image frame rate configuration information.

[0115] S10: Determine the target image attribute information of the captured image based on the image attribute configuration information.

[0116] Correspondingly, S404 may include, for example, acquiring images that conform to the target image attribute information for each virtual camera, based on the target model position information, target model pose information, target camera position information, target camera pose information, and target image attribute information.

[0117] Furthermore, in this embodiment of the application, in order to diversify the background image of the captured image of the object to be photographed, the background image of the captured image can be further rendered to obtain captured images of the object to be photographed under different backgrounds. Based on this, firstly, background rendering information of the captured image can be obtained; then, using the background rendering information, a preset background image of the captured image can be rendered as the target background image. Therefore, this application provides a possible implementation method, which may further include, for example, S11-12:

[0118] S11: Obtain background rendering information of the captured image.

[0119] S12: Based on the background rendering information, render the preset background image of the captured image as the target background image.

[0120] Based on the above explanation, see Figure 8 This diagram illustrates an image of a subject captured from the perspective of a virtual camera. The subject is an open-source Suzanne Monkey model. Figure 8 Image (a) in the middle represents a color image of the open-source monkey head model from the perspective of a virtual camera. Figure 8 In the middle (b), the depth image of the open-source monkey head model is shown from the perspective of the virtual camera.

[0121] The camera-based image acquisition method provided in the above embodiments constructs a virtual world coordinate system for a 3D virtual shooting scene and imports a 3D model of the object to be photographed into the 3D virtual shooting scene. It determines the target model's position and pose information within the virtual world coordinate system. By utilizing the target layout of multiple virtual cameras in the virtual shooting scene, it determines the target camera's position and pose information for each virtual camera within the virtual world coordinate system. For each virtual camera, it acquires an image of the object to be photographed from the virtual camera's perspective using the target model's position and pose information, as well as the target camera's position and pose information. Therefore, by importing a 3D model of the object to be photographed into the 3D virtual shooting scene to control its position and pose, and by deploying multiple virtual cameras to control their positions and poses, it eliminates the need for purchasing multiple physical cameras, building a physical hardware camera acquisition system, and synchronizing the physical cameras. This allows for the rapid and accurate synchronous acquisition of images from multiple virtual camera perspectives. In other words, this method saves significant time and effort, reduces hardware costs, and improves the efficiency and quality of image acquisition from multiple camera perspectives.

[0122] For the camera-based image acquisition method described above, see [link to relevant documentation]. Figure 9 The diagram shows a flowchart illustrating the execution steps of a camera-based image acquisition method. The execution steps are as follows:

[0123] Step 1: Import the 3D model of the object to be photographed into the 3D virtual shooting scene that constructs a virtual world coordinate system.

[0124] Step 2: Set the position and orientation of the 3D model in the virtual world coordinate system.

[0125] Step 3: Set the position and orientation of multiple virtual cameras in the virtual world coordinate system.

[0126] Step 4: Set the lighting conditions for the 3D virtual shooting scene.

[0127] Step 5: Render the image of the object to be photographed from the perspective of each virtual camera.

[0128] Furthermore, the aforementioned camera-based image acquisition method is applied to the camera acquisition system within the product's system architecture; see [link to relevant documentation]. Figure 10 The diagram illustrates different system architectures of the product. The product includes a camera acquisition system, a 3D reconstruction and rendering system, and a 3D display system. The camera acquisition system is deployed at the transmitting end, the 3D reconstruction and rendering system can be deployed at either the transmitting or receiving end, and the 3D display system is deployed at the receiving end. Figure 10 In the middle (a), it indicates that the 3D reconstruction rendering system is deployed at the sending end. Figure 10 In the middle (b), it means that the 3D reconstruction rendering system is deployed at the receiving end.

[0129] The camera acquisition system sends images of the object to be photographed from the perspectives of multiple virtual cameras to the 3D reconstruction and rendering system; the 3D reconstruction and rendering system performs 3D reconstruction and rendering on the images of the object to be photographed from the perspectives of multiple virtual cameras, obtains a 3D reconstructed and rendered image of the object to be photographed, and sends it to the 3D display system; the 3D display system displays the object to be photographed in 3D based on the 3D reconstructed and rendered image of the object to be photographed.

[0130] In view of the camera-based image acquisition method provided in the above embodiments, this application also provides a camera-based image acquisition device.

[0131] See Figure 11 , Figure 11 This is a schematic diagram of a camera-based image acquisition device provided in an embodiment of this application. Figure 11 As shown, the camera-based image acquisition device 1100 includes: an import unit 1101, a determination unit 1102, and an acquisition unit 1103;

[0132] Import unit 1101 is used to import the three-dimensional model of the object to be photographed in a three-dimensional virtual shooting scene that constructs a virtual world coordinate system;

[0133] The determining unit 1102 is used to determine the target model position information and target model pose information of the three-dimensional model in the virtual world coordinate system;

[0134] The determining unit 1102 is also used to determine the target camera position information and target camera attitude information of each virtual camera in the virtual world coordinate system according to the target layout of multiple virtual cameras in the three-dimensional virtual shooting scene.

[0135] The acquisition unit 1103 is used to acquire images of the object to be photographed from the perspective of the virtual camera for each virtual camera, based on the target model position information, target model pose information, target camera position information, and target camera pose information.

[0136] As one possible implementation, the determining unit 1102 is used for:

[0137] Determine the target camera position information based on the target layout;

[0138] Based on the target model's position information and the target camera's position information, determine the positional relationship between the 3D model and the virtual camera;

[0139] The virtual camera's attitude is adjusted based on its positional relationship to determine the target camera's attitude information.

[0140] As one possible implementation, the acquisition unit 1103 is used for:

[0141] For each virtual camera, the distance between the 3D model and the virtual camera is determined based on the target model position information, target model pose information, target camera position information, and target camera pose information.

[0142] The extrinsic parameters of the virtual camera are determined based on the target camera's position and attitude information.

[0143] Images are captured based on distance, extrinsic parameters, and preset intrinsic parameters of the virtual camera.

[0144] As one possible implementation, the determining unit 1102 is also used for:

[0145] Obtain the layout configuration information of multiple virtual cameras in a 3D virtual shooting scene;

[0146] Determine the target layout method based on the layout configuration information.

[0147] As one possible implementation, the determining unit 1102 is also used for:

[0148] Obtain the initial layout and layout adjustment information of multiple virtual cameras in a 3D virtual shooting scene;

[0149] Based on the initial layout method and layout adjustment information, determine the target layout method.

[0150] As one possible implementation, the determining unit 1102 is used for:

[0151] Obtain the position and orientation configuration information of the 3D model in the virtual world coordinate system;

[0152] Based on the position configuration information and attitude configuration information, determine the target model's position information and attitude information.

[0153] As one possible implementation, the determining unit 1102 is used for:

[0154] Obtain the initial model position information, initial model pose information, and spatial transformation information of the 3D model in the virtual world coordinate system;

[0155] Based on the initial model position information, initial model pose information, and spatial transformation information, determine the target model position information and target model pose information.

[0156] As one possible implementation, when the object to be photographed includes multiple structural parts, the 3D model includes multiple 3D sub-models corresponding to the multiple structural parts, and the spatial transformation information includes multiple spatial transformation sub-information corresponding to the multiple 3D sub-models.

[0157] As one possible implementation, the device further includes: a first acquisition unit;

[0158] The first acquisition unit is used to acquire the lighting configuration information of the three-dimensional virtual shooting scene;

[0159] The determining unit 1102 is also used to determine the target lighting conditions of the three-dimensional virtual shooting scene based on the lighting configuration information;

[0160] The acquisition unit 1103 is used to acquire images that meet the target lighting conditions for each virtual camera, based on the target model position information, target model pose information, target camera position information, target camera pose information, and target lighting conditions.

[0161] As one possible implementation, the method also includes: a second acquisition unit;

[0162] The second acquisition unit is used to acquire image attribute configuration information of the captured image;

[0163] The determining unit 1102 is also used to determine the target image attribute information of the captured image based on the image attribute configuration information;

[0164] The acquisition unit 1103 is used to acquire images that conform to the target image attribute information for each virtual camera, based on the target model position information, target model pose information, target camera position information, target camera pose information, and target image attribute information.

[0165] As one possible implementation, the method also includes: a third acquisition unit and a rendering unit;

[0166] The third acquisition unit is used to acquire the background rendering information of the captured image;

[0167] The rendering unit is used to render the preset background image of the captured image into the target background image based on the background rendering information.

[0168] The camera-based image acquisition device provided in the above embodiments constructs a virtual world coordinate system for a 3D virtual shooting scene and imports a 3D model of the object to be photographed into the 3D virtual shooting scene. It determines the target model's position and pose information within the virtual world coordinate system. Through the target layout of multiple virtual cameras in the virtual shooting scene, it determines the target camera's position and pose information for each virtual camera in the virtual world coordinate system. For each virtual camera, it acquires an image of the object to be photographed from the virtual camera's perspective using the target model's position and pose information, as well as the target camera's position and pose information. Therefore, by importing a 3D model of the object to be photographed into the 3D virtual shooting scene to control its position and pose, and by deploying multiple virtual cameras to control their positions and poses, it eliminates the need to purchase multiple physical cameras, build a physical hardware camera acquisition system, and synchronize the physical cameras. This allows for the rapid and accurate synchronous acquisition of images from multiple virtual camera perspectives. In other words, this method saves significant time and effort, reduces hardware costs, and improves the efficiency and quality of image acquisition from multiple camera perspectives.

[0169] In addition to the camera-based image acquisition method described above, this application also provides a camera-based image acquisition device to enable the above-mentioned camera-based image acquisition method to be implemented and applied in practice. The computer device provided in this application will be described below from the perspective of hardware physicalization.

[0170] See Figure 12 , Figure 12 This is a schematic diagram of a server structure provided in an embodiment of this application. The server 1200 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 1222 (e.g., one or more processors) and memory 1232, and one or more storage media 1230 (e.g., one or more mass storage devices) for storing application programs 1242 or data 1244. The memory 1232 and storage media 1230 can be temporary or persistent storage. The program stored in the storage media 1230 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the server. Furthermore, the CPU 1222 may be configured to communicate with the storage media 1230 and execute the series of instruction operations in the storage media 1230 on the server 1200.

[0171] Server 1200 may also include one or more power supplies 1226, one or more wired or wireless network interfaces 1250, one or more input / output interfaces 1258, and / or one or more operating systems 1241, such as Windows Server. TM Mac OS X TM Unix TM Linux TM FreeBSD TM etc.

[0172] The steps performed by the server in the above embodiments can be based on this Figure 12 The server structure shown.

[0173] CPU 1222 is used to perform the following steps:

[0174] Import the 3D model of the object to be photographed into the 3D virtual shooting scene that constructs a virtual world coordinate system;

[0175] Determine the target model's position and orientation information in the virtual world coordinate system;

[0176] Based on the target layout of multiple virtual cameras in a 3D virtual shooting scene, determine the target camera position information and target camera attitude information of each virtual camera in the virtual world coordinate system;

[0177] For each virtual camera, images of the object to be photographed are acquired from the virtual camera's perspective, based on the target model's position information, target model's pose information, target camera's position information, and target camera's pose information.

[0178] Optionally, the CPU 1222 may also execute method steps of any specific implementation of the camera-based image acquisition method in the embodiments of this application.

[0179] See Figure 13 , Figure 13 This is a schematic diagram of a terminal device provided in an embodiment of this application. For ease of explanation, only the parts related to the embodiment of this application are shown; for specific technical details not disclosed, please refer to the method section of the embodiment of this application. The terminal device can be any terminal device including mobile phones, tablets, PDAs, etc. Taking a mobile phone as an example:

[0180] Figure 13 This diagram illustrates a partial structural representation of a mobile phone related to the terminal device provided in this embodiment. (Reference) Figure 13The mobile phone includes components such as a radio frequency (RF) circuit 1310, a memory 1320, an input unit 1330, a display unit 1340, a sensor 1350, an audio circuit 1360, a Wi-Fi module 1370, a processor 1380, and a power supply 1390. Those skilled in the art will understand that... Figure 13 The mobile phone structure shown does not constitute a limitation on the mobile phone and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0181] The following is combined Figure 13 A detailed introduction to each component of a mobile phone:

[0182] RF circuit 1310 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with processor 1380; additionally, it transmits uplink data to the base station. Typically, RF circuit 1310 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), and a duplexer. Furthermore, RF circuit 1310 can also communicate wirelessly with networks and other devices. The aforementioned wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, and Short Messaging Service (SMS).

[0183] The memory 1320 can be used to store software programs and modules. The processor 1380 runs the software programs and modules stored in the memory 1320 to realize various functions and data processing of the mobile phone. The memory 1320 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 1320 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0184] The input unit 1330 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the mobile phone. Specifically, the input unit 1330 may include a touch panel 1331 and other input devices 1332. The touch panel 1331, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 1331), and drive the corresponding connection devices according to a pre-set program. Optionally, the touch panel 1331 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 1380, and can also receive and execute commands sent by the processor 1380. In addition, the touch panel 1331 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1331, the input unit 1330 may also include other input devices 1332. Specifically, other input devices 1332 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0185] The display unit 1340 can be used to display information input by the user or information provided to the user, as well as various menus of the mobile phone. The display unit 1340 may include a display panel 1341, which may optionally be configured as a Liquid Crystal Display (LCD), Organic Light-Emitting Diode (OLED), or similar display panel 1341. Further, a touch panel 1331 may cover the display panel 1341. When the touch panel 1331 detects a touch operation on or near it, it transmits the information to the processor 1380 to determine the type of touch event. Subsequently, the processor 1380 provides corresponding visual output on the display panel 1341 based on the type of touch event. Although in Figure 13 In this embodiment, the touch panel 1331 and the display panel 1341 are two separate components to realize the input and output functions of the mobile phone. However, in some embodiments, the touch panel 1331 and the display panel 1341 can be integrated to realize the input and output functions of the mobile phone.

[0186] The mobile phone may also include at least one sensor 1350, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 1341 according to the ambient light level, and the proximity sensor can turn off the display panel 1341 and / or the backlight when the phone is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, tapping), etc. Other sensors that may be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0187] Audio circuit 1360, speaker 1361, and microphone 1362 provide an audio interface between the user and the mobile phone. Audio circuit 1360 converts received audio data into electrical signals and transmits them to speaker 1361, where speaker 1361 converts them into sound signals for output. On the other hand, microphone 1362 converts collected sound signals into electrical signals, which are received by audio circuit 1360, converted into audio data, and then processed by processor 1380 before being transmitted via RF circuit 1310 to, for example, another mobile phone, or the audio data can be output to memory 1320 for further processing.

[0188] WiFi is a short-range wireless transmission technology. Mobile phones, through the WiFi module 1370, can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 13 WiFi module 1370 is shown, but it is understood that it is not an essential component of a mobile phone and can be omitted as needed without changing the essence of the invention.

[0189] The processor 1380 is the control center of the mobile phone, connecting various parts of the phone through various interfaces and lines. It performs various functions and processes data by running or executing software programs and / or modules stored in the memory 1320 and calling data stored in the memory 1320, thereby controlling the phone as a whole. Optionally, the processor 1380 may include one or more processing units; preferably, the processor 1380 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 1380.

[0190] The mobile phone also includes a power supply 1390 (such as a battery) that supplies power to various components. Preferably, the power supply can be logically connected to the processor 1380 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0191] Although not shown, mobile phones may also include a camera, Bluetooth module, etc., which will not be described in detail here.

[0192] In this embodiment of the application, the memory 1320 included in the mobile phone can store program code and transmit the program code to the processor.

[0193] The processor 1380 included in the mobile phone can execute the camera-based image acquisition method provided in the above embodiments according to the instructions in the program code.

[0194] This application also provides a computer-readable storage medium for storing a computer program for executing the camera-based image acquisition method provided in the above embodiments.

[0195] This application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the camera-based image acquisition method provided in the various optional implementations of the above aspects.

[0196] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium can be at least one of the following media: read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc., and other media capable of storing program code.

[0197] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0198] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A camera-based image acquisition method, characterized in that, The method includes: Import the 3D model of the object to be photographed into the 3D virtual shooting scene that constructs a virtual world coordinate system; Determine the target model's position information and target model's pose information in the virtual world coordinate system; Based on the target layout of multiple virtual cameras in the three-dimensional virtual shooting scene, determine the target camera position information of each virtual camera in the virtual world coordinate system; Based on the target model position information and the target camera position information, the positional relationship between the 3D model and the virtual camera is determined; The virtual camera is adjusted according to the positional relationship to determine the target camera posture information of each virtual camera in the virtual world coordinate system. The target layout method includes a wraparound or top-bottom layout. For each virtual camera, images of the object to be photographed are simultaneously acquired from the perspective of the virtual camera, based on the target model position information, the target model pose information, the target camera position information, and the target camera pose information.

2. The method according to claim 1, characterized in that, For each virtual camera, based on the target model position information, the target model pose information, the target camera position information, and the target camera pose information, the process of acquiring a captured image of the object to be photographed from the perspective of the virtual camera includes: For each virtual camera, the distance between the 3D model and the virtual camera is determined based on the target model position information, the target model pose information, the target camera position information, and the target camera pose information. The extrinsic parameters of the virtual camera are determined based on the target camera position information and the target camera attitude information. The captured image is acquired based on the distance, the extrinsic parameters, and the preset intrinsic parameters of the virtual camera.

3. The method according to any one of claims 1-2, characterized in that, The steps for determining the target layout include: Obtain the layout configuration information of multiple virtual cameras in the three-dimensional virtual shooting scene; The target layout method is determined based on the layout configuration information.

4. The method according to any one of claims 1-2, characterized in that, The steps for determining the target layout include: Obtain the initial layout and layout adjustment information of multiple virtual cameras in the three-dimensional virtual shooting scene; The target layout is determined based on the initial layout and the layout adjustment information.

5. The method according to any one of claims 1-2, characterized in that, Determining the target model's position information and pose information in the virtual world coordinate system includes: Obtain the position and orientation configuration information of the 3D model in the virtual world coordinate system; Based on the location configuration information and the attitude configuration information, the target model's location information and attitude information are determined.

6. The method according to any one of claims 1-2, characterized in that, Determining the target model's position information and pose information in the virtual world coordinate system includes: Obtain the initial model position information, initial model pose information, and spatial transformation information of the three-dimensional model in the virtual world coordinate system; Based on the initial model position information, the initial model pose information, and the spatial transformation information, the target model position information and the target model pose information are determined.

7. The method according to claim 6, characterized in that, When the object to be photographed includes multiple structural parts, the three-dimensional model includes multiple three-dimensional sub-models corresponding to the multiple structural parts, and the spatial transformation information includes multiple spatial transformation sub-information corresponding to the multiple three-dimensional sub-models.

8. The method according to claim 1, characterized in that, The method further includes: Obtain the lighting configuration information of the three-dimensional virtual shooting scene; Based on the lighting configuration information, the target lighting conditions of the three-dimensional virtual shooting scene are determined; For each virtual camera, based on the target model position information, the target model pose information, the target camera position information, and the target camera pose information, the process of acquiring a captured image of the object to be photographed from the perspective of the virtual camera includes: For each virtual camera, images that meet the target lighting conditions are captured based on the target model position information, the target model pose information, the target camera position information, the target camera pose information, and the target lighting conditions.

9. The method according to claim 1, characterized in that, The method further includes: Obtain the image attribute configuration information of the captured image; Based on the image attribute configuration information, determine the target image attribute information of the captured image; For each virtual camera, based on the target model position information, the target model pose information, the target camera position information, and the target camera pose information, the process of acquiring a captured image of the object to be photographed from the perspective of the virtual camera includes: For each virtual camera, images conforming to the target image attribute information are acquired based on the target model position information, the target model pose information, the target camera position information, the target camera pose information, and the target image attribute information.

10. The method according to claim 1, characterized in that, The method further includes: Obtain the background rendering information of the captured image; Based on the background rendering information, the preset background image of the captured image is rendered as the target background image.

11. A camera-based image acquisition device, characterized in that, The device includes: an import unit, a determination unit, and a data acquisition unit; The import unit is used to import the three-dimensional model of the object to be photographed into a three-dimensional virtual shooting scene that constructs a virtual world coordinate system. The determining unit is used to determine the target model position information and target model pose information of the three-dimensional model in the virtual world coordinate system; The determining unit is further configured to determine the target camera position information and target camera pose information of each virtual camera in the virtual world coordinate system according to the target layout method of multiple virtual cameras in the three-dimensional virtual shooting scene, wherein the target layout method includes a surrounding type or an up-down type; The acquisition unit is used to simultaneously acquire images of the object to be photographed from the perspective of the virtual camera for each virtual camera, based on the target model position information, the target model pose information, the target camera position information, and the target camera pose information. The determining unit is used for: Determine the target camera position information based on the target layout; Based on the target model's position information and the target camera's position information, determine the positional relationship between the 3D model and the virtual camera; The virtual camera's attitude is adjusted based on its positional relationship to determine the target camera's attitude information.

12. The apparatus according to claim 11, characterized in that, The acquisition unit is used for: For each virtual camera, the distance between the 3D model and the virtual camera is determined based on the target model position information, the target model pose information, the target camera position information, and the target camera pose information. The extrinsic parameters of the virtual camera are determined based on the target camera position information and the target camera attitude information. The captured image is acquired based on the distance, the extrinsic parameters, and the preset intrinsic parameters of the virtual camera.

13. The apparatus according to any one of claims 11-12, characterized in that, The determining unit is further configured to: Obtain the layout configuration information of multiple virtual cameras in the three-dimensional virtual shooting scene; The target layout method is determined based on the layout configuration information.

14. The apparatus according to any one of claims 11-12, characterized in that, The determining unit is further configured to: Obtain the initial layout and layout adjustment information of multiple virtual cameras in the three-dimensional virtual shooting scene; The target layout is determined based on the initial layout and the layout adjustment information.

15. A computer device, characterized in that, The device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the camera-based image acquisition method according to any one of claims 1-10 according to the instructions in the program code.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program, which, when executed by a processor, performs the camera-based image acquisition method according to any one of claims 1-10.

17. A computer program product, characterized in that, Includes a computer program or instructions; when the computer program or instructions are executed by a processor, the camera-based image acquisition method according to any one of claims 1-10 is performed.

Images