Three-dimensional reconstruction method and device, electronic equipment, storage medium and program product

Abstract

The invention provides a three-dimensional reconstruction method and device, electronic equipment, a storage medium and a program product. The three-dimensional reconstruction method comprises the steps that shot images of a detection object in different view fields are acquired; based on a conversion relation, position information included in the shot image is processed, multiple pieces of reconstruction position information are obtained, and the conversion relation is determined according to the view field range of image acquisition equipment used for shooting the image; and performing three-dimensional reconstruction on the detection object based on the plurality of pieces of reconstruction position information.

Description

Technical Field

[0001] The present disclosure relates to the fields of image processing, point cloud processing and industrial technology, and more specifically, to a three-dimensional reconstruction method, device, electronic device, storage medium and program product. Background Art

[0002] With the development of industrial technology, 3D model data is increasingly used in various industrial fields. 3D reconstruction refers to the process of restoring the geometric structure of a 3D scene or detected object from captured images or other observation data. The goal of 3D reconstruction is to create a 3D model, which can be a wireframe model, a surface model, or a voxel model.

[0003] Therefore, how to achieve more effective three-dimensional reconstruction of the detection object to ensure the accuracy of the obtained three-dimensional model data is an urgent problem to be solved. Summary of the invention

[0004] In view of this, the present disclosure provides a 3D reconstruction method, device, electronic device, storage medium and program product.

[0005] According to one aspect of the present disclosure, a three-dimensional reconstruction method is provided, comprising: acquiring captured images of a detection object under multiple fields of view; processing position information included in the captured images based on a conversion relationship to obtain multiple reconstructed position information, wherein the conversion relationship is determined based on the field of view range of an image acquisition device used for the captured images; and, based on the multiple reconstructed position information, performing three-dimensional reconstruction of the detection object.

[0006] According to another aspect of the present disclosure, a three-dimensional reconstruction device is provided, including: an acquisition module, used to acquire captured images of a detection object under multiple fields of view; a processing module, used to process position information included in the captured images based on a conversion relationship to obtain multiple reconstructed position information, wherein the conversion relationship is determined according to the field of view range of an image acquisition device used for the captured images; and a three-dimensional reconstruction module, used to perform three-dimensional reconstruction of the detection object based on the multiple reconstructed position information.

[0007] According to another aspect of the present disclosure, an electronic device is provided, comprising: one or more processors; and a memory for storing one or more instructions, wherein when the one or more instructions are executed by the one or more processors, the one or more processors implement the method described in the present disclosure.

[0008] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which executable instructions are stored. When the executable instructions are executed by a processor, the processor implements the method described in the present disclosure.

[0009] According to another aspect of the present disclosure, a computer program product is provided. The computer program product includes computer executable instructions. When the computer executable instructions are executed, they are used to implement the method described in the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0011] Figure 1 The system architecture to which the three-dimensional reconstruction method can be applied according to an embodiment of the present disclosure is schematically shown;

[0012] Figure 2 A flowchart of a three-dimensional reconstruction method according to an embodiment of the present disclosure is schematically shown;

[0013] Figure 3 The schematic diagram schematically shows an example of a process of processing the position information included in the captured image based on the conversion relationship to obtain the reconstructed position information according to an embodiment of the present disclosure;

[0014] Figure 4A to Figure 4C An exemplary schematic diagram schematically illustrates a first projection angle according to an embodiment of the present disclosure;

[0015] Figure 5A An example schematic diagram of a data acquisition system according to an embodiment of the present disclosure is schematically shown;

[0016] Figure 5B An example schematic diagram of a process of performing three-dimensional reconstruction on a detection object based on multiple reconstruction position information according to an embodiment of the present disclosure is schematically shown;

[0017] Figure 6 A block diagram schematically shows a three-dimensional reconstruction device according to an embodiment of the present disclosure; and

[0018] Figure 7 A block diagram of an electronic device suitable for implementing a three-dimensional reconstruction method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0019] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0020] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0021] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0022] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0023] In the technical solution of the present invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0024] In one example, the three-dimensional reconstruction method may include at least one of the following: a three-dimensional reconstruction method based on visual geometry and a three-dimensional reconstruction method based on deep learning.

[0025] The three-dimensional reconstruction method based on visual geometry refers to estimating the camera pose of the collected data through multi-field-of-view images, and then extracting features from the images and performing comparison and stitching to achieve the conversion from two-dimensional images to three-dimensional models.

[0026] The 3D reconstruction method based on deep learning refers to the use of the learning and fitting capabilities of deep neural networks to achieve the conversion from two-dimensional images to three-dimensional models.

[0027] However, since the three-dimensional model data obtained based on the above method only includes three-dimensional position information but not texture information, that is, the accuracy of the three-dimensional model data is relatively limited, how to more effectively realize three-dimensional reconstruction of the detection object to ensure the accuracy of the obtained three-dimensional model data is an urgent problem to be solved.

[0028] To this end, the present disclosure provides a three-dimensional reconstruction method including: acquiring captured images of a detection object under multiple fields of view; processing position information included in the captured images based on a conversion relationship to obtain multiple reconstructed position information, wherein the conversion relationship is determined according to the field of view range of an image acquisition device used to capture the image; and, based on the multiple reconstructed position information, performing three-dimensional reconstruction of the detection object.

[0029] Figure 1 The system architecture to which the 3D reconstruction method can be applied according to the embodiment of the present disclosure is schematically shown. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0030] like Figure 1 As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104 and a server 105. The network 104 is a medium for providing a communication link between different devices.

[0031] It should be noted that the 3D reconstruction method provided in the embodiment of the present disclosure can generally be executed by the server 105 . Accordingly, the 3D reconstruction device provided in the embodiment of the present disclosure can generally be set in the server 105 .

[0032] Alternatively, the 3D reconstruction method provided in the embodiment of the present disclosure may also be executed by the first terminal device 101, the second terminal device 102 or the third terminal device 103. Accordingly, the 3D reconstruction apparatus provided in the embodiment of the present disclosure may also be disposed in the first terminal device 101, the second terminal device 102 or the third terminal device 103.

[0033] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0034] It should be noted that the sequence numbers of the operations in the following method are only used as representations of the operations for the purpose of description, and should not be regarded as representing the execution order of the operations. Unless explicitly stated, the method does not need to be executed completely in the order shown.

[0035] The above describes the system architecture that can be used for the three-dimensional reconstruction method provided by the present disclosure. Figure 2 For example, the three-dimensional reconstruction process of the present disclosure is further described.

[0036] Figure 2 The flowchart of the three-dimensional reconstruction method according to the embodiment of the present disclosure is schematically shown.

[0037] like Figure 2 As shown, the 3D reconstruction method 200 includes operations S210 to S230.

[0038] In operation S210, captured images of the detection object in multiple fields of view are acquired.

[0039] In operation S220, the position information included in the captured image is processed based on the conversion relationship to obtain a plurality of reconstructed position information, wherein the conversion relationship is determined according to the field of view range of the image acquisition device used to capture the image.

[0040] In operation S230 , three-dimensional reconstruction is performed on the detection object based on the plurality of reconstruction position information.

[0041] The detection object may refer to a target or object that needs to be paid attention to during the three-dimensional reconstruction process. The acquisition method of the captured image can be configured according to actual business needs and is not limited here. For example, the acquisition method of the captured image may include at least one of the following: an acquisition method based on a digital camera, an acquisition method based on scanning, and an acquisition method based on video capture.

[0042] A captured image may refer to a two-dimensional image or photo captured by a camera or a camera, and the captured image may include data consisting of a series of position information on the surface of the target or object. The captured image includes multiple points. The position information may represent the position of each point in the two-dimensional plane relative to a certain reference origin. For example, the position information may include coordinate values ​​corresponding to the X coordinate and the Y coordinate.

[0043] After obtaining the captured image, the position information included in the captured image can be processed based on the conversion relationship to obtain multiple reconstructed position information. The reconstructed position information can refer to the position in three-dimensional space. The method for obtaining the reconstructed position information can be configured according to actual business needs and is not limited here.

[0044] For example, the method of processing the position information to obtain the reconstructed position information may include at least one of the following: a determination method based on depth estimation, a determination method based on feature matching, and a determination method based on bundle adjustment. The determination method based on depth estimation refers to the use of monocular vision technology to estimate depth information from images captured under multiple fields of view. The determination method based on feature matching refers to identifying and matching feature points in images captured under different fields of view, and using triangulation to determine the position in three-dimensional space. The determination method based on bundle adjustment may refer to combining images captured under multiple fields of view and known camera parameters, and using bundle adjustment to obtain reconstructed position information.

[0045] In one example, the conversion relationship may be determined according to the field of view of an image acquisition device used to capture the image. The field of view may refer to the observation angle or field of view of the image acquisition device when acquiring the captured image. In this case, based on the field of view of the image acquisition device, an affine transformation, a perspective transformation, or other geometric transformation may be performed on the position information to correct the position information and obtain the reconstructed position information.

[0046] After obtaining multiple reconstruction position information, the detection object can be reconstructed in three dimensions based on the multiple reconstruction position information, so as to utilize the information of multiple fields of view to perform three-dimensional reconstruction on the detection object. Three-dimensional reconstruction refers to the process of restoring a three-dimensional spatial structure from a two-dimensional captured image.

[0047] The shape of the detection object can be configured according to actual business needs and is not limited here. For example, the shape of the detection object may include at least one of the following: a sphere, a cube, a cylinder, a cone, a polyhedron, and an irregular shape. In one example, the spherical detection object may include a workpiece with a spherical or spherical shape, which may be a complete sphere or a partial sphere, such as a spherical surface and a hemispherical surface.

[0048] According to the embodiments of the present disclosure, by acquiring captured images from multiple fields of view, more spatial information can be obtained, thereby improving the accuracy of subsequent three-dimensional reconstruction, reducing information loss due to field of view limitations, and enabling subsequent three-dimensional reconstruction results to be close to the real surface of the detection object in all directions. By using the field of view range of the image acquisition device to establish a conversion relationship, and processing the position information of each captured image based on the conversion relationship, it can adapt to the needs of different scenes. On this basis, by performing three-dimensional reconstruction of the detection object based on multiple reconstructed position information, the three-dimensional reconstruction effect and the accuracy of the obtained three-dimensional model are improved.

[0049] In the embodiment of the present disclosure, the conversion relationship can be determined according to the field of view range of the image acquisition device used to capture the image. Figure 3The process of processing the position information included in the captured image to obtain the reconstructed position information is further described based on the conversion relationship determined based on the field of view range of the image acquisition device used to capture the image.

[0050] Figure 3 The diagram schematically shows an example of a process of processing position information included in a captured image based on a conversion relationship to obtain reconstructed position information according to an embodiment of the present disclosure.

[0051] In 300, taking the example that the conversion relationship is determined according to the field of view of the image acquisition device used to capture the image, the process of obtaining the reconstructed position information based on the position information is explained. The reconstruction process refers to the process of converting the position coordinates in the two-dimensional plane into the position coordinates in the three-dimensional space.

[0052] like Figure 3 As shown, according to the relationship between longitude and latitude coordinates and Cartesian coordinates, the field of view of the image acquisition device in the width direction is , the field of view of the image acquisition device in the height direction is , the image resolution of the captured image 301 is , the position information of any point A0 in the captured image 301 is ,but The value range is , The value range is .

[0053] In one example, the first projection angle can be determined based on the position information and the field of view of the image acquisition device. The first projection angle refers to the direction of projection from the position of the image acquisition device to the point on the two-dimensional plane to the three-dimensional space, and the first projection angle can describe the angle between the projection light and the target. The first projection angle may include an azimuth and a pitch angle. The pitch angle may refer to the angle between the positive direction of the Z axis to the position of the three-dimensional position information in the three-dimensional space and the XOY plane, usually varying between -π / 2 and π / 2, and the pitch angle may be used to describe the height or depth of the three-dimensional position information relative to the XOY plane. The azimuth angle may refer to the angle between the positive direction of the Z axis to the projection of the three-dimensional position information on the XOY plane and the positive direction of the Z axis, usually varying between 0 and 2π, and the azimuth angle may be used to describe the position of the three-dimensional position information on the horizontal plane.

[0054] In a specific example, according to the position information and the field of view of the image acquisition device, the process of determining the first projection angle can be shown in the following equations (1) and (2).

[0055] #timg# （1） #timg# （2）

[0056] in, Characterizing the height information in the image resolution of the captured image 301, The coordinate value corresponding to the y-axis in the position information, Characterizes the pitch angle, Characterizing width information in the image resolution of the captured image 301, The coordinate value corresponding to the x-axis in the position information, Characterize the azimuth.

[0057] After obtaining the first projection angle, the preset projection radius (Projection Radius) can be combined to determine the reconstructed position information according to the first projection angle. The preset projection radius refers to the distance from the center of the object to the spherical surface of the object. The preset projection radius can determine the scale of the projection in the three-dimensional space and the distance of the projection from the image acquisition device. The first projection angle (Projection Angle) and the preset projection radius are parameters used when reconstructing two-dimensional coordinates into three-dimensional coordinates, which define how to project points on a two-dimensional plane into three-dimensional space. For example, the position information of any point A0 in the captured image 301 The reconstructed position information of point A1 in the corresponding three-dimensional space .

[0058] In a specific example, according to the first projection angle and the preset projection radius, the process of determining the reconstructed position information can be shown in the following formulas (3) to (5).

[0059] #timg# （3） #timg# （4） #timg# （5）

[0060] in, Characterizes the preset projection radius, Represents the coordinate value of the reconstructed position information on the X-axis, Represents the coordinate value of the reconstructed position information in the height direction, Represents the coordinate value of the reconstructed position information on the Z axis.

[0061] After obtaining the reconstructed position information corresponding to each point A0 in the captured image 301, the detection object can be three-dimensionally reconstructed based on multiple reconstructed position information to obtain a reconstructed detection object 302. For example, in an application scenario where the detection object is a sphere, the captured image 301 may include a set of position information corresponding to the detection object obtained by using an image acquisition device, and correspondingly, the reconstructed detection object 302 may include a set of reconstructed position information of the sphere corresponding to the detection object.

[0062] According to the embodiments of the present disclosure, by combining the position information and the field of view of the image acquisition device, it is possible to better adapt to the visual characteristics of different scenes and ensure the accuracy of the first projection angle. On this basis, by flexibly determining the reconstructed position information according to the changed first projection angle and projection radius, the accuracy of the reconstructed position information is improved.

[0063] The above describes how to process the position information included in the captured image based on the conversion relationship to obtain the reconstructed position information. Figure 4A to Figure 4C The method of obtaining the first projection angle is further described.

[0064] Figure 4A to Figure 4C An example schematic diagram of a first projection angle according to an embodiment of the present disclosure is schematically shown.

[0065] like Figure 4A As shown, in an ideal application scenario where the detection object is a sphere, the angle between the points corresponding to two adjacent position information is uniform and unchanged.

[0066] like Figure 4B As shown, in an application scenario where the detection object is a sphere in actual situations, the angle between points corresponding to two adjacent position information increases as the absolute value of the coordinate value of the Y axis increases.

[0067] like Figure 4C As shown, in the application scenario where the detection object is a sphere, the distance between two adjacent position information can be determined according to the number of position information and the field of view of the image acquisition device. Since the position information is a point in a two-dimensional plane, the Euclidean distance formula can be used to calculate the distance between two adjacent position information. On this basis, the first projection distance can be determined according to the distance between the two adjacent position information. On this basis, the first projection angle is determined according to the first projection distance and the preset projection radius.

[0068] In a specific example, according to the position information and the field of view of the image acquisition device, the process of determining the first projection angle can be shown in the following formula (6).

[0069] #timg# （6）

[0070] in, Represents the distance between two adjacent position information. The number of position information, Characterizes the projection distance.

[0071] According to the embodiments of the present disclosure, the first projection distance is determined based on the distance between two adjacent position information. Since the distance between two adjacent position information is determined based on the number of position information and the field of view of the image acquisition device, the relative position of the detection object in the actual environment can be judged more accurately. On this basis, the first projection angle is determined by combining the first projection distance and the preset projection radius, and the first projection angle can be adapted to the change in the height direction by fitting the change law of the first projection angle, thereby improving the accuracy of the subsequent reconstruction of the position information.

[0072] The above describes the method of processing position information based on the conversion relationship. Figure 5A The process of acquiring images captured under different fields of view is further explained.

[0073] Figure 5A An example schematic diagram of a data acquisition system according to an embodiment of the present disclosure is schematically shown.

[0074] like Figure 5A As shown, the data acquisition system may include an image acquisition device 501, a rotation device 503, and a clamping device 504. The field of view may be determined according to an angle of the image acquisition device 501 relative to the detection object 505.

[0075] The detection object 505 can be placed on the rotating device 503 via the clamping device 504. The rotating device 503 can be used to change the rotation direction of the detection object 505. The clamping device 504 can be used to change the flipping direction of the detection object 505. Changing the rotation direction by the rotating device 503 and changing the flipping direction by the clamping device 504 is equivalent to changing the angle of the image acquisition device 501 relative to the detection object 505.

[0076] During the image acquisition process, the rotation device 503 and the clamping device 504 can be used to change the field of view, and the image acquisition device 501 can be used to acquire images of the detection object 505 in multiple fields of view to obtain captured images. For example, in an application scenario where the detection object is a sphere, the image acquisition device 501 can be used to acquire captured images of the detection object 505.

[0077] According to the embodiments of the present disclosure, the angle of the image acquisition device relative to the detection object is utilized to ensure the acquisition of different fields of view. By capturing images based on the acquired different fields of view, the detection object can be analyzed in an all-round manner, which helps to ensure the comprehensiveness and accuracy of the subsequent three-dimensional reconstruction effect.

[0078] The above describes how to obtain the captured image. Figure 5B The process of performing three-dimensional reconstruction of the detected object based on multiple reconstruction position information is described.

[0079] Figure 5B An example schematic diagram of a process of performing three-dimensional reconstruction on a detection object based on multiple reconstruction position information according to an embodiment of the present disclosure is schematically shown.

[0080] like Figure 5B As shown, when the detection object is a sphere, in the process of stitching a complete sphere, since the minimum number of acquisitions required is related to the field of view range of the image acquisition device 501, the field of view range of the image acquisition device 501 is The 3D reconstruction process is explained by taking the minimum number of acquisitions as 3 as an example.

[0081] For example, an image acquisition device 501 is used to acquire an image 506 of a spherical detection object at a viewing angle of 0 degrees. After the detection object 505 is flipped 60 degrees by the clamping device 504, an image 507 of the spherical detection object at a viewing angle of 60 degrees is acquired by the image acquisition device 501. After the detection object 505 is flipped 60 degrees by the clamping device 504, an image 508 of the spherical detection object at a viewing angle of 120 degrees is acquired by the image acquisition device 501. After obtaining the image 506 at a viewing angle of 0 degrees, the image 507 at a viewing angle of 60 degrees, and the image 508 at a viewing angle of 120 degrees, a complete sphere can be obtained based on the three images.

[0082] In one example, multiple reconstructed position information under the field of view can be processed based on the angle corresponding to the field of view to obtain annular point cloud data under the field of view. The annular point cloud data can refer to a set of point cloud data arranged in a ring in a plane or space. For example, the multiple reconstructed position information can be rotated according to the angle between the clamping device 504 and the image acquisition device 501 to obtain the transformed initial annular point cloud data under the field of view. On this basis, the initial annular point cloud data can be flipped according to the flipping angle between the images to obtain the annular point cloud data under the field of view.

[0083] In another example, the annular point cloud data in each field of view may be spliced ​​based on the rotation matrix determined by the annular point cloud data in the field of view and the annular point cloud data in other fields of view to obtain the target point cloud data. The splicing process of the annular point cloud data may include a coarse matching process and a fine matching process. The coarse matching process may be used to obtain an initial pose, and the fine matching process may be used to improve the registration accuracy.

[0084] For example, the rough matching process may refer to directly splicing the annular point cloud data under multiple fields of view to obtain the initial point cloud data, wherein there are overlapping areas between the annular point cloud data under the multiple fields of view.

[0085] Alternatively, the fine matching process may refer to performing fine matching based on the overlapped area features between the initial point cloud data obtained by the rough matching based on the preset algorithm, and then tuning the parameters of the preset algorithm for the spherical point cloud data after the fine matching to achieve the best matching effect. The preset algorithm can be used to align multiple annular point cloud data sets into a common coordinate system to achieve 3D reconstruction, scene modeling or other analysis tasks.

[0086] For image 506 , image 507 , and image 508 , downsampling processing may be performed on image 506 , image 507 , and image 508 , respectively, to reduce the amount of data of each image, thereby obtaining intermediate image 1 , intermediate image 2 , and intermediate image 3 .

[0087] In a specific example, intermediate image 1 can be used as the original image, intermediate image 2 can be used as the target image, and a first rotation matrix can be obtained after processing based on the second preset algorithm, and intermediate image 1 can be subjected to matrix transformation based on the first rotation matrix to obtain matched image 1. Intermediate image 3 can be used as the original image, and intermediate image 2 can be used as the target image, and a second rotation matrix can be obtained after processing based on the second preset algorithm, and intermediate image 3 can be subjected to matrix transformation based on the second rotation matrix to obtain matched image 3. On this basis, intermediate image 2, matched image 1 and matched image 3 can be merged, and a third preset algorithm can be used to fuse the merged multi-field textures, so that the transition at the splicing is more natural and the splicing marks are eliminated.

[0088] In another specific example, after obtaining the matched image 1, the intermediate image 2 and the matched image 1 can be merged, and the intermediate image 3 is used as the original image, and the merged intermediate image 2 and the matched image 1 are used as the target image. The third rotation matrix is ​​obtained after processing based on the second preset algorithm, and the intermediate image 3 is matrix transformed based on the third rotation matrix to obtain the matched image 3, so as to take into account the matching of image 1 and image 3. On this basis, the intermediate image 2, the matched image 1 and the matched image 3 can be merged, and the third preset algorithm can be used to fuse the merged multi-field texture, so that the transition at the splicing is more natural, and the splicing marks are eliminated, so as to realize the full-scale development of the spherical detection object, and the reconstructed three-dimensional model is closer to the real surface of the spherical detection object in the field of view effect.

[0089] After obtaining the target point cloud data, the target point cloud data can be texture mapped. Texture mapping refers to mapping a texture to the surface of a 3D model to increase the visual details and realism of the 3D model. For example, the texture includes color, pattern or material details. The texture mapping method can be configured according to actual business needs and is not limited here.

[0090] According to the embodiments of the present disclosure, by processing the multiple reconstructed position information under each field of view separately, the annular point cloud data under different fields of view can be obtained, so that information with more angles and dimensions can be obtained, which is helpful to improve the accuracy and integrity of subsequent three-dimensional reconstruction. On this basis, by determining the rotation matrix based on different annular point cloud data and splicing the multiple annular point cloud data based on the rotation matrix, the fusion of the reconstructed position information of different fields of view can be achieved, the integrity and consistency of the target point cloud data can be improved, and then more complete target point cloud data can be formed, and the accuracy of three-dimensional reconstruction can be improved.

[0091] Figure 6 The block diagram of a 3D reconstruction apparatus according to an embodiment of the present disclosure is schematically shown.

[0092] like Figure 6 As shown, the 3D reconstruction device 600 may include an acquisition module 610 , a processing module 620 and a 3D reconstruction module 630 .

[0093] The acquisition module 610 is used to acquire images of the detection object in multiple fields of view.

[0094] The processing module 620 is used to process the position information included in the captured image based on the conversion relationship to obtain multiple reconstructed position information. The conversion relationship is determined according to the field of view range of the image acquisition device used to capture the image.

[0095] The three-dimensional reconstruction module 630 is used to perform three-dimensional reconstruction on the detection object based on multiple reconstruction position information.

[0096] According to the embodiments of the present invention, any one or more of the modules, or at least part of the functions of any one of them, can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or in any one of the three implementation methods of software, hardware and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0097] It should be noted that the 3D reconstruction device part in the embodiment of the present disclosure corresponds to the 3D reconstruction method part in the embodiment of the present disclosure. The description of the 3D reconstruction device part specifically refers to the 3D reconstruction method part, which will not be repeated here.

[0098] Figure 7 A block diagram of an electronic device suitable for implementing a three-dimensional reconstruction method according to an embodiment of the present disclosure is schematically shown. Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0099] like Figure 7 As shown, the computer electronic device 700 according to the embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 702 or the program loaded from the storage part 708 to the random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include an on-board memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiment of the present disclosure.

[0100] In the RAM 703, various programs and data necessary for the operation of the electronic device 700 are stored. The processor 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704.

[0101] According to an embodiment of the present disclosure, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to the bus 704. The electronic device 700 may further include one or more of the following components connected to the input / output (I / O) interface 705: an input portion 706 including a keyboard, a mouse, etc.; an output portion 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 708 including a hard disk, etc.; and a communication portion 709 including a network interface card such as a LAN card, a modem, etc. The communication portion 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output (I / O) interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage portion 708 as needed.

[0102] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the three-dimensional reconstruction method according to the embodiment of the present disclosure is implemented.

[0103] In the present disclosure, a computer readable storage medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0104] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method provided by the embodiment of the present disclosure. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the object detection method provided by the embodiment of the present disclosure.

[0105] When the computer program is executed by the processor 701, the above functions defined in the system / device of the embodiment of the present disclosure are executed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0106] According to the embodiments of the present disclosure, program codes for executing the computer programs provided by the embodiments of the present disclosure may be written in any combination of one or more programming languages.

[0107] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. It should also be noted that in some alternative implementations, the functions marked in the blocks may also occur in an order different from that marked in the accompanying drawings.

[0108] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A three-dimensional reconstruction method, comprising: Acquire images of the detection object in multiple fields of view; Based on a conversion relationship, the position information included in the captured image is processed to obtain a plurality of reconstructed position information, wherein the conversion relationship is determined according to a field of view range of an image acquisition device used for capturing the image; and Based on the multiple reconstruction position information, the detection object is three-dimensionally reconstructed.

2. The method according to claim 1, wherein: Based on the conversion relationship, the position information included in the captured image is processed to obtain a plurality of reconstructed position information including: For each of the location information, Determining a first projection angle according to the position information and the field of view of the image acquisition device; and The reconstructed position information is determined according to the first projection angle.

3. The method according to claim 2, wherein: The first projection angle includes a pitch angle and an azimuth angle; Determining the first projection angle according to the position information and the field of view of the image acquisition device includes: Determining the pitch angle according to the coordinate value of the first direction in the position information and the field of view of the image acquisition device in the vertical direction; and The azimuth angle is determined according to the coordinate value of the second direction in the position information and the field of view of the image acquisition device in the horizontal direction, wherein the first direction and the second direction are orthogonal to each other.

4. The method according to claim 2, wherein: The determining, according to the first projection angle, the reconstructed position information comprises: The reconstructed position information is determined according to a preset projection radius and the first projection angle, wherein the preset projection radius represents a distance from a spherical center of the object to a spherical surface of the object.

5. The method according to claim 2, wherein: Determining the first projection angle according to the position information and the field of view of the image acquisition device includes: Determine a first projection distance according to a distance between two adjacent pieces of position information, wherein the distance between the two adjacent pieces of position information is determined according to the number of the pieces of position information and a field of view of the image acquisition device; and The first projection angle is determined according to the first projection distance.

6. The method according to any one of claims 1 to 5, wherein The field of view is determined based on the angle of the image acquisition device relative to the detection object; The acquiring of images of the detection object in multiple fields of view comprises: The image acquisition device is used to acquire images of the detection object placed on the rotating device using the multiple fields of view to obtain the captured images.

7. The method according to claim 6, wherein: The three-dimensional reconstruction of the detection object based on the multiple reconstruction position information includes: Based on an angle corresponding to the field of view, processing a plurality of reconstructed position information under the field of view to obtain annular point cloud data under the field of view; and Based on the rotation matrix determined by the annular point cloud data in the field of view and the annular point cloud data in other fields of view, the annular point cloud data in each field of view is spliced ​​to obtain target point cloud data.

8. The method according to claim 1, wherein: The detection object is a sphere.

9. A three-dimensional reconstruction device, comprising: An acquisition module is used to acquire images of the detection object in multiple fields of view; A processing module, configured to process the position information included in the captured image based on a conversion relationship to obtain a plurality of reconstructed position information, wherein the conversion relationship is determined according to a field of view range of an image acquisition device used for capturing the image; as well as The three-dimensional reconstruction module is used to perform three-dimensional reconstruction on the detection object based on the multiple reconstruction position information.

10. An electronic device, comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images