Three-dimensional reconstruction method and device, electronic equipment, storage medium and program product

Abstract

The invention provides a three-dimensional reconstruction method and apparatus, an electronic device, a storage medium and a program product. The three-dimensional reconstruction method comprises the steps of obtaining point cloud images of a detection object in different view fields; based on a conversion relation, processing the position information included in the point cloud image to obtain a plurality of pieces of reconstruction position information, the conversion relation being determined according to respective distance information of the plurality of pieces of position information in the point cloud image; and performing three-dimensional reconstruction on the detection object based on the plurality of pieces of reconstruction position information.

Description

Technical Field

[0001] The present disclosure relates to the fields of image processing, point cloud processing and industrial technology, and more specifically, to a three-dimensional reconstruction method, device, electronic device, storage medium and program product. Background Art

[0002] With the development of industrial technology, 3D model data is increasingly used in various industrial fields. 3D reconstruction refers to the process of restoring the geometric structure of a 3D scene or detected object from captured images, point cloud images or other observation data. The goal of 3D reconstruction is to create a 3D model, which can be a wireframe model, surface model or voxel model.

[0003] Therefore, how to achieve more effective three-dimensional reconstruction of the detection object to ensure the accuracy of the obtained three-dimensional model data is an urgent problem to be solved. Summary of the invention

[0004] In view of this, the present disclosure provides a 3D reconstruction method, device, electronic device, storage medium and program product.

[0005] According to one aspect of the present disclosure, a three-dimensional reconstruction method is provided, including: acquiring point cloud images of a detection object in multiple fields of view; processing position information included in the above point cloud images based on a conversion relationship to obtain multiple reconstructed position information, wherein the above conversion relationship is determined based on respective distance information of the multiple position information in the above point cloud images; and, based on the multiple reconstructed position information, performing three-dimensional reconstruction of the above detection object.

[0006] According to another aspect of the present disclosure, a three-dimensional reconstruction device is provided, including: an acquisition module, used to acquire point cloud images of a detection object in multiple fields of view; a processing module, used to process position information included in the above point cloud images based on a conversion relationship to obtain multiple reconstructed position information, wherein the above conversion relationship is determined based on respective distance information of the multiple position information in the above point cloud images; and a three-dimensional reconstruction module, used to perform three-dimensional reconstruction of the above detection object based on the multiple reconstructed position information.

[0007] According to another aspect of the present disclosure, an electronic device is provided, comprising: one or more processors; and a memory for storing one or more instructions, wherein when the one or more instructions are executed by the one or more processors, the one or more processors implement the method described in the present disclosure.

[0008] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which executable instructions are stored. When the executable instructions are executed by a processor, the processor implements the method described in the present disclosure.

[0009] According to another aspect of the present disclosure, a computer program product is provided. The computer program product includes computer executable instructions. When the computer executable instructions are executed, they are used to implement the method described in the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0011] Figure 1 The system architecture to which the three-dimensional reconstruction method can be applied according to an embodiment of the present disclosure is schematically shown;

[0012] Figure 2 A flowchart of a three-dimensional reconstruction method according to an embodiment of the present disclosure is schematically shown;

[0013] Figure 3 An example schematic diagram schematically shows a target distance information determination process according to an embodiment of the present disclosure;

[0014] Figure 4A~Figure 4B An example schematic diagram of a process of processing the position information included in a point cloud image based on a conversion relationship to obtain reconstructed position information according to an embodiment of the present disclosure is schematically shown;

[0015] Figure 5A An example schematic diagram of a data acquisition system according to an embodiment of the present disclosure is schematically shown;

[0016] Figure 5B An example schematic diagram of a process of performing three-dimensional reconstruction on a detection object based on multiple reconstruction position information according to an embodiment of the present disclosure is schematically shown;

[0017] Figure 6 A block diagram schematically shows a three-dimensional reconstruction device according to an embodiment of the present disclosure; and

[0018] Figure 7 A block diagram of an electronic device suitable for implementing a three-dimensional reconstruction method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0019] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0020] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0021] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0022] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0023] In the technical solution of the present invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0024] In one example, the three-dimensional reconstruction method may include at least one of the following: a three-dimensional reconstruction method based on visual geometry and a three-dimensional reconstruction method based on deep learning.

[0025] The three-dimensional reconstruction method based on visual geometry refers to estimating the camera pose of the collected data through multi-field-of-view images, and then extracting features from the images and performing comparison and stitching to achieve the conversion from two-dimensional images to three-dimensional models.

[0026] The 3D reconstruction method based on deep learning refers to the use of the learning and fitting capabilities of deep neural networks to achieve the conversion from two-dimensional images to three-dimensional models.

[0027] However, since the three-dimensional model data obtained based on the above method only includes three-dimensional position information but not texture information, that is, the accuracy of the three-dimensional model data is relatively limited, how to more effectively realize three-dimensional reconstruction of the detection object to ensure the accuracy of the obtained three-dimensional model data is an urgent problem to be solved.

[0028] To this end, the present disclosure provides a three-dimensional reconstruction method including: acquiring point cloud images of the detection object in multiple fields of view; based on a transformation relationship, processing the position information included in the point cloud image to obtain multiple reconstructed position information, the transformation relationship is determined according to the distance information of each of the multiple position information in the point cloud image; and, based on the multiple reconstructed position information, performing three-dimensional reconstruction of the detection object.

[0029] Figure 1 The system architecture to which the 3D reconstruction method can be applied according to the embodiment of the present disclosure is schematically shown. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0030] like Figure 1 As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104 and a server 105. The network 104 is a medium for providing a communication link between different devices.

[0031] It should be noted that the 3D reconstruction method provided in the embodiment of the present disclosure can generally be executed by the server 105 . Accordingly, the 3D reconstruction device provided in the embodiment of the present disclosure can generally be set in the server 105 .

[0032] Alternatively, the 3D reconstruction method provided in the embodiment of the present disclosure may also be executed by the first terminal device 101, the second terminal device 102 or the third terminal device 103. Accordingly, the 3D reconstruction apparatus provided in the embodiment of the present disclosure may also be disposed in the first terminal device 101, the second terminal device 102 or the third terminal device 103.

[0033] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0034] It should be noted that the sequence numbers of the operations in the following method are only used as representations of the operations for the purpose of description, and should not be regarded as representing the execution order of the operations. Unless explicitly stated, the method does not need to be executed completely in the order shown.

[0035] The above describes the system architecture that can be used for the three-dimensional reconstruction method provided by the present disclosure. Figure 2 For example, the three-dimensional reconstruction process of the present disclosure is further described.

[0036] Figure 2 The flowchart of the three-dimensional reconstruction method according to the embodiment of the present disclosure is schematically shown.

[0037] like Figure 2 As shown, the 3D reconstruction method 200 includes operations S210 to S230.

[0038] In operation S210, point cloud images of the detection object in multiple fields of view are acquired.

[0039] In operation S220, the position information included in the point cloud image is processed based on the conversion relationship to obtain a plurality of reconstructed position information, wherein the conversion relationship is determined according to respective distance information of the plurality of position information in the point cloud image.

[0040] In operation S230 , three-dimensional reconstruction is performed on the detection object based on the plurality of reconstruction position information.

[0041] The detection object may refer to a target or object that needs to be paid attention to during the three-dimensional reconstruction process. The method of acquiring the point cloud image can be configured according to actual business needs and is not limited here. For example, the method of acquiring point cloud data may include at least one of the following: an acquisition method based on laser radar, an acquisition method based on structured light projection, and an acquisition method based on stereo matching of binocular or multi-camera, etc.

[0042] A point cloud image may refer to data consisting of a series of position information on the surface of the target or object. A point cloud image represents the external shape, structure, and geometric information of the detected object. A point cloud image includes multiple points. The position information may represent the position of each point relative to a certain reference origin in three-dimensional space. For example, the position information may include coordinate values ​​corresponding to the X coordinate and the Y coordinate. The distance information of the position information may include a coordinate value corresponding to the Z coordinate.

[0043] After obtaining the point cloud image, the position information included in the point cloud image can be processed based on the conversion relationship to obtain multiple reconstructed position information. The reconstructed position information can refer to the position in the three-dimensional space. The method for obtaining the reconstructed position information can be configured according to actual business needs and is not limited here.

[0044] For example, the method of processing the position information to obtain the reconstructed position information may include at least one of the following: a method based on point cloud registration, a method based on point cloud feature extraction, and a method based on point cloud surface reconstruction. The method based on point cloud registration refers to registering point cloud images under different fields of view and aligning them to the same coordinate system to obtain the reconstructed position information. The method based on point cloud feature extraction refers to extracting features from point cloud images and determining the reconstructed position information based on the features of point cloud images under different fields of view. The method based on point cloud surface reconstruction refers to fitting the position information through spherical harmonics (Spherical Harmonics) or moving least squares (MLS) to obtain the reconstructed position information.

[0045] In one example, the conversion relationship may be determined based on the distance information of each of the plurality of position information in the point cloud image. In this case, the position information may be converted based on the distance information of each of the plurality of position information to obtain the reconstructed position information.

[0046] After obtaining multiple reconstructed position information, the detection object can be reconstructed in three dimensions based on the multiple reconstructed position information, so as to utilize the information of multiple fields of view to perform three-dimensional reconstruction on the detection object. Three-dimensional reconstruction refers to the process of restoring a three-dimensional spatial structure from a two-dimensional point cloud image.

[0047] The shape of the detection object can be configured according to actual business needs and is not limited here. For example, the shape of the detection object may include at least one of the following: a sphere, a cube, a cylinder, a cone, a polyhedron, and an irregular shape. In one example, the spherical detection object may include a workpiece with a spherical or spherical shape, which may be a complete sphere or a partial sphere, such as a spherical surface and a hemispherical surface.

[0048] According to the embodiments of the present disclosure, by acquiring point cloud images from multiple fields of view, more spatial information can be obtained, thereby improving the accuracy of subsequent three-dimensional reconstruction, reducing information loss due to field of view limitations, and enabling subsequent three-dimensional reconstruction results to be close to the real surface of the detection object in all directions. By flexibly establishing a conversion relationship by utilizing the field of view range of the image acquisition device and the distance information of each point cloud data, and processing the position information of the point cloud image based on the conversion relationship, it is possible to adapt to the needs of different scenarios. On this basis, by performing three-dimensional reconstruction of the detection object based on multiple reconstructed position information, the three-dimensional reconstruction effect and the accuracy of the obtained three-dimensional model are improved.

[0049] In the embodiment of the present disclosure, the conversion relationship can also be determined based on the distance information of each of the multiple position information in the point cloud image. Figure 3 The process of determining the target distance information in this case is further described.

[0050] Figure 3 An example schematic diagram of a target distance information determination process according to an embodiment of the present disclosure is schematically shown.

[0051] In 300 , the process of obtaining reconstructed position information based on position information is described by taking the case where the detection object is a sphere and the transformation relationship is determined according to the respective distance information of a plurality of position information in the point cloud image as an example.

[0052] When the reconstructed position information obtained based on the two-dimensional position information is used for three-dimensional reconstruction, the reconstructed detection object is a standard sphere, that is, the radius from the center of the sphere to each point on the sphere is the same. However, in actual applications, the detection object may not be a standard sphere, so there is a deviation between the reconstruction result of three-dimensional reconstruction based on two-dimensional position information alone and the actual detection object. Therefore, three-dimensional reconstruction based on position information can be introduced, and by calculating the real distance from the center of the sphere to each point on the sphere, the reconstruction result 304 of three-dimensional reconstruction combined with position information is closer to the actual detection object.

[0053] like Figure 3 As shown, the point cloud image 301 includes M rows and N columns of position information. M and N are both positive integers. The distance information corresponding to each point in the point cloud image 301 represents the real distance from the point cloud acquisition device of the point cloud image 301 to the surface of the detection object. Column 302 on the point cloud image 301 corresponds to a circle of data 303 on the detection object. For example, in an application scenario where the detection object is a sphere, the point cloud image 301 may include a set of position information corresponding to the detection object obtained by using an image acquisition device, and correspondingly, the reconstruction result 304 of the three-dimensional reconstruction obtained based on the position information may include a set of reconstructed position information of the sphere corresponding to the detection object.

[0054] In one example, the target distance information can be obtained by calibrating and calculating the characterization sphere. For example, the distance information with the largest value can be determined in the distance information of the N position information in each row to obtain M candidate distance information to filter the noise of the distance information. The number of candidate distance information of each column can be determined, and the column whose number meets the predetermined condition is determined as the target column, which is the column with the largest value of distance information, so the target column can be considered as the column corresponding to the maximum value point.

[0055] On this basis, the target distance information can be determined according to the target column including M distance information. For example, the first mean of the M distance information can be determined, and the first mean is determined as the target distance information. Alternatively, the M distance information can be sorted according to the numerical relationship, and the distance information at the front predetermined position in the sorted M distance information is selected, and the second mean of the distance information at the front predetermined position is determined, and the second mean is determined as the target distance information.

[0056] According to the embodiments of the present disclosure, by respectively determining the distance information with the largest value in each row and performing column-by-column counting to screen and obtain the target column, it is possible to filter the noise in the point cloud image to accurately identify the target column, thereby improving the accuracy of the target column determination. On this basis, by determining the target distance information based on the distance information included in the target column, the accuracy of the target distance information is improved, which in turn helps to improve the accuracy of the subsequent reconstructed position information.

[0057] The above describes how to obtain target distance information. Figure 4A~Figure 4B The process of processing the position information included in the point cloud image to obtain the reconstructed position information is further described based on the conversion relationship determined based on the distance information of each of the multiple position information in the point cloud image.

[0058] Figure 4A~Figure 4B The diagram schematically shows an example of a process of processing the position information included in a point cloud image based on a conversion relationship to obtain reconstructed position information according to an embodiment of the present disclosure.

[0059] Combination Figure 4A and Figure 4B , taking the application scenario where the detection object is a sphere as an example, the process of obtaining the reconstructed position information is explained.

[0060] like Figure 4A As shown, the position information set corresponding to the detection object shown in the top view 401 of the point cloud image is a circle. For a standard sphere, the center of the sphere is The position is the rotation center of the rotating device where the detection object is placed, and the distance from the position information corresponding to the target distance information to the sphere center can be used as the preset projection radius. According to the preset projection radius and the target distance information, the translation amount from the point cloud camera coordinates to the world coordinates can be determined, that is, the distance 402 from the reference distance of the point cloud camera to the sphere center.

[0061] like Figure 4BAs shown, the set of position information corresponding to the detection object shown in the top view 403 of the point cloud image is a circle, and the second projection distance can be determined according to the distance 402 from the reference distance of the point cloud camera to the center of the sphere and the distance information of each position information. According to the position information, the second projection angle is determined. On this basis, each position information can be transformed according to the second projection angle and the second projection distance to determine the actual coordinates corresponding to each point, and obtain the reconstructed position information.

[0062] In a specific example, according to the distance information of each of the multiple position information in the point cloud image, the process of processing the position information to obtain the reconstructed position information can be shown in the following formulas (1) to (6).

[0063] #timg# （1） #timg# （2） #timg# （3） #timg# （4） #timg# （5） #timg# （6）

[0064] in, Characterize the target distance information, Characterizes the projection distance, Characterize the second projection angle, and Representation location information, Represents the row value, Characterizes the column value, Represents the number of rows, Characterize the number of columns, Represents the actual distance between pixels on each line, The coordinate value representing the first direction, The coordinate value representing the second direction, The coordinate value representing the third direction.

[0065] According to the embodiments of the present disclosure, by determining the target distance information from multiple distance information, the morphological features of the detected object in space can be effectively captured. On this basis, by combining the position information and the target distance information, the second projection angle and projection distance corresponding to the position information can be accurately calculated, thereby improving the accuracy of the subsequent reconstruction of the position information.

[0066] The above describes the method of processing position information based on the conversion relationship and the method of processing position information based on the conversion relationship. Figure 5A The process of acquiring point cloud images under different fields of view is further explained.

[0067] Figure 5A An example schematic diagram of a data acquisition system according to an embodiment of the present disclosure is schematically shown.

[0068] like Figure 5AAs shown, the data acquisition system may include a point cloud acquisition device 502 for point cloud images, a rotation device 503, and a clamping device 504. The field of view may be determined according to an angle of the point cloud acquisition device 502 relative to the detection object 505.

[0069] The detection object 505 can be placed on the rotating device 503 via the clamping device 504. The rotating device 503 can be used to change the rotation direction of the detection object 505. The clamping device 504 can be used to change the flipping direction of the detection object 505. Changing the rotation direction by the rotating device 503 and changing the flipping direction by the clamping device 504 is equivalent to changing the angle of the point cloud acquisition device 502 relative to the detection object 505.

[0070] During the image acquisition process, the rotation device 503 and the clamping device 504 can be used to change the field of view, and the point cloud acquisition device 502 can be used to acquire images of the detection object 505 in multiple fields of view to obtain a point cloud image. For example, in an application scenario where the detection object is a sphere, the point cloud acquisition device 502 can be used to acquire a point cloud image of the detection object 505.

[0071] According to the embodiments of the present disclosure, the angle of the point cloud acquisition device relative to the detection object is used to ensure the acquisition of different fields of view. By acquiring point cloud images of different fields of view, a comprehensive analysis of the detection object can be performed, which helps to ensure the comprehensiveness and accuracy of the subsequent three-dimensional reconstruction effect.

[0072] The above describes how to obtain point cloud images. Figure 5B The process of performing three-dimensional reconstruction of the detected object based on multiple reconstruction position information is described.

[0073] Figure 5B An example schematic diagram of a process of performing three-dimensional reconstruction on a detection object based on multiple reconstruction position information according to an embodiment of the present disclosure is schematically shown.

[0074] like Figure 5B As shown, when the detection object is a sphere, in the process of stitching a complete sphere, since the minimum number of acquisitions required is related to the field of view of the point cloud acquisition device 502, the field of view of the point cloud acquisition device 502 is The 3D reconstruction process is explained by taking the minimum number of acquisitions as 3 as an example.

[0075] For example, the first circle data of the spherical detection object is collected by the point cloud acquisition device 502 to obtain a point cloud image 506 with a field of view of 0 degrees. After the detection object 505 is flipped 60 degrees by the clamping device 504, the second circle data of the spherical detection object is collected by the point cloud acquisition device 502 to obtain a point cloud image 507 with a field of view of 60 degrees. After the detection object 505 is flipped 60 degrees by the clamping device 504, the third circle data of the spherical detection object is collected by the point cloud acquisition device 502 to obtain a point cloud image 508 with a field of view of 120 degrees. After obtaining the point cloud image 506 with a field of view of 0 degrees, the point cloud image 507 with a field of view of 60 degrees, and the point cloud image 508 with a field of view of 120 degrees, the three point cloud images can be spliced ​​to obtain a complete sphere.

[0076] In one example, multiple reconstructed position information under the field of view can be processed based on the angle corresponding to the field of view to obtain annular point cloud data under the field of view. The annular point cloud data can refer to a set of point cloud data arranged in a ring in a plane or space. For example, the multiple reconstructed position information can be rotated according to the angle between the clamping device 504 and the point cloud acquisition device 502 to obtain the transformed initial annular point cloud data under the field of view. On this basis, the initial annular point cloud data can be flipped according to the flipping angle between the images to obtain the annular point cloud data under the field of view.

[0077] In another example, the annular point cloud data in each field of view may be spliced ​​based on the rotation matrix determined by the annular point cloud data in the field of view and the annular point cloud data in other fields of view to obtain the target point cloud data. The splicing process of the point cloud image may include a coarse matching process and a fine matching process. The coarse matching process may be used to obtain the initial pose, and the fine matching process may be used to improve the registration accuracy.

[0078] For example, the rough matching process may refer to directly splicing the annular point cloud data under multiple fields of view to obtain the initial point cloud data, wherein there are overlapping areas between the annular point cloud data under the multiple fields of view.

[0079] Alternatively, the fine matching process may refer to performing fine matching based on a preset algorithm according to the overlapping area features between the initial point cloud data obtained by coarse matching, and then optimizing the parameters of the preset algorithm for the spherical point cloud data after fine matching to achieve the best matching effect.

[0080] In one example, for the reconstructed position information obtained based on the position information, a second preset method designed for the point cloud position information can be used for fine matching. The stitching result obtained based on the position information is a spherical point cloud having both the real spherical surface position information and texture information. The first preset algorithm and the second preset algorithm can be used to align multiple point cloud data sets into a common coordinate system to achieve three-dimensional reconstruction, scene modeling or other analysis tasks.

[0081] The first preset method and the second preset method can be configured according to actual business needs and are not limited here. For example, the first preset method and the second preset method can include at least one of the following: a method based on iterative closest point (ICP), a method based on point-to-plane, a method based on feature matching, and a method based on a machine learning model.

[0082] The iterative closest point based method refers to obtaining the reconstructed position information by iteratively finding the closest point pair and minimizing the distance between the point pairs. The point-to-plane based method refers to using the distance from the point to the target point cloud plane as an error metric to obtain the reconstructed position information. The feature matching based method refers to extracting features from point cloud images in different fields of view, and determining the correspondence between point clouds through feature matching to obtain the reconstructed position information. The machine learning model based method can refer to using a machine learning model to predict the transformation relationship between point clouds to obtain the reconstructed position information.

[0083] For point cloud image 506, point cloud image 507 and point cloud image 508, point cloud image 506, point cloud image 507 and point cloud image 508 can be downsampled respectively to reduce the data amount of each point cloud image to obtain intermediate point cloud image 1, intermediate point cloud image 2 and intermediate point cloud image 3.

[0084] In a specific example, the intermediate point cloud image 1 can be used as the original point cloud image, and the intermediate point cloud image 2 can be used as the target point cloud image. After processing based on the second preset algorithm, a first rotation matrix can be obtained, and the intermediate point cloud image 1 can be matrix transformed based on the first rotation matrix to obtain the matched point cloud image 1. The intermediate point cloud image 3 can be used as the original point cloud image, and the intermediate point cloud image 2 can be used as the target point cloud image. After processing based on the second preset algorithm, a second rotation matrix can be obtained, and the intermediate point cloud image 3 can be matrix transformed based on the second rotation matrix to obtain the matched point cloud image 3. On this basis, the intermediate point cloud image 2, the matched point cloud image 1 and the matched point cloud image 3 can be merged, and the third preset algorithm can be used to fuse the multi-field texture of the merged point cloud, so that the transition at the splicing is more natural and the splicing marks are eliminated.

[0085] In another specific example, after obtaining the matched point cloud image 1, the intermediate point cloud image 2 and the matched point cloud image 1 can be merged, and the intermediate point cloud image 3 is used as the original point cloud image, and the merged intermediate point cloud image 2 and the matched point cloud image 1 are used as the target point cloud image, and the third rotation matrix is ​​obtained after processing based on the second preset algorithm, and the intermediate point cloud image 3 is matrix transformed based on the third rotation matrix to obtain the matched point cloud image 3, so as to take into account the matching of the point cloud image 1 and the point cloud image 3. On this basis, the intermediate point cloud image 2, the matched point cloud image 1 and the matched point cloud image 3 can be merged, and the third preset algorithm can be used to fuse the multi-field texture of the merged point cloud, so that the transition at the splicing is more natural, and the splicing marks are eliminated, so as to realize the full-scale development of the sphere detection object, and the reconstructed three-dimensional model is closer to the real surface of the sphere detection object in the field of view effect.

[0086] After obtaining the target point cloud data, the target point cloud data can be texture mapped. Texture mapping refers to mapping a texture to the surface of a 3D model to increase the visual details and realism of the 3D model. For example, the texture includes color, pattern or material details. The texture mapping method can be configured according to actual business needs and is not limited here.

[0087] For example, since the point cloud acquisition device obtains the brightness image of the laser band while obtaining the point cloud image, the brightness image corresponds to the points in the point cloud image one by one. The brightness image includes at least one of the following: color information, brightness value, surface material, texture details, and ambient lighting. Based on the brightness image, the brightness information can be filled into the texture coordinate information of the target point cloud data.

[0088] According to the embodiments of the present disclosure, by processing the multiple reconstructed position information under each field of view separately, the annular point cloud data under different fields of view can be obtained, so that information with more angles and dimensions can be obtained, which is helpful to improve the accuracy and integrity of subsequent three-dimensional reconstruction. On this basis, by determining the rotation matrix based on different annular point cloud data and splicing the multiple annular point cloud data based on the rotation matrix, the fusion of the reconstructed position information of different fields of view can be achieved, the integrity and consistency of the point cloud data can be improved, and then more complete target point cloud data can be formed, and the accuracy of three-dimensional reconstruction can be improved.

[0089] Figure 6 The block diagram of a 3D reconstruction apparatus according to an embodiment of the present disclosure is schematically shown.

[0090] like Figure 6 As shown, the 3D reconstruction device 600 may include an acquisition module 610 , a processing module 620 and a 3D reconstruction module 630 .

[0091] The acquisition module 610 is used to acquire point cloud images of the detection object in multiple fields of view.

[0092] The processing module 620 is used to process the position information included in the point cloud image based on the conversion relationship to obtain multiple reconstructed position information. The conversion relationship is determined according to the distance information of each of the multiple position information in the point cloud image.

[0093] The three-dimensional reconstruction module 630 is used to perform three-dimensional reconstruction on the detection object based on multiple reconstruction position information.

[0094] According to the embodiments of the present invention, any one or more of the modules, or at least part of the functions of any one of them, can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or in any one of the three implementation methods of software, hardware and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0095] It should be noted that the 3D reconstruction device part in the embodiment of the present disclosure corresponds to the 3D reconstruction method part in the embodiment of the present disclosure. The description of the 3D reconstruction device part specifically refers to the 3D reconstruction method part, which will not be repeated here.

[0096] Figure 7 A block diagram of an electronic device suitable for implementing a three-dimensional reconstruction method according to an embodiment of the present disclosure is schematically shown. Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0097] like Figure 7As shown, the computer electronic device 700 according to the embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 702 or the program loaded from the storage part 708 to the random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include an on-board memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiment of the present disclosure.

[0098] In the RAM 703, various programs and data necessary for the operation of the electronic device 700 are stored. The processor 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704.

[0099] According to an embodiment of the present disclosure, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to the bus 704. The electronic device 700 may further include one or more of the following components connected to the input / output (I / O) interface 705: an input portion 706 including a keyboard, a mouse, etc.; an output portion 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 708 including a hard disk, etc.; and a communication portion 709 including a network interface card such as a LAN card, a modem, etc. The communication portion 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output (I / O) interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage portion 708 as needed.

[0100] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the three-dimensional reconstruction method according to the embodiment of the present disclosure is implemented.

[0101] In the present disclosure, a computer readable storage medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0102] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method provided by the embodiment of the present disclosure. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the object detection method provided by the embodiment of the present disclosure.

[0103] When the computer program is executed by the processor 701, the above functions defined in the system / device of the embodiment of the present disclosure are executed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0104] According to the embodiments of the present disclosure, program codes for executing the computer programs provided by the embodiments of the present disclosure may be written in any combination of one or more programming languages.

[0105] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. It should also be noted that in some alternative implementations, the functions marked in the blocks may also occur in an order different from that marked in the accompanying drawings.

[0106] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A three-dimensional reconstruction method, comprising: Obtain point cloud images of the detection object in multiple fields of view; Based on the conversion relationship, the position information included in the point cloud image is processed to obtain a plurality of reconstructed position information, wherein the conversion relationship is determined according to the respective distance information of the plurality of position information in the point cloud image; as well as Based on the multiple reconstruction position information, the detection object is three-dimensionally reconstructed.

2. The method according to claim 1, wherein: Based on the conversion relationship, the position information included in the point cloud image is processed to obtain a plurality of reconstructed position information including: For each of the location information, determining a second projection angle and a second projection distance according to the position information and target distance information determined based on the plurality of distance information; and The reconstructed position information is determined according to the position information, the second projection angle, and the second projection distance.

3. The method according to claim 2, wherein: The point cloud image includes M rows and N columns of position information, where M and N are both positive integers; The target distance information is obtained by: Among the N position information in each row, determine the distance information with the largest value to obtain M candidate distance information; Determine the number of candidate distance information for each column, and determine the column whose number meets a predetermined condition as the target column; as well as The target distance information is determined based on the M distance information of the target column.

4. The method according to claim 2, wherein: Determining the second projection angle and the second projection distance according to the position information and the target distance information determined based on the multiple distance information includes: Determining the second projection angle according to the position information; Determining intermediate distance information according to a preset projection radius and the target distance information, wherein the preset projection radius is used to represent the distance from the spherical center of the object to the spherical surface of the object; and The second projection distance is determined according to the target distance information and the intermediate distance information.

5. The method according to claim 2, wherein: The determining, according to the position information, the second projection angle, and the second projection distance, the reconstructed position information comprises: Determining a coordinate value in a first direction and a coordinate value in a third direction according to the second projection angle and the second projection distance; and A coordinate value of a second direction is determined according to the position information, wherein the first direction, the second direction and the third direction are orthogonal to each other.

6. The method according to any one of claims 1 to 5, wherein The field of view is determined according to the angle of the point cloud acquisition device relative to the detection object; The step of acquiring point cloud images of the detection object in multiple fields of view includes: The point cloud acquisition device is used to acquire the detection object placed on the rotating device using the multiple fields of view to obtain the point cloud image.

7. The method according to claim 6, wherein: The three-dimensional reconstruction of the detection object based on the multiple reconstruction position information includes: Based on an angle corresponding to the field of view, processing a plurality of reconstructed position information under the field of view to obtain annular point cloud data under the field of view; and Based on the rotation matrix determined by the annular point cloud data in the field of view and the annular point cloud data in other fields of view, the annular point cloud data in each field of view is spliced ​​to obtain target point cloud data.

8. The method according to claim 1, wherein: The detection object is a sphere.

9. A three-dimensional reconstruction device, comprising: An acquisition module is used to acquire point cloud images of the detection object in multiple fields of view; A processing module, configured to process the position information included in the point cloud image based on a conversion relationship to obtain a plurality of reconstructed position information, wherein the conversion relationship is determined according to respective distance information of the plurality of position information in the point cloud image; as well as The three-dimensional reconstruction module is used to perform three-dimensional reconstruction on the detection object based on the multiple reconstruction position information.

10. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images