Three-dimensional reconstruction method, electronic equipment and product

Abstract

The invention provides a three-dimensional reconstruction method, electronic equipment and a product, and the method comprises the steps: carrying out the non-coding point detection of a first image and a second image of a shot object, and obtaining the non-coding point data in the first image and the non-coding point data in the second image; at least one non-coding point is arranged on the shot object; matching the non-coding point data in the first image with the non-coding point data in the second image to obtain a matching pair corresponding to each non-coding point; wherein the matching pairs corresponding to the non-coding points comprise first non-coding point data of the non-coding points in the first image and second non-coding point data of the non-coding points in the second image; and calculating a three-dimensional coordinate of each non-coding point based on the matching pair corresponding to each non-coding point. According to the scheme, the non-coding points are arranged on the shot object, compared with setting of coding points, the cost of three-dimensional reconstruction is reduced, the three-dimensional coordinates of the non-coding points can be automatically determined, and the efficiency and accuracy of three-dimensional reconstruction are improved.

Description

Technical Field

[0001] This application relates to the field of 3D scanning and reconstruction technology, and in particular to a 3D reconstruction method, electronic device and product. Background Technology

[0002] Binocular stereo vision, an important branch of computer vision, aims to mimic the human binocular vision mechanism by using two cameras (left and right) to calculate the 3D information of a target scene. In binocular camera-based 3D reconstruction, matching the left and right images captured by the cameras is crucial to the reconstruction quality. Currently, matching the left and right images can be achieved by placing coded points on the photographed object and using these points to match the left and right images. However, these coded points are specially made markers, which are costly. Alternatively, matching the left and right images can be achieved through manual intervention, but this is less efficient, relies heavily on operational experience, and has lower accuracy.

[0003] Therefore, how to improve the efficiency and accuracy of 3D reconstruction and reduce its cost is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] Based on the above needs, this application proposes a three-dimensional reconstruction method, electronic device, and product, which can improve the efficiency and accuracy of three-dimensional reconstruction and reduce the cost of three-dimensional reconstruction.

[0005] To achieve the above objectives, this application proposes the following technical solution: According to a first aspect of the embodiments of this application, a three-dimensional reconstruction method is provided, comprising: Non-coded point detection is performed on the first image and the second image of the photographed object to obtain non-coded point data in the first image and non-coded point data in the second image; wherein, at least one non-coded point is provided on the photographed object; the first image and the second image are images obtained by photographing the photographed object with two cameras with the same shooting parameters but different shooting angles; The non-coded point data in the first image is matched with the non-coded point data in the second image to obtain matching pairs corresponding to each non-coded point; wherein, the matching pairs corresponding to the non-coded points include the first non-coded point data of the non-coded point in the first image and the second non-coded point data of the non-coded point in the second image; The three-dimensional coordinates of each non-coded point are calculated based on the matching pairs corresponding to each non-coded point.

[0006] Optionally, non-coded point detection is performed on the first image and the second image of the captured object respectively to obtain non-coded point data in the first image and non-coded point data in the second image, including: Non-coded point detection is performed on the first image and the second image of the captured object to obtain the non-coded point detection box information in the first image and the non-coded point detection box information in the second image. Based on the non-coded point detection box information in the first image and the non-coded point detection box information in the second image, the center coordinates of the non-coded points in the first image and the center coordinates of the non-coded points in the second image are determined.

[0007] Optionally, the non-coded point data in the first image is matched with the non-coded point data in the second image to obtain matching pairs corresponding to each non-coded point, including: Epipolar constraints are applied to the non-coded point data in the first image and the non-coded point data in the second image to determine candidate matching pairs corresponding to each non-coded point; wherein, there are at least one candidate matching pair corresponding to each non-coded point. All candidate matching pairs are filtered and matched to determine the matching pair corresponding to each non-coding point; where each non-coding point corresponds to only one matching pair.

[0008] Optionally, all candidate matching pairs are filtered and matched to determine the matching pairs corresponding to each non-coding point, including: According to the preset number of iterations, all candidate matching pairs are iteratively filtered and matched to obtain the inlier set corresponding to each iteration, wherein the inlier set contains at least one matching pair; The set of inliers containing the most matching pairs is taken as the target inlier set, and all matching pairs in the target inlier set are the matching pairs corresponding to each non-coded point.

[0009] Optionally, iterative filtering and matching are performed on all candidate matching pairs to obtain the inlier set corresponding to each iteration, including: A preset number of matching pairs are randomly selected from all candidate matching pairs, and the target perspective transformation matrix is ​​calculated using the pre-set conversion rules between the matching pairs and the perspective transformation matrix. Using the target perspective transformation matrix, the predicted matching non-coding point data corresponding to the first non-coding point data in the remaining matching pairs is calculated, and the distance between the predicted matching non-coding point data and the second non-coding point data in the remaining matching pairs is calculated as the prediction difference corresponding to the remaining matching pairs; wherein, the remaining matching pairs are the matching pairs other than the matching pair samples among all candidate matching pairs; All remaining matching pairs whose predicted differences are less than a preset threshold are stored in the inlier set.

[0010] Optionally, the preset number of matching pairs includes matching pairs on at least two polar lines.

[0011] Optionally, calculating the three-dimensional coordinates of each non-coded point based on the matching pairs corresponding to each non-coded point includes: The difference in the x-coordinate between the first non-coded point data and the second non-coded point data in the matching pair corresponding to each non-coded point is taken as the disparity corresponding to each non-coded point. Based on the focal lengths of the two cameras, the distance between the optical centers of the two cameras, and the parallax corresponding to each non-coded point, the three-dimensional coordinates of each non-coded point are calculated using a predetermined three-dimensional coordinate calculation rule.

[0012] Optionally, non-coded point detection is performed on the first image and the second image of the captured object respectively to obtain non-coded point detection bounding box information in the first image and non-coded point detection bounding box information in the second image, including: The first and second images of the captured object are respectively input into a pre-trained target detection model to obtain non-coding point detection box information in the first image and non-coding point detection box information in the second image. The target detection model includes a depthwise separable convolutional network.

[0013] According to a second aspect of the embodiments of this application, an electronic device is provided, including: a memory and a processor; The memory is connected to the processor and is used to store programs; The processor is used to implement the above-described three-dimensional reconstruction method by running the program in the memory.

[0014] According to a third aspect of the embodiments of this application, a computer program product is provided, including computer program instructions, which, when executed by a processor, cause the processor to implement the above-described three-dimensional reconstruction method.

[0015] The proposed 3D reconstruction method involves detecting non-coded points in a first image and a second image of a photographed object, respectively, to obtain non-coded point data in the first image and non-coded point data in the second image. At least one non-coded point is set on the photographed object. The first and second images are obtained by capturing the object with two cameras using the same shooting parameters but different shooting angles. The non-coded point data in the first image is matched with the non-coded point data in the second image to obtain matching pairs corresponding to each non-coded point. Each matching pair includes the first non-coded point data in the first image and the second non-coded point data in the second image. Based on the matching pairs corresponding to each non-coded point, the 3D coordinates of each non-coded point are calculated. By setting non-coded points on the photographed object, the technical solution of this application reduces the cost of 3D reconstruction compared to setting coded points, and automatically determines the 3D coordinates of each non-coded point, improving the efficiency and accuracy of 3D reconstruction compared to manual intervention. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating a three-dimensional reconstruction method provided in an embodiment of this application.

[0018] Figure 2 This is a schematic diagram of the structure of a depthwise separable convolutional network provided in an embodiment of this application.

[0019] Figure 3 This is a schematic diagram of an epipolar constraint mismatch provided in an embodiment of this application.

[0020] Figure 4 This is a schematic diagram illustrating the calculation of three-dimensional coordinates provided in an embodiment of this application.

[0021] Figure 5 This is a schematic diagram of the process for determining matching pairs provided in an embodiment of this application.

[0022] Figure 6 This is a schematic diagram of the iterative filtering and matching process provided in the embodiments of this application.

[0023] Figure 7 This is a flowchart illustrating another three-dimensional reconstruction method provided in an embodiment of this application.

[0024] Figure 8This is a schematic diagram of the structure of a three-dimensional reconstruction device provided in an embodiment of this application.

[0025] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0026] The technical solutions of this application are applicable to three-dimensional scanning and reconstruction scenarios. By employing the technical solutions of this application, the efficiency and accuracy of three-dimensional reconstruction can be improved, while reducing the cost of three-dimensional reconstruction.

[0027] Binocular stereo vision, an important branch of computer vision, aims to calculate the 3D information of a target scene by using two cameras (left and right) to mimic the human binocular vision mechanism. In 3D reconstruction based on binocular cameras, the matching between the left and right images captured by the cameras is crucial to the quality of the reconstruction.

[0028] Currently, matching left and right images mainly relies on encoded points or manual intervention, but this has significant drawbacks: Matching left and right images relies on coded points. However, coded points are unique identifiers, typically made of specialized materials (such as high-reflectivity stickers or targets) with specific patterns (such as QR codes, barcodes, or geometric combinations) printed on their surfaces. Each coded point's pattern corresponds to a unique ID, which can be identified through image processing or laser scanning. Therefore, coded points are expensive, thus increasing the cost of 3D reconstruction.

[0029] Manual intervention relies on the operator's experience, is cumbersome and time-consuming, and makes it difficult to achieve large-scale, automated 3D reconstruction tasks, thus affecting the efficiency and accuracy of 3D reconstruction.

[0030] Based on this, this application proposes a three-dimensional reconstruction method. This technical solution can set non-coded points on the photographed object and automatically determine the three-dimensional coordinates of each non-coded point, thereby solving the problems of high cost, low efficiency and low accuracy of three-dimensional reconstruction in the prior art.

[0031] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0032] Exemplary methods See Figure 1 As shown in the figure, this application proposes a three-dimensional reconstruction method. The method includes: S101. Perform non-coded point detection on the first image and the second image of the captured object respectively to obtain non-coded point data in the first image and non-coded point data in the second image.

[0033] In this embodiment, before image capture, at least one non-coded point needs to be set on the object to be captured. Furthermore, if more than two non-coded points are set on the object, all non-coded points cannot be on the same line. In this embodiment, setting non-coded points on the object is less costly than using special materials or having a unique pattern printed on them, thus reducing the cost of 3D reconstruction.

[0034] Furthermore, in this embodiment, the shape of the non-coded point can be various, such as a circle, square, or polygon. A circular non-coded point is preferred. Additionally, the edge lines of the non-coded point can be solid black, while the inner area can be solid white, making the color distinction of the non-coded point more obvious and facilitating its detection in the captured image. Moreover, this embodiment can use matte non-coded points, which can be brightened by illumination during image capture, further enhancing their detection.

[0035] In this embodiment, two cameras with the same shooting parameters but different shooting angles are used to simultaneously capture images of the object, thereby obtaining two images, namely the first image and the second image. For example, a binocular camera can be used to capture images, in which case the first image and the second image are obtained by capturing images of the object from different directions at the same time.

[0036] In this embodiment, after acquiring the first and second images of the captured object, non-coded point detection needs to be performed on both images. This involves identifying non-coded points in both images and extracting their coordinates as non-coded point data, thereby obtaining the non-coded point data in the first and second images. In this embodiment, non-coded point detection can be performed on the images using the following methods: The first method is based on template matching. Templates of different scales with the same shape as non-coded points are predefined. The templates are slid across the image for matching, and the similarity between the template and the image region where the template is slid is calculated. The positions where the similarity reaches a preset threshold are taken as the non-coded point data in the image.

[0037] The second approach, based on deep learning, trains a convolutional neural network (CNN) to directly output the coordinates of non-coded points. The training of the CNN utilizes pre-collected sample images carrying pre-labeled real non-coded point data. These sample images are input into the CNN, which outputs predicted non-coded point data. The goal is to minimize the difference between the predicted and real non-coded point data, thereby adjusting the CNN's parameters.

[0038] The third method is contour matching combined with ellipse fitting. This method is for the detection of circular non-coded points. First, contour detection is used to detect the target region in the image that matches the shape of the non-coded point. Then, ellipse fitting is used to determine the center position of the target region as the non-coded point data.

[0039] In one specific implementation, non-coded point detection is performed on a first image and a second image of the captured object to obtain non-coded point data in the first image and non-coded point data in the second image, including the following steps: First, non-coded point detection is performed on the first image and the second image of the captured object to obtain non-coded point detection box information in the first image and non-coded point detection box information in the second image.

[0040] This embodiment first requires feature extraction from the first and second images of the captured object, preferably using multi-level feature extraction to obtain the image features of the first and second images. Then, a detection box is used to perform sliding detection in the first image, and the feature of the current position of the detection box is extracted from the image features of the first image. This feature is then compared with the features of non-coded points. If the similarity reaches a preset threshold, the current position of the detection box is used as the non-coded point detection box information in the first image. The detection box is then traversed and slid across the first image to obtain all non-coded point detection box information. The method for determining the non-coded point detection box information in the second image is the same as that for the first image, and will not be repeated here.

[0041] Specifically, this embodiment can achieve non-coding point detection based on deep neural network object detection technology. That is, based on deep neural network object detection technology, convolutional neural networks are used to extract image features layer by layer, combined with a candidate region generation mechanism to lock the potential target range, and finally, object classification and bounding box regression are completed through parallel processing to obtain non-coding point detection box information. This embodiment can pre-train an object detection model, and then input the first and second images of the captured object into the object detection model respectively. This object detection model can perform multi-level feature extraction to obtain image features of the first image and image features of the second image. Based on the image features of the first image, it outputs non-coding point detection box information in the first image, and based on the image features of the second image, it outputs non-coding point detection box information in the second image. The training of the object detection model first requires collecting image samples carrying pre-labeled true non-coding point detection box information. Then, the image samples are input into the object detection model, which outputs predicted non-coding point detection box information. The model parameters are adjusted to minimize the difference between the true and predicted non-coding point detection box information. In this embodiment, the YOLO series of object detection models can be used.

[0042] Furthermore, such as Figure 2 As shown, this embodiment can use the YOLOX-s model. Alternatively, a lightweight depthwise separable convolution can replace the conventional convolution in the object detection model; that is, the object detection model includes a depthwise separable convolutional network. A depthwise separable convolutional network combines depthwise convolution and pointwise convolution, offering a lightweight convolutional operation. In the depthwise convolution stage, each input channel uses an independent convolutional kernel to avoid increasing the number of parameters. In the pointwise convolution stage, by using a 1×1 convolutional kernel to process the output of the depthwise convolution, information from different channels can be combined and the feature dimension reduced, thereby reducing computational complexity. Figure 2 In the diagram, filter1 represents the first channel-wise convolution, filter2 represents the second channel-wise convolution, and filter3 represents the third channel-wise convolution. The pointwise convolution consists of n 1×1 convolution kernels.

[0043] Furthermore, the excessive stacking of convolutional blocks in the decoupled head structure used in the YOLOX-s model leads to significant parameter redundancy and a decrease in detection speed. This embodiment removes some convolutional blocks from the original decoupled head structure and replaces the CBS convolutional blocks with DSCBM convolutional blocks. This further reduces the computational load and improves the detection speed while maintaining detection performance. Specifically, the CBS convolutional block consists of a convolutional layer (Conv), a batch normalization (BN) layer, and an activation function (such as SiLU), while the DSCBM convolutional block consists of depthwise separable convolutions (Conv), batch normalization, and an activation function.

[0044] Second, based on the non-coded point detection box information in the first image and the non-coded point detection box information in the second image, the center coordinates of the non-coded points in the first image and the center coordinates of the non-coded points in the second image are determined.

[0045] In this embodiment, after detecting the non-coded point detection box information in the first image and the non-coded point detection box information in the second image, the center coordinates of the non-coded points within the detection box corresponding to the non-coded point detection box information are calculated to obtain the center coordinates of the non-coded points in the first image and the center coordinates of the non-coded points in the second image. The center coordinates of the non-coded points in the first image are used as the non-coded point data in the first image, and the center coordinates of the non-coded points in the second image are used as the non-coded point data in the second image.

[0046] Specifically, this embodiment first needs to extract edges from the image within the detection box corresponding to the non-coded point detection box information in the first image, thereby extracting the contour point set within the detection box. This contour point set represents the contour of the non-coded point within the detection box. Edge extraction can be performed using the Canny operator to detect edges within the image of the detection box, thus extracting the contour point set. Then, based on the extracted contour point set, the center coordinates of the contour corresponding to the contour point set are calculated. These center coordinates are the center coordinates of the non-coded point within the detection box. For circular non-coded points, this embodiment uses the least squares method to fit an ellipse to calculate the center coordinates of the non-coded point. The general equation of an ellipse can be expressed as: =1, where Here, 'a' and 'b' are the coordinates of the ellipse's center, and 'a' and 'b' are the semi-axis lengths of the ellipse along the x and y axes, respectively. Solving the ellipse's equations using the least squares method yields the optimal ellipse parameters, allowing us to calculate the ellipse's center coordinates. These coordinates are then used as the center coordinates of the non-coded points within the detection box. For non-circular non-coded points, such as square or polygonal points, we first calculate the circumcircle corresponding to the contour point set within the detection box, and then use the least squares method to fit the ellipse to calculate the center coordinates. This method allows us to calculate the center coordinates of the non-coded points within the detection box corresponding to each non-coded point in the first image, thus identifying all non-coded point data in the first image. Furthermore, the calculation method for the center coordinates of non-coded points in the second image is the same as that in the first image, and will not be repeated here.

[0047] S102. Match the non-coded point data in the first image with the non-coded point data in the second image to obtain matching pairs corresponding to each non-coded point.

[0048] The non-coded point data in the first image determined by the above steps includes the center coordinates of each non-coded point detected in the first image, and the non-coded point data in the second image includes the center coordinates of each non-coded point detected in the second image.

[0049] This embodiment requires matching the non-coded point data in the first image with the non-coded point data in the second image. That is, by using the center coordinates of each non-coded point detected in the first image and the center coordinates of each non-coded point detected in the second image, it is determined whether the non-coded point detected in the first image and the non-coded point detected in the second image are the same non-coded point. The center coordinates of the non-coded point in the first image that belongs to the same coded point are taken as the first non-coded point data corresponding to the coded point, and the center coordinates of the non-coded point in the second image that belongs to the same non-coded point are taken as the second non-coded point data corresponding to the non-coded point. The first non-coded point data and the second non-coded point data of the non-coded point are taken as the matching pair corresponding to the non-coded point. In other words, the matching pair corresponding to the non-coded point includes the first non-coded point data of the non-coded point in the first image and the second non-coded point data of the non-coded point in the second image.

[0050] Specifically, in this embodiment, the non-coded point data in the first image is matched with the non-coded point data in the second image, which can be done using epipolar constraints. In the binocular stereo vision model, given the intrinsic and extrinsic parameters of the binocular camera, the pixel coordinates of points that are corresponding (i.e., the same non-coded point) are... and (i.e., the matching pairs corresponding to non-coding points) satisfy the formula, This constraint relationship is called epipolar constraint, where matrix F is called the fundamental matrix, representing the relationship between image points projected from a spatial point onto different imaging planes. It is related to both the intrinsic and extrinsic parameters of the camera. Epipolar constraint indicates the non-coded points in the left image. A corresponding point can definitely be found on the epipolar line in the right-hand image. Therefore, the two-dimensional matching search for the first and second images is transformed into a one-dimensional search along the epipolar line, saving computation and eliminating false matching points. This embodiment, based on epipolar constraints, can perform epipolar non-coded point matching on the epipolar line between the non-coded point data in the first and second images. Specifically, it matches whether the center coordinates of the non-coded points in the first image and the center coordinates of the non-coded points in the second image lie on the same epipolar line. The center coordinates of the non-coded points in the first and second images that lie on the same epipolar line are considered a matching pair, thereby determining the matching pairs in the first and second images.

[0051] However, polar constraints can produce errors, such as... Figure 3 As shown, I l For the first image, I r For the second image, P is a non-coded point in the object being photographed, and O... l O is the optical center of the camera that takes the first image. r Let f be the optical center of the camera that captured the second image, f be the camera focal length, and the origin be the top left corner of the first image. Let u be the horizontal axis and v be the vertical axis. First image I l Points in With the second image I r Points in and points Since they are all on the same epipolar line, epipolar constraints may be used to determine... and and All match, thus making and as well as and All were determined to be matching pairs of non-coding point P, but and The matching pairs are not non-coded points, resulting in a matching error. Therefore, in this embodiment, while using epipolar constraints to match non-coded point data in the first image and non-coded point data in the second image, it is necessary to further filter the matched pairs to improve their accuracy. Therefore, in one specific implementation, this step includes: First, epipolar constraints are applied to the non-coded point data in the first image and the non-coded point data in the second image to determine the candidate matching pairs corresponding to each non-coded point.

[0052] In this embodiment, epipolar constraints are first applied to the non-coded point data in the first image and the non-coded point data in the second image to determine the candidate matching pairs corresponding to each non-coded point. The specific steps have been described above and will not be repeated here.

[0053] Second, all candidate matching pairs are filtered and matched to determine the matching pairs corresponding to each non-coding point.

[0054] After calculating candidate matching pairs using epipolar constraints, each candidate matching pair needs to be further filtered to remove incorrectly matched data. First, from all candidate matching pairs, pairs where the first non-coding point data matches multiple second non-coding point data, and pairs where the second non-coding point data matches multiple first non-coding point data, are extracted as pending matching pairs. Then, matching analysis is performed on each pending matching pair where the first non-coding point data matches multiple second non-coding point data. From the multiple second non-coding point data, one data point that truly matches the first non-coding point data is selected and paired with that first non-coding point data. All other matching pairs formed by the first non-coding point data and other second non-coding point data are discarded. The process involves matching the first non-coded point data with multiple second non-coded point data. Image features within a preset range of the location of the first non-coded point data are extracted as the first feature. Image features within the preset range of the locations of all second non-coded point data that match the first non-coded point data are also extracted as the second features corresponding to each second non-coded point data. Similarity is calculated between the first feature and each second feature, and the second non-coded point data corresponding to the second feature with the highest similarity is considered the data that truly matches the first non-coded point data. Then, matching analysis is performed on each pair of second non-coded point data that forms a matching pair with multiple first non-coded point data. The first non-coded point data that truly matches the second non-coded point data is selected and paired with the second non-coded point data, while all other matching pairs formed by second non-coded point data are discarded. The specific steps for matching analysis of each pair of to-be-selected matching pairs where the second non-coding point data and multiple first non-coding point data form matching pairs are the same as the specific steps for matching analysis of each pair of to-be-selected matching pairs where the first non-coding point data and multiple second non-coding point data form matching pairs, and will not be repeated here.

[0055] Through the above steps, erroneous matching pairs are removed from the candidate matching pairs, and the remaining matching pairs are the matching pairs corresponding to each non-coding point. Furthermore, each non-coding point corresponds to only one matching pair.

[0056] S103. Calculate the three-dimensional coordinates of each non-coded point based on the matching pairs corresponding to each non-coded point.

[0057] In this embodiment, after determining the matching pairs corresponding to each non-coded point, it is necessary to calculate the disparity between the first and second images of the non-coded point based on the first and second non-coded point data contained in the matching pairs. Specifically, the difference between the abscissa of the first and second non-coded point data is used as the disparity between the first and second images. Then, based on the disparity corresponding to the non-coded point and a predetermined 3D coordinate calculation rule, the 3D coordinates of the non-coded point are calculated. The specific steps are as follows: First, the difference in the abscissa between the first non-coded point data and the second non-coded point data in the matching pair corresponding to each non-coded point is taken as the disparity corresponding to each non-coded point.

[0058] Second, based on the focal length of the two cameras, the distance between the optical centers of the two cameras, and the parallax corresponding to each non-coded point, the three-dimensional coordinates of each non-coded point are calculated using a predetermined three-dimensional coordinate calculation rule.

[0059] like Figure 4 As shown, I l For the first image, I r For the second image, P is a non-coded point in the object being photographed, and O... l O is the optical center of the camera that takes the first image. r Let f be the optical center of the camera that captured the second image, f be the camera's focal length, and the first image have its origin at the top left corner, with u as the horizontal axis and v as the vertical axis. The second image also has its origin at the top left corner, and the axes are oriented in the same direction as those of the first image. The x-coordinate value of the principal point in the first image. This represents the x-coordinate of the principal point in the second image. (A spatial point is a non-coded point on the photographed object.) The images projected onto the imaging plane of the binocular camera are respectively the first image I. l Points in Second image I r Points in Baseline B is the optical center O. l and light center O r The distance between them, Z is the distance from point P to the baseline. According to and Based on the similarity relationship, the following formula can be derived: In this embodiment, the two cameras that captured the first and second images have the same shooting parameters and are both horizontally positioned. For example, the two cameras in a stereo camera are identical and horizontally positioned. and If the values ​​are the same, then the above formula can be simplified to: Since disparity is the difference in the x-coordinate between the first non-coded point data and the second non-coded point data, then disparity... Therefore, it can be concluded that The Z calculated by the above formula is the depth value from the non-coding point P to the binocular vision system. Therefore, the distance of the non-coding point from the camera can be determined by the disparity value. According to the pinhole imaging model, we know that... Substituting the formula for calculating Z into this formula, we can obtain, The final calculation formula is the predetermined three-dimensional coordinate calculation rule. Here, x and y are the center coordinates of non-coded points in the image. When calculating using the center coordinates of non-coded points in the first image, x is... y is When calculating using the center coordinates of non-coded points in the first image, x is... y is This embodiment can calculate the three-dimensional coordinates of a non-coded point using either the center coordinates of the non-coded point in the first image or the center coordinates of the non-coded point in the second image; this embodiment does not impose any limitations.

[0060] Based on the above rules for calculating 3D coordinates, it can be seen that calculating the 3D coordinates of non-coded points requires utilizing the focal length of the binocular camera. The distance between the optical centers of the two cameras in a binocular camera and the disparity corresponding to non-coded points Substituting this data into the 3D coordinate calculation rules, the 3D coordinates of the non-coded points can be calculated, including the focal length. The distance between the optical centers of the two cameras in a binocular camera All parameters (including internal and external parameters) of the camera are obtained through camera calibration. This method allows for the calculation of the 3D coordinates of all non-coded points set on the photographed object, thereby achieving 3D reconstruction.

[0061] As described above, the 3D reconstruction method proposed in this application involves detecting non-coded points in a first image and a second image of a photographed object, respectively, to obtain non-coded point data in the first image and non-coded point data in the second image. At least one non-coded point is provided on the photographed object. The first and second images are obtained by capturing the object with two cameras using the same shooting parameters but different shooting angles. The non-coded point data in the first image is matched with the non-coded point data in the second image to obtain matching pairs corresponding to each non-coded point. Each matching pair includes first non-coded point data in the first image and second non-coded point data in the second image. Based on the matching pairs corresponding to each non-coded point, the 3D coordinates of each non-coded point are calculated. By using the technical solution of this application, setting non-coded points on the photographed object reduces the cost of 3D reconstruction compared to setting coded points, and automatically determines the 3D coordinates of each non-coded point, improving the efficiency and accuracy of 3D reconstruction compared to manual intervention.

[0062] As an optional implementation method, see [link to implementation details]. Figure 5 As shown, in another embodiment of this application, step S102, which involves matching the non-coded point data in the first image with the non-coded point data in the second image to obtain matching pairs corresponding to each non-coded point, includes the following steps: filtering and matching all candidate matching pairs to determine the matching pairs corresponding to each non-coded point. S501. According to the preset number of iterations, perform iterative filtering and matching on all candidate matching pairs to obtain the set of interior points corresponding to each iteration.

[0063] This embodiment employs an iterative filtering and matching method, performing multiple filtering and matching operations on all candidate matching pairs, and storing the matched pairs from each filtering and matching process into the interior point set corresponding to each iteration. Each iteration's interior point set contains at least one matching pair. This embodiment pre-sets the number of iterations, and the iterative filtering and matching is performed according to this number of iterations. The specific steps for each iterative filtering and matching are as follows: First, a preset number of matching pairs are randomly selected from all candidate matching pairs, and the target perspective transformation matrix is ​​calculated using the pre-set conversion rules between the matching pairs and the perspective transformation matrix.

[0064] In this embodiment, the expression for the pre-set perspective transformation matrix is: The pre-defined conversion rules between the matching pairs and the perspective transformation matrix are as follows: in, This represents the first non-coded point data in the i-th candidate matching pair. This represents the second non-coded point data in the i-th candidate matching pair.

[0065] This embodiment requires randomly selecting a preset number of matching pairs from all candidate matching pairs and calculating the target perspective transformation matrix using a pre-set conversion rule between the matching pairs and the perspective transformation matrix. The preset number is determined based on the conversion rule between the matching pairs and the perspective transformation matrix. For example, the conversion rule requires at least four matching pairs to calculate all elements in the perspective transformation matrix; therefore, the preset number is set to at least four pairs. By substituting the data from the randomly selected matching pairs into the conversion rule, the element data in the perspective transformation matrix can be calculated, thus determining the target perspective transformation matrix.

[0066] Furthermore, in order to ensure that the target perspective transformation matrix can be accurately calculated, the randomly selected matching pairs cannot all be located on the same epipolar line. In other words, the preset number of matching pairs includes matching pairs on at least two epipolar lines.

[0067] Second, using the target perspective transformation matrix, the predicted matching non-coding point data corresponding to the first non-coding point data in the remaining matching pairs is calculated, and the distance between the predicted matching non-coding point data and the second non-coding point data in the remaining matching pairs is calculated as the prediction difference corresponding to the remaining matching pairs.

[0068] After calculating the target perspective transformation matrix, it is necessary to use this matrix to calculate the predicted matching non-coded point data corresponding to the first non-coded point data in the remaining matching pairs. That is, multiply the first non-coded point data in the remaining matching pairs by the target perspective transformation matrix to obtain the predicted matching non-coded point data corresponding to the first non-coded point data in the remaining matching pairs. Here, the remaining matching pairs are the matching pairs other than the matching pair samples among all candidate matching pairs.

[0069] This embodiment also requires calculating the distance between the predicted matching non-coding point data corresponding to the first non-coding point data in the remaining matching pairs and the second non-coding point data in the remaining matching pairs, and using this distance as the prediction difference corresponding to the remaining matching pairs. The distance between the predicted matching non-coding point data and the second non-coding point data can be calculated using Euclidean distance.

[0070] This embodiment calculates the prediction differences corresponding to all remaining matching pairs using the method described above.

[0071] Third, all remaining matching pairs whose prediction differences are less than a preset threshold are stored in the inlier set.

[0072] In this embodiment, the predicted difference corresponding to each remaining matching pair is compared with a preset threshold, and the remaining matching pairs with predicted differences less than the preset threshold are stored in the inlier set, so as to obtain the inlier set after this iteration of screening and matching.

[0073] Each iteration of the matching process follows the steps described above, but the randomly selected matching pairs in each iteration cannot be completely identical; they can be partially the same.

[0074] S502. The set of inliers containing the most matching pairs is taken as the target inlier set, and all matching pairs in the target inlier set are the matching pairs corresponding to each non-coded point.

[0075] Through the above steps, an inlier set can be determined in each iteration of the matching process. In this embodiment, the inlier set containing the most matching pairs is selected as the target inlier set. All matching pairs contained in the target inlier set are the matching pairs corresponding to each non-coded point.

[0076] In a specific embodiment, such as Figure 6 As shown, the process of filtering and matching all candidate matching pairs to determine the matching pairs corresponding to each non-coding point includes the following steps: S601. Randomly select a preset number of matching pairs from all candidate matching pairs, and calculate the target perspective transformation matrix using the pre-set conversion rules between the matching pairs and the perspective transformation matrix.

[0077] S602. Using the target perspective transformation matrix, calculate the predicted matching non-coding point data corresponding to the first non-coding point data in the remaining matching pairs, and calculate the distance between the predicted matching non-coding point data and the second non-coding point data in the remaining matching pairs as the prediction difference corresponding to the remaining matching pairs.

[0078] S603. Store the remaining matching pairs whose prediction difference is less than the preset threshold into the inlier set.

[0079] S604. If the number of matching pairs in the inlier set is greater than the number of matching pairs in the target inlier set, then the inlier set is used as the target inlier set. If the number of matching pairs in the inlier set is not greater than the number of matching pairs in the target inlier set, then the target inlier set is not updated.

[0080] S605. Update the iteration count and determine whether the iteration count has reached the preset iteration count. If yes, proceed to step S606; otherwise, proceed to step S601. The preset iteration count is the pre-set iteration count described in the above embodiments.

[0081] S606. All matching pairs in the target interior point set are the matching pairs corresponding to each non-coded point.

[0082] Steps S601-S603 in this embodiment have been specifically described in the above embodiments, and will not be repeated in this embodiment.

[0083] As an optional implementation, this application also proposes a three-dimensional reconstruction method. See [link to relevant documentation]. Figure 7 As shown, the method includes: S701. Acquire the left and right images of the object captured by the binocular camera, and perform epipolar correction on the left and right images to obtain the first image and the second image.

[0084] This embodiment uses a binocular camera to capture images of the object, obtaining a left image captured by the left camera and a right image captured by the right camera. In this embodiment, the left and right cameras have the same parameters and are placed on the same horizontal line, ensuring that the optical axes of the left and right cameras are parallel and that the imaging planes are only offset horizontally. Furthermore, this embodiment requires epipolar correction to correct the left and right images, transforming the convergent binocular vision system into a forward-parallel binocular vision system. The corrected epipolar lines are parallel to the baseline and strictly aligned, satisfying the optical axis parallel structure model.

[0085] S702. Perform non-coded point detection on the first image and the second image of the captured object respectively to obtain non-coded point data in the first image and non-coded point data in the second image.

[0086] S703. Match the non-coded point data in the first image with the non-coded point data in the second image to obtain matching pairs corresponding to each non-coded point.

[0087] S704. Calculate the three-dimensional coordinates of each non-coded point based on the matching pairs corresponding to each non-coded point.

[0088] In this embodiment, the specific execution method of steps S702-S704 is the same as that of steps S101-S103 in the above embodiment, and will not be described in detail in this embodiment.

[0089] Exemplary device Accordingly, this application also provides a three-dimensional reconstruction device, see [link to relevant documentation]. Figure 8 As shown, the device includes: The target detection module 100 is used to perform non-coded point detection on the first image and the second image of the captured object, respectively, to obtain non-coded point data in the first image and non-coded point data in the second image; wherein, at least one non-coded point is set on the captured object; the first image and the second image are images obtained by capturing the captured object with two cameras with the same shooting parameters but different shooting angles. The matching module 110 is used to match the non-coded point data in the first image with the non-coded point data in the second image to obtain matching pairs corresponding to each non-coded point; wherein, the matching pairs corresponding to the non-coded points include the first non-coded point data of the non-coded point in the first image and the second non-coded point data of the non-coded point in the second image. The 3D coordinate calculation module 120 is used to calculate the 3D coordinates of each non-coded point based on the matching pairs corresponding to each non-coded point.

[0090] As can be seen from the above description, the three-dimensional reconstruction device proposed in this application sets non-coded points on the photographed object, which reduces the cost of three-dimensional reconstruction compared to setting coded points, and can automatically determine the three-dimensional coordinates of each non-coded point, which improves the efficiency and accuracy of three-dimensional reconstruction compared to manual intervention.

[0091] As an optional implementation, another embodiment of this application discloses a target detection module 100, including a detection box determination unit and a coordinate determination unit.

[0092] The detection box determination unit is used to perform non-coded point detection on the first image and the second image of the captured object respectively, and obtain non-coded point detection box information in the first image and non-coded point detection box information in the second image; The coordinate determination unit is used to determine the center coordinates of non-coded points in the first image and the center coordinates of non-coded points in the second image based on the non-coded point detection box information in the first image and the non-coded point detection box information in the second image.

[0093] As an optional implementation, another embodiment of this application discloses that the matching module 110 includes: an epipolar constraint unit and a screening matching unit.

[0094] The epipolar constraint unit is used to perform epipolar constraint on the non-coded point data in the first image and the non-coded point data in the second image to determine the candidate matching pairs corresponding to each non-coded point; wherein, there are at least one candidate matching pair corresponding to each non-coded point. The filtering and matching unit is used to filter and match all candidate matching pairs to determine the matching pair corresponding to each non-coding point; wherein, one non-coding point corresponds to only one matching pair.

[0095] As an optional implementation, another embodiment of this application discloses a screening and matching unit, including an iterative screening unit and a determination unit.

[0096] The iterative filtering unit is used to iteratively filter and match all candidate matching pairs according to a pre-set number of iterations to obtain the inlier set corresponding to each iteration, wherein the inlier set contains at least one matching pair; The determining unit is used to select the inlier set containing the largest number of matching pairs as the target inlier set, and all matching pairs in the target inlier set are the matching pairs corresponding to each non-coded point.

[0097] As an optional implementation, another embodiment of this application discloses an iterative filtering unit, specifically used for: A preset number of matching pairs are randomly selected from all candidate matching pairs, and the target perspective transformation matrix is ​​calculated using the pre-set conversion rules between the matching pairs and the perspective transformation matrix. Using the target perspective transformation matrix, the predicted matching non-coding point data corresponding to the first non-coding point data in the remaining matching pairs is calculated, and the distance between the predicted matching non-coding point data and the second non-coding point data in the remaining matching pairs is calculated as the prediction difference corresponding to the remaining matching pairs; where the remaining matching pairs are the matching pairs other than the matching pair samples among all candidate matching pairs. The remaining matching pairs whose prediction difference is less than a preset threshold are all stored in the inlier set.

[0098] As an optional implementation, another embodiment of this application discloses that a preset number of matching pairs samples include matching pairs on at least two polar lines.

[0099] As an optional implementation, another embodiment of this application discloses a three-dimensional coordinate calculation module 120, specifically used for: The difference in the x-coordinate between the first non-coded point data and the second non-coded point data in the matching pair corresponding to each non-coded point is taken as the disparity corresponding to each non-coded point. Based on the focal lengths of the two cameras, the distance between the optical centers of the two cameras, and the parallax corresponding to each non-coded point, the three-dimensional coordinates of each non-coded point are calculated using a predetermined three-dimensional coordinate calculation rule.

[0100] As an optional implementation, another embodiment of this application discloses a detection frame determination unit, specifically used for: The first and second images of the captured object are input into a pre-trained target detection model to obtain the non-coding point detection box information in the first image and the non-coding point detection box information in the second image. The object detection model includes depthwise separable convolutional networks.

[0101] As an optional implementation, another embodiment of this application discloses a three-dimensional reconstruction apparatus, which further includes an epipolar correction module.

[0102] The epipolar correction module is used to acquire the left and right images of the object captured by the binocular camera, and to perform epipolar correction on the left and right images to obtain the first image and the second image.

[0103] The three-dimensional reconstruction apparatus provided in this embodiment belongs to the same concept as the three-dimensional reconstruction method provided in the above embodiments of this application. It can execute the three-dimensional reconstruction method provided in any of the above embodiments of this application and has the corresponding functional modules and beneficial effects for executing the three-dimensional reconstruction method. Technical details not described in detail in this embodiment can be found in the specific processing content of the three-dimensional reconstruction method provided in the above embodiments of this application, and will not be repeated here.

[0104] Exemplary electronic devices Another embodiment of this application also provides an electronic device, see [link to relevant documentation] Figure 9 As shown, the device includes: Memory 200 and processor 210; The memory 200 is connected to the processor 210 and is used to store programs; The processor 210 is configured to implement the three-dimensional reconstruction method disclosed in any of the above embodiments by running the program stored in the memory 200.

[0105] Specifically, the aforementioned electronic device may also include: a bus, a communication interface 220, an input device 230, and an output device 240.

[0106] The processor 210, memory 200, communication interface 220, input device 230, and output device 240 are interconnected via a bus. Among them: A bus can include a pathway for transmitting information between various components of a computer system.

[0107] Processor 210 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present invention. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0108] Processor 210 may include a main processor, as well as a baseband chip, modem, etc.

[0109] The memory 200 stores a program that executes the technical solution of this invention, and may also store an operating system and other key business functions. Specifically, the program may include program code, which includes computer operation instructions. More specifically, the memory 200 may include read-only memory (ROM), other types of static storage devices capable of storing static information and instructions, random access memory (RAM), other types of dynamic storage devices capable of storing information and instructions, disk storage, flash memory, etc.

[0110] Input device 230 may include a device for receiving user input data and information, such as a keyboard, mouse, camera, scanner, light pen, voice input device, touch screen, pedometer, or gravity sensor.

[0111] Output device 240 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0112] The communication interface 220 may include a device that uses any transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0113] The processor 210 executes the program stored in the memory 200 and calls other devices, and can be used to implement each step of any of the three-dimensional reconstruction methods provided in the above embodiments of this application.

[0114] Exemplary computer program products and storage media In addition to the methods and apparatus described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the three-dimensional reconstruction methods according to various embodiments of this application as described in the "Exemplary Methods" section of this specification.

[0115] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0116] Furthermore, embodiments of this application may also be storage media storing a computer program, which is executed by a processor in the steps of the three-dimensional reconstruction methods according to various embodiments of this application described in the "Exemplary Methods" section above.

[0117] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0118] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0119] The steps in the methods of the various embodiments of this application can be adjusted, merged, or deleted in order according to actual needs, and the technical features described in each embodiment can be replaced or combined.

[0120] The modules and sub-modules in the various embodiments of the present application's devices and terminals can be merged, divided, and deleted according to actual needs.

[0121] It should be understood that the disclosed terminals, devices, and methods can be implemented in other ways, given the several embodiments provided in this application. For example, the terminal embodiments described above are merely illustrative. For instance, the division of modules or sub-modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple sub-modules or modules may be combined or integrated into another module, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0122] The modules or submodules described as separate components may or may not be physically separate. The components that constitute a module or submodule may or may not be physical modules or submodules; that is, they may be located in one place or distributed across multiple network modules or submodules. Some or all of the modules or submodules can be selected to achieve the purpose of this embodiment's solution, depending on actual needs.

[0123] Furthermore, the functional modules or sub-modules in the various embodiments of this application can be integrated into one processing module, or each module or sub-module can exist physically separately, or two or more modules or sub-modules can be integrated into one module. The integrated modules or sub-modules described above can be implemented in hardware or in the form of software functional modules or sub-modules.

[0124] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0125] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software unit executed by a processor, or a combination of both. The software unit can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0126] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0127] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A three-dimensional reconstruction method, characterized in that, include: Non-coded point detection is performed on the first image and the second image of the photographed object to obtain non-coded point data in the first image and non-coded point data in the second image; wherein, at least one non-coded point is provided on the photographed object; the first image and the second image are images obtained by photographing the photographed object with two cameras with the same shooting parameters but different shooting angles; The non-coded point data in the first image is matched with the non-coded point data in the second image to obtain matching pairs corresponding to each non-coded point; wherein, the matching pairs corresponding to the non-coded points include the first non-coded point data of the non-coded point in the first image and the second non-coded point data of the non-coded point in the second image; The three-dimensional coordinates of each non-coded point are calculated based on the matching pairs corresponding to each non-coded point.

2. The method according to claim 1, characterized in that, Non-coded point detection is performed on a first image and a second image of the captured object, respectively, to obtain non-coded point data in the first image and non-coded point data in the second image, including: Non-coded point detection is performed on the first image and the second image of the captured object to obtain the non-coded point detection box information in the first image and the non-coded point detection box information in the second image. Based on the non-coded point detection box information in the first image and the non-coded point detection box information in the second image, the center coordinates of the non-coded points in the first image and the center coordinates of the non-coded points in the second image are determined.

3. The method according to claim 1, characterized in that, Matching the non-coded point data in the first image with the non-coded point data in the second image yields matching pairs corresponding to each non-coded point, including: Epipolar constraints are applied to the non-coded point data in the first image and the non-coded point data in the second image to determine candidate matching pairs corresponding to each non-coded point; wherein, there are at least one candidate matching pair corresponding to each non-coded point. All candidate matching pairs are filtered and matched to determine the matching pair corresponding to each non-coding point; where each non-coding point corresponds to only one matching pair.

4. The method according to claim 3, characterized in that, All candidate matching pairs are filtered and matched to determine the matching pairs corresponding to each non-coding point, including: According to the preset number of iterations, all candidate matching pairs are iteratively filtered and matched to obtain the inlier set corresponding to each iteration, wherein the inlier set contains at least one matching pair; The set of inliers containing the most matching pairs is taken as the target inlier set, and all matching pairs in the target inlier set are the matching pairs corresponding to each non-coded point.

5. The method according to claim 4, characterized in that, Iteratively filter and match all candidate pairs to obtain the inlier set corresponding to each iteration, including: A preset number of matching pairs are randomly selected from all candidate matching pairs, and the target perspective transformation matrix is ​​calculated using the pre-set conversion rules between the matching pairs and the perspective transformation matrix. Using the target perspective transformation matrix, the predicted matching non-coding point data corresponding to the first non-coding point data in the remaining matching pairs is calculated, and the distance between the predicted matching non-coding point data and the second non-coding point data in the remaining matching pairs is calculated as the prediction difference corresponding to the remaining matching pairs; wherein, the remaining matching pairs are the matching pairs other than the matching pair samples among all candidate matching pairs; All remaining matching pairs whose predicted differences are less than a preset threshold are stored in the inlier set.

6. The method according to claim 5, characterized in that, The preset number of matching pairs includes matching pairs on at least two polar lines.

7. The method according to claim 1, characterized in that, The calculation of the three-dimensional coordinates of each non-coded point based on the matching pairs corresponding to each non-coded point includes: The difference in the x-coordinate between the first non-coded point data and the second non-coded point data in the matching pair corresponding to each non-coded point is taken as the disparity corresponding to each non-coded point. Based on the focal lengths of the two cameras, the distance between the optical centers of the two cameras, and the parallax corresponding to each non-coded point, the three-dimensional coordinates of each non-coded point are calculated using a predetermined three-dimensional coordinate calculation rule.

8. The method according to claim 2, characterized in that, Non-coded point detection is performed on a first image and a second image of the captured object, respectively, to obtain non-coded point detection bounding box information in the first image and the second image, including: The first and second images of the captured object are respectively input into a pre-trained target detection model to obtain non-coding point detection box information in the first image and non-coding point detection box information in the second image. The target detection model includes a depthwise separable convolutional network.

9. An electronic device, characterized in that, include: Memory and processor; The memory is connected to the processor and is used to store programs; The processor is configured to implement the three-dimensional reconstruction method as described in any one of claims 1 to 8 by running a program in the memory.

10. A computer program product, characterized in that, It includes computer program instructions that, when executed by a processor, cause the processor to implement the three-dimensional reconstruction method as described in any one of claims 1 to 8.

Images