A mapping method for curved surfaces and irregular surfaces

Abstract

The application relates to a mapping method for curved surfaces and irregular surfaces, and relates to the technical field of mapping. Image acquisition of a target region is performed by using a binocular endoscope camera; in the image acquisition process, a two-dimensional feature point extraction model is used to obtain feature point pixel coordinates of a target region image, a binocular depth estimation model is used to estimate the depth of each pixel point in the target region image, and a depth map is obtained; three-dimensional point cloud construction is performed in combination with camera parameters and the depth map, and gradient distance field processing is performed on the three-dimensional point cloud to obtain an improved truncated distance field; the improved truncated distance field corresponding to each feature point coordinate is searched out, and the feature point and the improved truncated distance field are combined to perform improved light beam adjustment method processing to obtain a camera pose; the pose corresponding to the camera pose is calibrated with a starting coordinate to obtain a three-dimensional mapping result; matching, calibration and pose estimation are performed in an image sequence collected by the binocular endoscope camera, and time sequence three-dimensional mapping is realized.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a mapping method for curved and irregular surfaces. Background Technology

[0002] The statements in this section are merely background information relating to this disclosure and do not necessarily constitute prior art.

[0003] 3D mapping technology is a multidisciplinary, cross-disciplinary technology that integrates machine learning, image recognition, and information fusion. This method involves many popular fields, such as computer image processing, computer graphics, computer vision, and computer-aided design. Currently, image-based 3D mapping technology has significant applications in many areas, such as digital cities, aerospace, remote sensing mapping, and digital cultural heritage. Existing 3D mapping methods generally only use two-dimensional feature points to extract pixel coordinates, and then use depth information obtained from binocular vision or deep learning to optimize the 3D mapping information using bundle adjustment. However, in reality, 2D feature points on different images often have matching errors. Simple 2D matching cannot perfectly reproduce changes in observation angles. This technology performs well in large-scale, low-interference scenarios, but when lighting conditions are insufficient or scene accuracy requirements are high, it cannot perform well in matching, reducing the algorithm's accuracy and affecting the subsequent 3D mapping results. Existing 3D mapping techniques only extract and utilize information from 2D feature points and 3D depth in images, without fully considering information such as the curvature and degree of bending of the surfaces displayed in the image. This lack of information leads to greater errors and poorer mapping results in scenarios involving numerous curved surfaces, such as surgical navigation and endoscopic observation. Therefore, traditional 3D mapping techniques are unsuitable for surgical navigation and medical observation fields with higher mapping requirements. Existing depth estimation algorithms only provide distance information in the depth direction, failing to provide more restrictive geometric information such as angles and orientations. For most mapping algorithms that combine depth estimation, the algorithm simply adds a depth coordinate to the feature point information in addition to the horizontal / vertical coordinates, thus extending 2D point matching to 3D point matching. While this increases the dimensionality of the information modality, the information provided is far from sufficient to fully characterize the irregularity of curved surfaces. Therefore, it is necessary to provide a mapping method for curved and irregular surfaces, which can achieve higher-precision 3D mapping in surgical navigation and endoscopic observation scenarios. Summary of the Invention

[0004] In order to solve the above-mentioned technical problems, or at least partially solve the above-mentioned technical problems, the present invention provides a mapping method for curved surfaces and irregular surfaces.

[0005] In a first aspect, the present invention provides a mapping method for curved surfaces and irregular surfaces, comprising: Images of the target area are acquired using a binocular endoscope camera; During the image acquisition process, a two-dimensional feature point extraction model is used to obtain the pixel coordinates of feature points in the target area image, and a binocular depth estimation model is used to estimate the depth of each pixel in the target area image to obtain a depth map. A 3D point cloud is constructed by combining camera parameters and a depth map obtained from a binocular depth estimation model, and an improved truncated distance field is obtained by performing gradient distance field processing on the 3D point cloud. The improved truncation range field corresponding to the coordinates of each feature point is retrieved, and the feature points and the improved truncation range field are combined and processed by the improved bundle adjustment method to obtain the camera pose. The pose corresponding to the camera posture is calibrated with the starting coordinates to obtain the 3D mapping result; Matching, calibration, and pose estimation are performed within the image sequence acquired by a binocular endoscope camera to achieve temporal 3D mapping.

[0006] Furthermore, the stereo depth estimation model adopts the STTR depth estimation model, and the two-dimensional feature point extraction model adopts the SuperPoint model.

[0007] Furthermore, an improved truncated range field is obtained by performing gradient range field processing on the 3D point cloud, denoted as: ; Where i represents the point number in the 3D point cloud, u represents the horizontal pixel coordinate of the point, v represents the vertical pixel coordinate of the point, depth is the depth map calculated by the stereo depth estimation model, and depth(u,v) is the depth value of the depth map in the (u,v) coordinate system. It is centered around (u,v) The depth values ​​at each location within the region are calculated independently at each of the k*k locations, and the maximum and minimum values ​​in the set are taken; t represents the translation part of the camera pose T to be solved, and max and min are the maximum and minimum values ​​calculated.

[0008] Furthermore, by combining feature points and an improved truncation range field to perform an improved bundle adjustment method, the camera pose is obtained, including: Establish the correspondence between the 3D spatial coordinates and 2D coordinates of feature points based on camera pose; The first loss function for solving the camera pose is determined based on the correspondence; The first loss function is determined based on the correspondence. The first loss function is used to solve the camera pose in the 3D mapping of the improved bundle adjustment method. The second loss function is determined based on the improved truncation distance field of the feature points. The second loss function is used to solve the camera pose in the 3D mapping using the improved bundle adjustment method. The first and second losses are combined to obtain the final total loss function used for mapping irregular surfaces.

[0009] Furthermore, based on the camera pose, the correspondence between the 3D spatial coordinates and 2D coordinates of feature points is established as follows: ; Where i represents the feature point number; This represents the horizontal pixel coordinates of feature point i in the image. The vertical pixel coordinates of feature point i in the image represent the two-dimensional coordinates of the feature point. The K parameter is obtained through a two-dimensional feature point extraction model; K is the camera intrinsic parameter, which is obtained through manual camera calibration before the camera is put into operation. These are the three-dimensional coordinates of the feature point in the three-dimensional point cloud space. The three-dimensional spatial coordinates of the feature point are: ; The scale factor is calculated based on the baseline length distance in camera calibration. T represents the camera pose to be solved, which includes the rotation component R and the translation component t. The mathematical form of the camera pose is as follows: .

[0010] Furthermore, the first loss function is expressed as follows: .

[0011] Furthermore, the calculation process of the second loss function includes: The weighting factor of the feature point is calculated based on the included angle of the normal to the feature point and the depth. The second loss function is determined based on the weights and the improved cutoff distance field, and is expressed in the following form: ; Where PSDF and w represent the calculated improved cutoff distance field and weighting factor, respectively.

[0012] Furthermore, the weighting factor of the feature point is calculated based on the angle between the normals and the depth. ; ; Where w is the weighting factor, θ is the angle between the normals of the feature points calculated under the current pose of the camera, n is the normal direction of the feature points in three-dimensional space, which is obtained by point cloud PCA, and R is the rotation part of the camera pose T.

[0013] Secondly, the present invention provides a mapping device for curved and irregular surfaces, comprising: at least one processing unit, wherein the processing unit is connected to a storage unit and a binocular endoscope camera via a bus unit; the storage unit stores a computer program, and when the computer program is executed by the processing unit, the mapping method for curved and irregular surfaces is implemented.

[0014] Thirdly, the present invention provides a mapping system for curved and irregular surfaces, comprising: an image processing module, a binocular depth estimation module, a three-dimensional point cloud processing module, a two-dimensional feature point extraction module, and a three-dimensional mapping module; The image processing module acquires RGB images of the target area through a binocular camera, and inputs the RGB images and camera parameters from a pre-calibrated camera intrinsic parameter file into the binocular depth estimation module and the two-dimensional feature point extraction module. The binocular depth estimation module uses a binocular depth estimation model to process the depth of each pixel in the RGB image to obtain a depth map, which is then passed to the 3D point cloud processing module. The two-dimensional feature point extraction module uses a two-dimensional feature point extraction model to obtain feature points of the image transmitted by the image processing module, and then inputs the pixel coordinates of the feature points into the three-dimensional mapping module for subsequent mapping. The 3D point cloud processing module combines camera parameters and the depth map obtained from the binocular depth estimation module to construct a 3D point cloud, and performs gradient distance field processing on the 3D point cloud to obtain an improved truncated distance field, which is provided to the 3D mapping module for subsequent mapping. The 3D mapping module performs subsequent mapping based on the improved truncated distance field and the feature point information extracted by the 2D feature point extraction module. After obtaining the pixel coordinates of the 2D feature points, the 3D mapping module retrieves the improved truncated distance field corresponding to each feature point coordinate, and combines the feature points and the improved truncated distance field to obtain the camera pose using the improved bundle adjustment method. The pose corresponding to the camera pose is then calibrated with the starting coordinates to obtain the 3D mapping result. Matching, calibration, and pose estimation are performed within the image sequence acquired by the binocular endoscope camera to achieve temporal 3D mapping.

[0015] The technical solutions provided in the embodiments of the present invention have the following advantages compared with the prior art: The method of this invention can make full use of the gradient information of uneven surfaces to carry out three-dimensional mapping, which improves the mapping error caused by incomplete information due to the use of only pixel coordinates in traditional mapping. It realizes high-precision mapping of irregular scenes, lays the foundation for high-precision mapping of medical surgical environments and complex outdoor environments, and provides a guarantee for subsequent industrial intelligence. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a mapping method for curved and irregular surfaces provided in an embodiment of the present invention; Figure 2 This is an architectural diagram of a mapping method for curved and irregular surfaces provided in an embodiment of the present invention. Figure 3 This is a schematic diagram of a mapping system for curved and irregular surfaces provided in an embodiment of the present invention. Figure 4 This is a schematic diagram of a mapping device for curved or irregular surfaces provided in an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0021] Example 1 In fields such as surgical navigation and endoscopic observation, because the surfaces of human tissues are generally irregular curved surfaces, existing mapping algorithms cannot guarantee the accuracy of matching, pose estimation, and mapping when directly applied to these fields. To address these issues, such as... Figure 1 and Figure 2 As shown, the present invention provides a mapping method for curved surfaces and irregular surfaces, comprising: The S100 uses a binocular endoscope camera to acquire images of the target area. The binocular endoscope camera mimics the working mechanism of the human eye, equipped with two independent cameras or optical channels spaced a certain interpupillary distance apart. The two cameras simultaneously capture images of the same object from different angles, creating parallax and providing depth information.

[0022] S200, during the image acquisition process, uses a two-dimensional feature point extraction model to obtain the pixel coordinates of feature points in the current target area image, and a binocular depth estimation model to estimate the depth information of each pixel in the current target area image.

[0023] As an example, the binocular depth estimation model of this invention employs the STTR depth estimation model, an end-to-end depth estimation network based on the Transformer architecture. The STTR model uses a Transformer with self-attention and cross-attention mechanisms to perform binocular stereo matching, allowing explicit pixel matching and imposing uniqueness constraints. During pixel matching, the STTR model uses a relative pixel distance to encode feature descriptors and employs an attention mechanism to represent unique feature descriptors. The STTR depth estimation model has low memory consumption and can run the training process on most hardware. It is suitable for depth estimation using binocular endoscope cameras, obtaining a depth map within the full view of the binocular endoscope camera, which is used to calculate subsequent 3D feature point coordinates and an improved truncated symbolic distance field.

[0024] Feature point matching is a crucial step in 3D mapping; it maps feature points in two or more images to each other, enabling image association and matching. Through feature point matching, tasks such as target tracking, image registration, and 3D reconstruction can be performed. As an example, this application's 2D feature point extraction model employs the SuperPoint model. The SuperPoint model is a fully convolutional neural network that uses self-supervised deep learning to achieve end-to-end feature point detection and description, outperforming traditional methods such as the SIFT and ORB algorithms in terms of real-time performance, accuracy, and robustness.

[0025] After depth estimation is completed, the images acquired by the binocular endoscope camera are used as the reference for inter-frame matching of the video sequence. After matching, the x and y coordinates of the feature points in the left eye image are calculated. At the same time, according to the triangulation principle and the depth map, the two-dimensional feature points can be extended into three-dimensional feature points with (X, Y, Z) coordinates.

[0026] The S300 combines camera parameters and disparity maps obtained from a binocular depth estimation model to construct a 3D point cloud, and performs gradient distance field processing on the 3D point cloud to obtain an improved truncated distance field.

[0027] Based on the 3D point cloud, the improved cutoff distance field PSDF is calculated using the following formula: ; Where i represents the point number in the 3D point cloud, u represents the horizontal pixel coordinate of the point, v represents the vertical pixel coordinate of the point, depth is the depth map calculated by the stereo depth estimation model, and depth(u,v) is the depth value of the depth map in the (u,v) coordinate system. It is centered around (u,v) The depth values ​​at each location within the region are calculated independently at each of the k*k locations, and the maximum and minimum values ​​in the set are taken; t represents the translation part of the camera pose T to be solved, and max and min are the maximum and minimum values ​​calculated.

[0028] The improved truncation range field corresponding to the coordinates of each feature point is retrieved, and the feature points and the improved truncation range field are combined to obtain the camera attitude by the improved bundle adjustment method.

[0029] After obtaining the feature points of the image using the SuperPoint model, the pixel coordinates of these feature points in the 2D image are obtained, denoted as... The three-dimensional coordinates of the feature points are denoted as XY coordinates are obtained using 2D pixel coordinates p and triangulation. Z coordinates are obtained from the 2D pixel coordinates p and the depth map, resulting in the corresponding 3D spatial coordinates. The depth map calculated by the stereo depth estimation model is denoted as depth. The depth value depth(u,v) corresponding to the 2D coordinates of the feature point is directly used as the Z value of the 3D spatial coordinates, i.e.: .

[0030] The improved cutoff distance field corresponding to the coordinates of each feature point can be retrieved using the three-dimensional coordinates of the feature points.

[0031] The process of obtaining the camera pose by combining feature points and an improved truncated range field using an improved bundle adjustment method includes: Establish the correspondence between the 3D spatial coordinates and 2D coordinates of feature points based on camera pose. According to the basic principles of computer vision, the 3D spatial coordinates of feature points... with two-dimensional coordinates The two have the following correspondence: ; Where i represents the feature point number; This represents the horizontal pixel coordinates of feature point i in the image. The vertical pixel coordinates of feature point i in the image are obtained through a two-dimensional feature point extraction model; K is the camera intrinsic parameter, which is obtained through manual camera calibration before the camera is put into operation. In this invention, the Zhang Zhengyou calibration method is used for calibration. These are the three-dimensional coordinates of the feature points in the three-dimensional point cloud space; is the scale factor, calculated based on the baseline length distance in camera calibration. T represents the camera pose, including the rotation component R and the translation component t, and is an unknown parameter in the 3D mapping of this application. The mathematical form of the camera pose is as follows: ; The first loss function is determined based on the correspondence. This first loss function is used in the process of solving the camera pose in 3D mapping using the improved bundle adjustment method, and is expressed as follows: ; In addition to the first loss function, this invention adds a second loss function constraint based on an improved truncation distance field for graph construction.

[0032] The calculation process of the second loss function includes: The weight factor of the feature point is calculated based on the angle between the feature point's normal and the depth information: ; ; In the above formula, w is the weighting factor, θ is the angle between the normals of the feature points calculated under the current pose of the camera, n is the normal direction of the feature point in three-dimensional space, which is obtained by the point cloud PCA algorithm, and R is the rotation part of the camera pose T.

[0033] The second loss function is determined based on the weights and the improved cutoff distance field, and is expressed in the following form: ; Where PSDF and w represent the calculated improved cutoff distance field and weight factor, respectively. This function is solved iteratively, with the subscript ref indicating that the latest calculated improved cutoff distance field is used as the reference value, and the subscript cur indicating that the previously calculated improved cutoff distance field is used as the current value.

[0034] Introducing the improved truncation range field into the loss function, compared to the loss E of the traditional bundle adjustment method... distance The total loss is calculated by combining the results.

[0035] E distance With E geom After summing, the final total loss function used for mapping irregular surfaces is: ; The camera pose T is calculated iteratively using the total loss function.

[0036] For most mapping algorithms that incorporate depth estimation, feature point information merely adds depth information to the horizontal / vertical information, thus extending 2D point matching to 3D point matching. While this increases the dimensionality of the information modalities, it fails to provide more robust geometric information such as angles and orientations, and the provided depth information is far from sufficient to represent the irregularities of surfaces. To address this, this invention proposes a gradient calculation mode based on an improved truncated distance field. This mode can extract richer geometric information from irregular objects and uneven surfaces captured in the image, calculate the surface normal vectors at the feature point locations, and incorporate the angle information of these surface normal vectors into the mapping process to provide additional angle and orientation data, ensuring the accuracy and effectiveness of subsequent mapping.

[0037] This invention combines an improved truncation distance field with a three-dimensional feature point method, further increasing the information in the loss function of the bundle adjustment method, which can effectively improve the robustness and accuracy of the algorithm in curved surface environments.

[0038] The method of this invention can make full use of the gradient information of uneven surfaces for three-dimensional mapping, which improves the mapping error caused by incomplete geometric information due to the use of only depth information in traditional mapping. It realizes high-precision mapping of irregular scenes and lays the foundation for high-precision mapping of medical surgical environments and complex outdoor environments.

[0039] When performing bundle adjustment optimization, other constraints are added, such as fixed markers, additional observation tools such as lidar, and information provided by inertial navigation.

[0040] Example 2 like Figure 3 As shown, the present invention provides a mapping system for curved and irregular surfaces, comprising: The system includes an image processing module, a binocular depth estimation module, a 3D point cloud processing module, a 2D feature point extraction module, and a 3D mapping module.

[0041] The image processing module acquires RGB images using a stereo camera and, combined with a pre-calibrated camera intrinsic parameter file, inputs the RGB image matrix and camera parameters into the depth estimation module and the 2D feature point extraction module. The depth estimation module internally uses the STTR algorithm based on a transformer architecture to process the image matrix, resulting in a disparity map between the stereo cameras. This disparity map is then passed to the 3D point cloud processing module. The 3D point cloud processing module combines the camera parameters and the disparity map obtained from the depth estimation module to construct a 3D point cloud and performs gradient signed distance field processing on the 3D point cloud. The improved truncation range field obtained after processing, along with the feature point information extracted by the 2D feature point extraction module, is then fed into the 3D mapping module for subsequent mapping. The 2D feature point extraction module extracts feature points from the image transmitted by the image processing module using the Superpoint algorithm, and subsequently inputs the pixel coordinates of the feature points into the 3D mapping module. After obtaining the pixel coordinates of the 2D feature points, the 3D mapping module retrieves the improved truncation range field corresponding to each feature point coordinate, and combines the two for improved bundle adjustment processing. The processing result is the camera pose, which is then calibrated with the starting coordinates to obtain the 3D mapping result.

[0042] Example 3 See Figure 4As shown, this embodiment of the invention provides a mapping device for curved and irregular surfaces, characterized by comprising: at least one processing unit, wherein the processing unit is connected to a storage unit and a binocular endoscope camera via a bus unit; the storage unit, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the software program, computer-executable program, and module corresponding to the mapping method for curved and irregular surfaces in this embodiment of the invention. The processing unit implements the aforementioned mapping method for curved and irregular surfaces by running the software program, computer-executable program, and module stored in the storage unit, including: Images of the target area are acquired using a binocular endoscope camera; During the image acquisition process, a two-dimensional feature point extraction model is used to obtain the pixel coordinates of feature points in the target area image, and a binocular depth estimation model is used to estimate the depth of each pixel in the target area image to obtain a depth map. A 3D point cloud is constructed by combining camera parameters and a depth map obtained from a binocular depth estimation model, and an improved truncated distance field is obtained by performing gradient distance field processing on the 3D point cloud. The improved truncation range field corresponding to the coordinates of each feature point is retrieved, and the feature points and the improved truncation range field are combined and processed by the improved bundle adjustment method to obtain the camera pose. The pose corresponding to the camera posture is calibrated with the starting coordinates to obtain the 3D mapping result; Matching, calibration, and pose estimation are performed on the image sequence acquired by the binocular endoscope camera to achieve temporal 3D mapping.

[0043] Of course, the computer program stored in the storage unit of the mapping device for curved and irregular surfaces provided in the embodiments of the present invention is not limited to the method operation described above, but can also execute related operations in the mapping method for curved and irregular surfaces provided in any embodiment of the present invention.

[0044] Example 4 This invention provides a computer-readable storage medium storing a computer program. When executed, the computer program implements the mapping method for curved and irregular surfaces, including: Images of the target area are acquired using a binocular endoscope camera; During the image acquisition process, a two-dimensional feature point extraction model is used to obtain the pixel coordinates of feature points in the target area image, and a binocular depth estimation model is used to estimate the depth of each pixel in the target area image to obtain a depth map. A 3D point cloud is constructed by combining camera parameters and a depth map obtained from a binocular depth estimation model, and an improved truncated distance field is obtained by performing gradient distance field processing on the 3D point cloud. The improved truncation range field corresponding to the coordinates of each feature point is retrieved, and the feature points and the improved truncation range field are combined and processed by the improved bundle adjustment method to obtain the camera pose. The pose corresponding to the camera posture is calibrated with the starting coordinates to obtain the 3D mapping result; Matching, calibration, and pose estimation are performed on the image sequence acquired by the binocular endoscope camera to achieve temporal 3D mapping.

[0045] The computer-readable storage medium provided in the embodiments of the present invention stores a computer program that is not limited to the method operation described above, but can also execute related operations in the mapping method for curved and irregular surfaces provided in any embodiment of the present invention.

[0046] In the embodiments provided by this invention, it should be understood that the disclosed structures and methods can be implemented in other ways. For example, the structural embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, structures, or units, and may be electrical, mechanical, or other forms.

[0047] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0048] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0049] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A mapping method for curved and irregular surfaces, characterized in that, include: Images of the target area are acquired using a binocular endoscope camera; During the image acquisition process, on the one hand, a two-dimensional feature point extraction model is used to obtain the feature point pixel coordinates of the target area image, and on the other hand, a binocular depth estimation model is used to estimate the depth of each pixel in the target area image to obtain a depth map. A 3D point cloud is constructed by combining camera parameters and a depth map obtained from a stereo depth estimation model. An improved truncated range field is then obtained by performing gradient range field processing on the 3D point cloud. This improved truncated range field is represented as follows: ； Where i represents the point number in the 3D point cloud, u represents the horizontal pixel coordinate of the point, v represents the vertical pixel coordinate of the point, depth is the depth map calculated by the stereo depth estimation model, and depth(u,v) is the depth value of the depth map in the (u,v) coordinate system. It is centered around (u,v) The depth values ​​at each location within the region are calculated independently at each of the k*k locations, and the maximum and minimum values ​​in the set are taken; t represents the translation part of the camera pose T to be solved, and max and min are the maximum and minimum values ​​calculated respectively. The improved truncation range field corresponding to the coordinates of each feature point is retrieved, and the feature points and the improved truncation range field are combined to obtain the camera pose using an improved bundle adjustment method, including: Based on the camera pose, establish the correspondence between the 3D spatial coordinates and 2D coordinates of feature points; determine the first loss function for solving the camera pose based on the correspondence; the first loss function, used for solving the camera pose in 3D mapping using the improved bundle adjustment method; determine the second loss function based on the improved truncation distance field of the feature points, also used for solving the camera pose in 3D mapping using the improved bundle adjustment method; combine the first and second losses to obtain the final total loss function used for mapping irregular surfaces; the calculation process of the second loss function includes: The weight factor of the feature point is calculated based on the angle between the normals and the depth of the feature point: ； ； Where w is the weighting factor, θ is the angle between the normals of the feature points calculated under the current pose of the camera, n is the normal direction of the feature points in three-dimensional space, which is obtained by point cloud PCA, and R is the rotation part of the camera pose T. The second loss function is determined based on the weights and the improved cutoff distance field, and is expressed in the following form: ； Where PSDF and w represent the calculated improved cutoff distance field and weight factor, respectively; the subscript ref indicates that the improved cutoff distance field calculated in the latest iteration is used as the reference value; and the subscript cur indicates that the improved cutoff distance field calculated in the previous iteration is used as the current value. The pose corresponding to the camera posture is calibrated with the starting coordinates to obtain the 3D mapping result; Matching, calibration, and pose estimation are performed on the image sequence acquired by the binocular endoscope camera to achieve temporal 3D mapping.

2. The mapping method for curved and irregular surfaces according to claim 1, characterized in that, The stereo depth estimation model adopts the STTR depth estimation model, and the two-dimensional feature point extraction model adopts the SuperPoint model.

3. The mapping method for curved and irregular surfaces according to claim 1, characterized in that, The correspondence between the 3D spatial coordinates and 2D coordinates of feature points is established based on the camera pose as follows: ； Where i represents the feature point number; This represents the horizontal pixel coordinates of feature point i in the image. The vertical pixel coordinates of feature point i in the image represent the two-dimensional coordinates of the feature point. The K parameter is obtained through a two-dimensional feature point extraction model; K is the camera intrinsic parameter, which is obtained through manual camera calibration before the camera is put into operation. These are the three-dimensional coordinates of the feature point in the three-dimensional point cloud space. The three-dimensional spatial coordinates of the feature point are... ; The scale factor is calculated based on the baseline length distance in camera calibration. T represents the camera pose to be solved, which includes the rotation component R and the translation component t. The mathematical form of the camera pose is as follows: 。 4. The mapping method for curved and irregular surfaces according to claim 1, characterized in that, The first loss function is expressed as follows: ； Let K be the scale factor, K be the camera intrinsic parameter, and T be the camera pose to be solved. , These are the two-dimensional and three-dimensional coordinates of the feature point, respectively.

5. A mapping device for curved or irregular surfaces, characterized in that, include: At least one processing unit is provided, which is connected to a storage unit and a binocular endoscope camera via a bus unit; the storage unit stores a computer program, which, when executed by the processing unit, implements the mapping method for curved and irregular surfaces as described in any one of claims 1-4.

6. A mapping system for curved and irregular surfaces, characterized in that, It includes: an image processing module, a binocular depth estimation module, a 3D point cloud processing module, a 2D feature point extraction module, and a 3D mapping module; The image processing module acquires RGB images of the target area through a binocular camera, and inputs the RGB images and camera parameters from a pre-calibrated camera intrinsic parameter file into the binocular depth estimation module and the two-dimensional feature point extraction module. The binocular depth estimation module uses a binocular depth estimation model to process the depth of each pixel in the RGB image to obtain a depth map, which is then passed to the 3D point cloud processing module. The two-dimensional feature point extraction module uses a two-dimensional feature point extraction model to obtain feature points of the image transmitted by the image processing module, and then inputs the pixel coordinates of the feature points into the three-dimensional mapping module for subsequent mapping. The 3D point cloud processing module combines camera parameters and the depth map obtained from the binocular depth estimation module to construct a 3D point cloud. It then performs gradient distance field processing on the 3D point cloud to obtain an improved truncated distance field, which is provided to the 3D mapping module for subsequent mapping. The improved truncated distance field obtained by gradient distance field processing on the 3D point cloud is represented as follows: ； Where i represents the point number in the 3D point cloud, u represents the horizontal pixel coordinate of the point, v represents the vertical pixel coordinate of the point, depth is the depth map calculated by the stereo depth estimation model, and depth(u,v) is the depth value of the depth map in the (u,v) coordinate system. It is centered around (u,v) The depth values ​​at each location within the region are calculated independently at each of the k*k locations, and the maximum and minimum values ​​in the set are taken; t represents the translation part of the camera pose T to be solved, and max and min are the maximum and minimum values ​​calculated respectively. The 3D mapping module performs subsequent mapping based on the improved truncated distance field and the feature point information extracted by the 2D feature point extraction module. After obtaining the pixel coordinates of the 2D feature points, the 3D mapping module retrieves the improved truncated distance field corresponding to each feature point coordinate, and combines the feature points and the improved truncated distance field to perform improved bundle adjustment processing to obtain the camera pose. The pose corresponding to the camera pose is then calibrated with the starting coordinates to obtain the 3D mapping result. Matching, calibration, and pose estimation are performed within the image sequence acquired by the binocular endoscope camera to achieve temporal 3D mapping. The process of retrieving the improved truncated distance field corresponding to each feature point coordinate and combining the feature points and the improved truncated distance field to perform improved bundle adjustment processing to obtain the camera pose includes: Based on the camera pose, establish the correspondence between the 3D spatial coordinates and 2D coordinates of feature points; determine the first loss function for solving the camera pose based on the correspondence; the first loss function, used for solving the camera pose in 3D mapping using the improved bundle adjustment method; determine the second loss function based on the improved truncation distance field of the feature points, also used for solving the camera pose in 3D mapping using the improved bundle adjustment method; combine the first and second losses to obtain the final total loss function used for mapping irregular surfaces; the calculation process of the second loss function includes: The weight factor of the feature point is calculated based on the angle between the normals and the depth of the feature point: ； ； Where w is the weighting factor, θ is the angle between the normals of the feature points calculated under the current pose of the camera, n is the normal direction of the feature points in three-dimensional space, which is obtained by point cloud PCA, and R is the rotation part of the camera pose T. The second loss function is determined based on the weights and the improved cutoff distance field, and is expressed in the following form: ； Where PSDF and w represent the calculated improved cutoff distance field and weight factor, respectively. The subscript ref indicates that the improved cutoff distance field calculated in the latest iteration is used as the reference value, and the subscript cur indicates that the improved cutoff distance field calculated in the previous iteration is used as the current value.

Images