A spatial positioning method for mapping

Abstract

The application provides a space positioning method for mapping, and relates to the technical field of laser radar positioning. The method comprises the following steps: projecting color grid light to a region to be mapped by color coding, forming a positioning grid, and collecting two images of the on-off grid light by using an image acquisition device, extracting the grid intersection points as control points and obtaining the pixel coordinates of the control points by color recognition; emitting measuring light to the control points from at least three measuring light projection points, and solving the three-dimensional coordinates of the control points by space distance intersection; combining the pixel coordinates and the three-dimensional coordinates of the control points, and inversely calculating the pose of the image acquisition device by using a heteroscedastic total least squares model; identifying the pixel offset of a target feature point and its adjacent control point, combining the pose to convert the actual distance offset of the target feature point, and calculating the three-dimensional coordinates of the target feature point. The application effectively fuses optical active projection and optical ranging, and realizes efficient three-dimensional space positioning.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of laser radar positioning, in particular to a spatial positioning method for mapping. BACKGROUND

[0002] In the field of existing engineering mapping and spatial positioning, obtaining the three-dimensional coordinates of the target to be measured with high precision and high efficiency is crucial for digital modeling and deformation monitoring.

[0003] The traditional visual positioning method mainly relies on passive imaging and feature matching. The basic principle is to use the texture features of the target to be measured under natural light, and to match the same name points and reconstruct three-dimensional through multi-view images. However, the passive method is prone to reconstruction failure due to insufficient feature point extraction or matching error in complex scenes with missing texture, severe light changes or occlusions, causing model voids and precision overshoot. Although there are some attempts to introduce active projection light sources in existing technologies, such as using random speckle or complex color grating for encoding, they usually rely on a large number of color types and fixed, pre-set encoding patterns, and cannot dynamically adjust the recognition strategy according to the on-site environmental interference, so there are still problems of high decoding error rate and heavy computing burden. This not only causes the frequent jumping of control point coordinates, deviating from the real space position, but also introduces cumulative errors to the subsequent camera pose solution, burying long-term measurement risks. Secondly, the feature matching process based on passive images often consumes a lot of computing resources, resulting in insufficient use of geometric information and significant time waste in the feature extraction and optimization process; while some single-point laser scanning-based methods are inefficient due to point-by-point measurement, and are in a secondary or auxiliary position when large-area rapid mapping is required, causing technical idling and reducing the overall efficiency of the system.

[0004] In addition, under outdoor, large-scale or dynamic operating conditions, the lighting conditions of the mapping site and the dynamic changes of the camera pose are severe, and the traditional camera calibration and pose solving method based on static assumptions lacks accurate modeling of the dynamic characteristics of the imaging device itself (such as the rolling shutter effect), making it difficult to dynamically adapt the projection model and real-time imaging geometry, affecting the accuracy and reliability of the mapping results. Therefore, it is necessary to further expand and improve on the basis of existing technologies, and comprehensively consider the color coding robustness of the control points, the dynamic correction of the camera pose, and the spatial constraint relationship between the feature points and the control points, to more comprehensively improve the accuracy and efficiency of spatial positioning. SUMMARY

[0005] In view of the deficiencies of the prior art, the present application provides a spatial positioning method for mapping, comprising:

[0006] projecting a grid light on the area to be mapped by the optical encoding grating of the structured light projection to form a positioning grid;

[0007] A first image and a second image are acquired using an image acquisition device. The first image is acquired when the grid light is projected, and the second image is acquired when the grid light is turned off. Both the first image and the second image include the area to be mapped and the positioning grid.

[0008] Based on the first image and the second image, meridians and parallels are extracted by color recognition, and the coordinates of the intersection of the meridians and parallels are calculated as the pixel coordinates of the control points.

[0009] Based on a preset measurement light projection point, the distance between the projection point of the measurement light and the control point is obtained by emitting measurement light towards the control point;

[0010] Based on the distance measurement values, spatial distance intersection calculations are performed to generate the three-dimensional coordinates of the control points;

[0011] Based on the pixel coordinates and three-dimensional coordinates of the control points, the pose of the image acquisition device is obtained through spatial resection calculation.

[0012] Based on the first image and the second image, the location feature points of the target to be mapped are identified respectively;

[0013] The control point is determined within a preset pixel range, centered on the location feature point;

[0014] Calculate the pixel offset between the positioning feature point and the control point;

[0015] Based on the parameters of the image acquisition device, the pixel offset is converted into an actual distance offset. The parameters of the image acquisition device include interior orientation elements and pose.

[0016] The three-dimensional coordinates of the positioning feature point are calculated based on the three-dimensional coordinates of the control point and the actual distance offset.

[0017] In one optional implementation, the process of projecting grid light onto the area to be mapped using an optically encoded grating projected by structured light to form a positioning grid includes:

[0018] The grid light generates longitude and latitude lines with corresponding color combinations through a preset encoding rule. The corresponding color combinations include: a first color combination and a second color combination.

[0019] The color combination includes at least three different colors, and the first color combination and the second color combination do not overlap.

[0020] The meridians and parallels are projected through a grating according to their corresponding color combinations to generate the positioning grid.

[0021] In one optional implementation, acquiring the first image and the second image via the image acquisition device includes:

[0022] The parameters of the image acquisition device include interior orientation elements, which include at least: principal point coordinates, focal length, and distortion coefficients;

[0023] Based on a preset shooting interval, the image acquisition device is used to acquire a first image and a second image;

[0024] A coordinate system is constructed on a plane, with the meridians as the vertical axis and the parallels as the horizontal axis.

[0025] Based on the coordinate system, the intersections of the meridians and the parallels of latitude constitute control points, and the pixel coordinates of the control points are calculated using an image processing algorithm.

[0026] In one optional implementation, the step of obtaining a distance measurement value between the projection point of the measurement light and the control point by emitting measurement light towards the control point based on a preset measurement light projection point includes:

[0027] At least three measurement light projection points are set up at a preset height, and measurement light beams are emitted to the control points respectively, and the measurement light reflection signals of the control points are received.

[0028] The flight time of the measurement light is calculated by modulating the wave based on the reflected signal of the measurement light received at each measurement light projection point.

[0029] Based on the speed of light and the flight time, the spatial straight-line distance from each measurement light projection point to the control point is calculated.

[0030] In one optional implementation, calculating the spatial straight-line distance from each measurement light projection point to the control point based on the speed of light and the time of flight includes:

[0031] Based on the spatial straight-line distance from each measurement light projection point to the control point, a spatial distance intersection equation system is established by calculating the spatial distance intersection, with the three-dimensional coordinates of the control point as unknowns and the known three-dimensional coordinates of each measurement light projection point and the spatial straight-line distance to the control point as known conditions.

[0032] The system of equations is solved using numerical calculation methods to generate the three-dimensional coordinates of the control points.

[0033] In one optional implementation, the step of obtaining the pose of the image acquisition device by spatial resection calculation based on the pixel coordinates and three-dimensional coordinates of the control points includes:

[0034] Obtain the pixel coordinates and three-dimensional coordinates of each control point in the first image and the second image;

[0035] Based on the color encoding and decoding results, color crosstalk, and exposure differences, the pixel observation confidence parameters of each control point are determined; and based on the signal-to-noise ratio of the distance measurements of multiple light projection points and their spatial geometric weakness, the confidence parameters of the three-dimensional coordinates of each control point are determined.

[0036] Based on the pixel observation confidence parameters and the three-dimensional coordinate confidence parameters, a heteroscedastic total least squares pose inverse model is constructed that simultaneously considers pixel observation errors and three-dimensional coordinate errors. Robust weights with heavy-tailed distribution are introduced for the reprojection residuals to obtain the joint optimization objective for pose solving.

[0037] By employing a decomposed alternating update and damped iterative strategy to minimize the joint optimization objective, based on the small corrections to the six-DOF pose parameters including position coordinates and rotation angles, as well as the three-dimensional coordinates of each control point, an intermediate solution for the pose parameters of the image acquisition device is obtained.

[0038] In an optional implementation, when the image acquisition device is a rolling shutter imaging device, obtaining the pose of the image acquisition device through spatial resection calculation includes:

[0039] Based on the line-by-line exposure characteristics of the rolling shutter imaging device, a dynamic pose model of the image acquisition device that continuously changes with exposure time during image acquisition is established.

[0040] Based on the exposure time of the image row where each control point is located, a dynamic pose projection corresponding to the exposure time is performed on the three-dimensional coordinates of each control point to obtain the control point pixel coordinates to be processed after exposure timing correction.

[0041] Based on the error between the control point pixel coordinates to be processed after exposure timing correction and the actual observed control point pixel coordinates, the pose of the image acquisition device after exposure timing correction is determined by iteratively optimizing the parameters of the dynamic pose model.

[0042] In one optional implementation, converting the pixel offset into an actual distance offset based on the parameters of the image acquisition device includes:

[0043] Obtain the focal length and imaging distance of the image acquisition device, and determine the ratio coefficient between the pixel size and the actual size;

[0044] Based on the scaling factor, the pixel offset is converted into the actual distance offset to be processed in the world coordinate system. The pixel offset includes: horizontal pixel offset and vertical pixel offset.

[0045] The actual distance offset to be processed is a three-dimensional vector, including horizontal distance offset, vertical distance offset and depth distance offset;

[0046] Through the transformation operation, the horizontal pixel offset is converted into the horizontal distance offset of the world coordinate system, and the vertical pixel offset is converted into the vertical distance offset of the world coordinate system.

[0047] Based on the pose of the image acquisition device, the pitch angle of the pose is obtained;

[0048] The depth direction distance offset is calculated using a trigonometric function of the vertical direction distance offset and the pitch angle;

[0049] Based on a preset weighting principle, the actual distance offset of the positioning feature point relative to the control point is obtained by performing a weighted average calculation on all the actual distance offsets to be processed corresponding to the control point.

[0050] In one optional implementation, calculating the three-dimensional coordinates of the positioning feature point based on the three-dimensional coordinates of the control point and the actual distance offset includes:

[0051] Based on the three-dimensional coordinates of the control point and the actual distance offset, the first and second three-dimensional coordinates of the positioning feature point are obtained by spatial vector addition in the first and second images, respectively.

[0052] In an optional implementation, calculating the three-dimensional coordinates of the positioning feature point based on the three-dimensional coordinates of the control point and the actual distance offset further includes:

[0053] A first weight and a second weight are assigned to the first and second three-dimensional coordinates to be processed, respectively.

[0054] Based on the first weight and the second weight, the first and second three-dimensional coordinates to be processed corresponding to the positioning feature points are respectively calculated by averaging the horizontal, vertical and coordinate components to obtain the three-dimensional coordinates of the positioning feature points of the target to be mapped.

[0055] Compared with existing technologies, this invention employs at least three colors of coded gratings. By using a non-overlapping design between the first and second color combinations, the number of colors is significantly reduced, lowering system complexity and color crosstalk, while ensuring effective identification of meridians and parallels. Furthermore, by combining structured light projection and time-of-flight ranging, the three-dimensional coordinates of control points are calculated through spatial distance intersection. A dynamic pose model is established for rolling shutter devices, achieving precise pose correction under exposure timing. Control points are extracted through differential processing of the first and second images, and associated with local pixel ranges centered on the positioning feature points. Through pixel offset conversion and actual distance calculation, the spatial constraints of the control points are fully utilized, effectively improving the accuracy of three-dimensional coordinate reconstruction of the positioning feature points. Attached Figure Description

[0056] Figure 1 A flowchart of a spatial positioning method for surveying provided by the present invention;

[0057] Figure 2 This is a schematic diagram of a spatial positioning grid for surveying provided by the present invention;

[0058] Figure 3 A schematic diagram illustrating a method for generating three-dimensional coordinates of control points provided by the present invention;

[0059] Figure 4 This invention provides a flowchart for calculating the three-dimensional coordinates of a positioning feature point.

[0060] Attached reference numerals: 1. Control point; 201. Measurement light projection point one; 202. Measurement light projection point two; 203. Measurement light projection point three; 3. Positioning feature point; 4. Image acquisition device. Detailed Implementation

[0061] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0062] See Figure 1 The diagram shows a flowchart of a spatial positioning method for surveying provided in an embodiment of this disclosure. The method includes steps S101 to S107, wherein:

[0063] S101: The optical coding grating projected by structured light projects grid light onto the area to be surveyed, forming a positioning grid;

[0064] S102: Acquire a first image and a second image using an image acquisition device. The first image is acquired when the grid light is projected, and the second image is acquired when the grid light is turned off. Both the first image and the second image include the area to be mapped and the positioning grid.

[0065] S103: Based on the first image and the second image, extract the meridians and parallels respectively through color recognition, and calculate the coordinates of the intersection of the meridians and parallels as the pixel coordinates of the control points;

[0066] S104: Based on a preset measurement light projection point, by emitting measurement light towards the control point, the distance measurement value between the projection point of the measurement light and the control point is obtained;

[0067] S105: Based on the distance measurement value, perform spatial distance intersection calculation to generate the three-dimensional coordinates of the control point;

[0068] Based on the pixel coordinates and three-dimensional coordinates of the control points, the pose of the image acquisition device is obtained through spatial resection calculation.

[0069] S106: Based on the first image and the second image, identify the positioning feature points of the target to be mapped, and determine the control point within a preset pixel range with the positioning feature points as the center;

[0070] S107: Calculate the pixel offset between the positioning feature point and the control point;

[0071] Based on the parameters of the image acquisition device, the pixel offset is converted into an actual distance offset. The parameters of the image acquisition device include interior orientation elements and pose.

[0072] The three-dimensional coordinates of the positioning feature point are calculated based on the three-dimensional coordinates of the control point and the actual distance offset.

[0073] As an optional implementation, a grid of light is projected onto the area to be surveyed using an optically encoded grating for structured light projection, forming a positioning grid. In a specific implementation, the grid light generates meridians and parallels with corresponding color combinations according to a preset encoding rule. The corresponding color combinations include a first color combination and a second color combination.

[0074] The color combination includes at least three different colors, and the first color combination and the second color combination do not overlap.

[0075] The meridians and parallels are projected through a grating according to their corresponding color combinations to generate the positioning grid.

[0076] As an optional implementation, the grid light can use a laser as the light source;

[0077] Lasers have advantages such as good directionality, high brightness, and excellent monochromaticity. By equipping multiple laser devices with different wavelengths, various basic colors of light required for encoding can be generated.

[0078] In specific implementation, the grid light is optically encoded according to a preset encoding rule;

[0079] The encoding rule assigns a unified first color combination to all meridians and a unified second color combination, different from the first color combination, to all parallels of latitude.

[0080] Both the first color combination and the second color combination are composed of at least three different basic colors of light, and the specific colors they include are completely different and have no overlap, thus achieving non-overlapping color combinations.

[0081] For example, the first color combination can be selected as red, green and blue, while the second color combination can be selected as other colors besides these three, such as cyan, yellow and magenta.

[0082] Based on the aforementioned encoding rules, during image processing and recognition, it is only necessary to directly and unambiguously determine whether a line belongs to longitude or latitude based on the color information of the pixel, which simplifies the recognition logic and improves robustness.

[0083] In a specific implementation, the color combination is specified to include at least three different colors. By using the three colors to form a non-repeating coding unit in a local range, it is ensured that each grid intersection has a completely different color combination identity among its four directly adjacent intersections (up, down, left, and right).

[0084] For example, consider a grid consisting of infinitely extending meridians and parallels, such that any central intersection point has a color identity that is completely different from the color identities of its four adjacent intersection points above, below, to the left, and to the right, forming the minimum color combinations of meridians and parallels respectively.

[0085] The central intersection is formed by the intersection of a meridian of one color and a parallel of a different color. The intersection relationship defines the unique color identity of the central intersection.

[0086] Moving upwards along the meridian that passes through the central intersection, you will encounter the previous intersection, which is formed by the intersection of the same meridian and another parallel of a different color.

[0087] To ensure that the identity of the central intersection is different from that of the upper intersection, the latitude color of the central intersection must be different from that of the upper intersection.

[0088] Similarly, as you move downwards along this meridian, the color of the latitude line at the central intersection must also be different from the color of the latitude line at the intersection below.

[0089] Moving left along the parallel of latitude that passes through the central intersection, you will encounter another intersection on the left. This intersection is formed by the intersection of a different colored meridian and the same parallel of latitude. To distinguish their identities, the meridian at the central intersection must be a different color than the meridian at the intersection on the left.

[0090] Similarly, moving to the right along this parallel of latitude, the color of the meridian at the central intersection point must also be different from the color of the meridian at the intersection point on the right.

[0091] In summary, to ensure that the color of the central intersection point is completely different from that of its four adjacent intersection points, it is necessary to ensure that the color of the meridian at the central intersection point is not the same as the color of the meridians to its left and right.

[0092] At the same time, the color of the parallel at the central intersection cannot be the same as the color of the parallels above and below it; therefore, three different colors constitute the minimum color combination that satisfies all these distinction requirements.

[0093] The grating, as an optical modulation element, converts the laser source into a dense array of parallel lines with a specific coded pattern.

[0094] In specific implementation, diffractive optical elements can be selected as gratings. The diffractive optical elements can perform wavefront modulation on the incident laser and diffract to generate the entire required meridian array and parallel array with color-coded information.

[0095] For example, during the projection process, the laser beams of the primary color light are first collimated and then jointly irradiate the diffractive optical element;

[0096] The diffractive optical element redistributes the incident laser energy, diffracting it in space to generate a group of meridian beams consisting only of the first color combination and a group of parallel beams consisting only of the second color combination, respectively.

[0097] The beam group is focused and aberration corrected by a projection lens, projecting a non-overlapping color-coded positioning grid onto the area to be mapped.

[0098] See Figure 2 The diagram shown is a schematic representation of a spatial positioning grid for mapping according to an embodiment of this disclosure, including:

[0099] Control point 1, measurement light projection point 1 201, measurement light projection point 2 202 and measurement light projection point 3 203, as well as positioning feature point 3 and image acquisition device 4.

[0100] In a specific implementation, an image acquisition device is used to acquire a first image and a second image. The first image is acquired when the grid light is projected, and the second image is acquired when the grid light is turned off. Both the first image and the second image include the area to be mapped and the positioning grid.

[0101] The image acquisition device includes interior orientation elements, which at least include: principal point coordinates, focal length, and distortion coefficients;

[0102] Based on a preset shooting interval, the image acquisition device is used to acquire a first image and a second image;

[0103] A coordinate system is constructed on the plane, with the meridians as the vertical axis and the parallels of latitude as the horizontal axis. Based on this coordinate system, the intersections of the meridians and parallels of latitude constitute control points, and the pixel coordinates of these control points are calculated using image processing algorithms.

[0104] In specific implementations, the image acquisition device may be a high-resolution industrial digital camera;

[0105] The parameters of the image acquisition device are divided into interior orientation elements and exterior orientation elements. Interior orientation elements describe the parameters of the camera's internal geometry and optical characteristics. It is a fixed set of attributes, including: principal point coordinates, focal length, and distortion coefficients.

[0106] The principal point coordinates are the intersection of the camera's optical axis and the imaging plane, and are the origin reference of the image coordinate system.

[0107] Focal length is the distance from the optical center of the lens to the imaging plane, which determines the camera's magnification and field of view.

[0108] The distortion coefficient is used to quantify the degree of image distortion caused by the lens's manufacturing process.

[0109] The exterior orientation elements describe the position and orientation of the image acquisition device in the object space coordinate system at the moment of shooting.

[0110] In practice, the shooting interval should be much shorter than the time scale of scene changes, usually requiring continuous shooting within milliseconds to ensure that the background scene between the two images remains consistent, so that the pure positioning grid information can be effectively separated through image difference processing.

[0111] In practice, multiple image acquisition devices are deployed to collect images simultaneously from different perspectives.

[0112] A single viewpoint cannot obtain depth information and has blind spots. Multiple devices form a measurement network, which can observe the same control point from multiple directions. The intersection of rays can effectively reduce random errors, and geometric constraints can improve the stability and accuracy of the solution results.

[0113] Each device needs to capture a first image (with the light on) and a second image (with the light off). By performing a difference operation on the two images, the interference of the scene's own texture and constant ambient light is eliminated, and the mesh image is extracted.

[0114] For example, after extracting the grid image, an image processing algorithm is used to define the overall direction of all extracted meridians in the image as the vertical axis of the coordinate system, and the overall direction of all parallels of latitude as the horizontal axis, thus constructing a planar coordinate system.

[0115] For example, each intersection of a meridian and a parallel becomes a control point in that coordinate system.

[0116] As an optional implementation, based on the coordinate system, the grid image after differential processing is binarized by solving the mathematical intersection point of the straight line containing the center line of each meridian and each parallel of latitude, and using the row number and column number of the intersection point in the image pixel array as its final pixel coordinates.

[0117] See Figure 3 The diagram shown is a schematic representation of a method for generating three-dimensional coordinates of control points according to an embodiment of this disclosure, including:

[0118] S301: At least three measurement light projection points are set up based on a preset height, and measurement light beams are emitted to the control points respectively, and the measurement light reflection signals of the control points are received;

[0119] S302: Based on the measurement light reflection signals received at each measurement light projection point, the flight time of the measurement light is calculated using a modulated wave;

[0120] S303: Based on the speed of light and the flight time, the spatial straight-line distance from each measurement light projection point to the control point is calculated;

[0121] S304: Based on the spatial straight-line distance from each measurement light projection point to the control point, the three-dimensional coordinates of the control point are obtained through spatial distance intersection calculation. As an optional implementation, at least three measurement light projection points are arranged based on a preset height, including measurement light projection point one, measurement light projection point two, and measurement light projection point three, which respectively emit measurement light beams to the control point and receive the measurement light reflection signals from the control point;

[0122] In practice, to achieve unambiguous calculation of three-dimensional spatial coordinates, at least three measurement light projection points that are not on the same straight line need to be set up for spatial distance measurement.

[0123] The flight time of the measurement light is calculated by modulating the wave based on the reflected signal of the measurement light received at each measurement light projection point.

[0124] The modulation wave refers to a high-frequency electrical signal that carries information by regularly changing the intensity or phase of a laser according to a specific high-frequency electrical signal waveform.

[0125] In practice, when the laser beam irradiates the surface of the object where the control point is located, diffuse reflection will occur, and part of the reflected light will return along the original path.

[0126] After receiving the reflected light signal, it is converted into an electrical signal by a photoelectric converter.

[0127] By comparing the phase difference between the modulated wave at the time of transmission and the received reflected wave, the propagation time of the laser from transmission to reception, i.e., the flight time, can be calculated.

[0128] In practice, given the speed of light in the air, the precise straight-line distance between the projection point of the measuring light and the target control point is calculated by multiplying the speed of light by the time of flight.

[0129] Based on the speed of light and the flight time, the spatial straight-line distance from each measurement light projection point to the control point is calculated.

[0130] As an optional implementation, spatial distance intersection calculation is performed based on the distance measurement values ​​to generate the three-dimensional coordinates of the control points.

[0131] For example, using the three-dimensional coordinates of the control point as unknowns and the known three-dimensional coordinates of each measurement light projection point and the spatial straight-line distance to the control point as known conditions, a system of spatial distance intersection equations based on distance is established.

[0132] The system of equations is solved using numerical calculation methods to generate the three-dimensional coordinates of the control points.

[0133] As an alternative implementation method, spatial distance intersection uses multiple distance observations from multiple known location points to the same unknown point to inversely determine the position of the unknown point in space through geometric relationships.

[0134] As an optional implementation, the three-dimensional coordinates of the control points can be calculated by spatial spherical intersection;

[0135] For example, the three-dimensional coordinates of the control points are used as unknowns to be solved. The three-dimensional coordinates of each measurement light projection point are calibrated by measurement, and together with the observed distance to the control points, they constitute the known conditions for the calculation.

[0136] With each of the measurement light projection points as the center and the spatial straight-line distance from each to the control point as the radius, three spatial spheres are constructed;

[0137] Calculate the two intersection points of the three spatial spheres;

[0138] Based on the preset spatial location range of the control point, a unique correct intersection point is selected from the two intersection points and used as the three-dimensional coordinates of the control point.

[0139] Based on the pixel coordinates and three-dimensional coordinates of the control points, the pose of the image acquisition device is obtained through spatial resection calculation.

[0140] The pose of the image acquisition device includes a set of parameters with six degrees of freedom, including:

[0141] The position refers to the three-dimensional coordinates of the image acquisition device in the world coordinate system.

[0142] The orientation refers to the rotation angle of the optical axis of the image acquisition device relative to the three coordinate axes of the world coordinate system, which represents the orientation, tilt, and flip of the image acquisition device in space.

[0143] Obtain the pixel coordinates of each control point in the first image and the second image, as well as the three-dimensional coordinates of each control point obtained previously;

[0144] As an optional implementation, the pixel observation confidence parameters of each control point are determined based on the color encoding and decoding results, color crosstalk, and exposure differences.

[0145] The pixel observation confidence parameter is used to quantify the reliability of the pixel coordinates of each control point. It includes: the ease of color encoding and decoding of the control point, the severity of crosstalk from adjacent colors, and the sharpness of the point's outline due to local overexposure or underexposure in the image.

[0146] For example, a clear, interference-free control point that decodes smoothly is assigned a high confidence level.

[0147] As an optional implementation, the control points are assigned color or pattern encoding, and each control point region in the image is decoded using an image recognition algorithm to generate intrinsic indicators such as "confidence" and "ambiguity".

[0148] The output obtained through the decoding algorithm is the color encoding decoding result.

[0149] As an optional implementation, color crosstalk results are obtained by analyzing the color statistical characteristics of the control point region and its adjacent surrounding region in the image. For example, the difference between the main color inside the control point and the background color outside is calculated; the smaller the difference, the more severe the crosstalk.

[0150] As an optional implementation, exposure difference results are obtained by reading the pixel brightness values ​​(or grayscale values) of the area where the control point is located on the image. Within a preset brightness range, the exposure difference is quantified by calculating the deviation between the average brightness of that point and the ideal range.

[0151] In practice, a comprehensive pixel observation confidence score is generated for each control point. The pixel observation confidence score includes:

[0152] Decoding reliability score: assesses the crispness of the decoding process. Points requiring fewer iterations and yielding a unique result receive a high score; points that are hesitant in the decoding process, require multiple attempts, or have multiple candidate results receive a low score.

[0153] Color crosstalk affects the score: Analyze the similarity between the target point's color and the colors of its surrounding areas. A target point with a significant color difference from the surrounding background and sharp boundaries scores high; a target point with colors blended into the background and blurred boundaries scores low.

[0154] Exposure quality score: Checks the pixel brightness value of the small area where the control point is located. Points with brightness within the ideal range and rich detail score high; points with loss of detail due to being too bright (pale) or too dark (black) score low.

[0155] In practice, the three sub-scores are combined into a final pixel observation confidence parameter through weighted aggregation.

[0156] As an optional implementation, the confidence parameters of the three-dimensional coordinates of each control point are determined based on the signal-to-noise ratio of the distance measurements of multiple measurement light projection points and their spatial geometric weakness.

[0157] For example, the signal-to-noise ratio is an internal parameter that characterizes the signal quality when a distance measurement sensor (such as a laser rangefinder or a TOF camera) outputs distance data.

[0158] In the case of multiple distance measurements from multiple measurement light projection points at different spatial locations to the control point, each measurement yields a distance value and a signal-to-noise ratio (SNR) that reflects the quality of the measurement. The higher the SNR, the less the measurement value is affected by noise and the more reliable it is.

[0159] The spatial geometric weakness is calculated based on the three-dimensional coordinates of all measurement light projection points and corresponding control points, solving for the minimum angle between the vectors they form in space.

[0160] For example, if all these points are distributed in an open area in different directions (like the apex of a pyramid), the geometry is robust. If all these points are almost squeezed into a straight line, then the angle between them when viewed from different directions is small;

[0161] The closer the minimum included angle is to a right angle, the lower the weakness; the closer the included angle is to zero, the higher the weakness.

[0162] In practical implementation, the confidence parameter of the three-dimensional coordinates of the control point is used to quantify the reliability of the three-dimensional coordinates of each control point. The confidence includes: the signal-to-noise ratio of multiple ranging signals involved in solving the coordinates of the control point, and the spatial geometric configuration formed by these measurement light projection points and the control point (if all stations are almost collinear with the point, the configuration is weak, the solved coordinates are unstable, and the confidence is low).

[0163] As an optional implementation, the average signal-to-noise ratio of all distance measurements involved in the solution is calculated, and the calculated spatial geometric weakness is converted into a score. These two scores are then combined to generate the final three-dimensional coordinate confidence parameter.

[0164] In specific implementation, based on the pixel observation confidence parameter and the three-dimensional coordinate confidence parameter, a heteroscedastic total least squares pose inverse model is constructed that simultaneously considers pixel observation error and three-dimensional coordinate error. Robust weights with heavy-tailed distribution are introduced for the reprojection residuals to obtain the joint optimization objective for pose solution.

[0165] The heteroscedastic total least squares pose inverse model is solved by constructing a joint optimization objective function, which includes at least two terms:

[0166] The first term is the reprojection error of the control point's three-dimensional coordinates projected onto the image plane. This first term is weighted by the pixel observation confidence parameters of each control point.

[0167] The second item is the correction amount of the three-dimensional coordinates of each control point itself, which is weighted by the confidence parameters of the three-dimensional coordinates of each control point.

[0168] Based on the first and second terms, the pose parameters of the image acquisition device and the correction amount of the three-dimensional coordinates of each control point are simultaneously optimized through an iterative algorithm to find the parameter combination that minimizes the above-mentioned weighted joint objective function value.

[0169] Based on the preset device pose, the three-dimensional coordinates of the control points are projected onto the two-dimensional image plane through the camera imaging model to obtain the preset pixel coordinates;

[0170] The preset pixel coordinates are compared with the observed pixel coordinates actually extracted from the image. The straight-line distance between the two (in pixels) is the projection residual of that point. The smaller the residual, the more accurate the current pose assumption.

[0171] In practice, even after confidence screening, there may still be undetected abnormal control points (such as sudden occlusion or incorrect matching).

[0172] As an optional implementation, in the iterative calculation, the projected residuals of all control points are checked by using a heavy-tailed distribution. For points with particularly large residuals, their weights in the current iteration optimization are automatically and dynamically reduced.

[0173] As an optional implementation, a decomposed alternating update and damped iterative strategy is adopted to minimize the joint optimization objective on the small corrections of the six-degree-of-freedom pose parameters including position coordinates and rotation angles and the three-dimensional coordinates of each control point, so as to obtain the intermediate solution of the pose parameters of the image acquisition device.

[0174] In practice, the pose parameters and the three-dimensional coordinate correction of the control point are solved synchronously through the alternating update of the decomposition and the damping iteration strategy.

[0175] For example, the strategy decomposes the joint optimization problem into two alternating subproblems: first, the control point coordinates are fixed, and the pose parameters are optimized to minimize the objective function; then, the pose parameters are fixed, and the correction amount of each control point coordinate is optimized.

[0176] In practice, a damping factor is introduced to control the step size for each parameter update to prevent iteration divergence.

[0177] By repeatedly executing the above alternating optimization process, the iteration terminates when the change in parameters and the change in the objective function value are below the threshold. At this time, the output pose parameters are the intermediate solution, and the correction amount of the coordinates of each control point is obtained.

[0178] Based on the line-by-line exposure characteristics of the rolling shutter imaging device, a dynamic pose model of the image acquisition device that continuously changes with exposure time during image acquisition is established.

[0179] The rolling shutter imaging device exposes the image sensor line by line from top to bottom at an extremely fast speed, so there is a tiny time difference between the exposure time of the top pixel and the exposure time of the bottom pixel in a frame.

[0180] As an optional implementation, the motion trajectory of the rolling shutter imaging device is fitted by establishing a motion model that changes continuously over time.

[0181] For example, a quadratic polynomial in time can be used to characterize the changes in the camera's position and orientation during the exposure process;

[0182] The coefficients of the quadratic polynomial are the dynamic pose parameters that need to be optimized.

[0183] Based on each control point, the instantaneous pose (position and attitude) of the camera at a specific moment is retrieved from the dynamic pose model according to the exposure time of the image row.

[0184] The instantaneous pose is used to project the three-dimensional coordinates of the control point onto the image plane for exposure timing correction.

[0185] In practice, based on the exposure time of the image row where each control point is located, dynamic pose projection corresponding to the exposure time is performed on the three-dimensional coordinates of each control point to obtain the control point pixel coordinates to be processed after exposure timing correction.

[0186] The theoretical pixel coordinates of the control points to be processed after exposure timing correction are compared with the observed pixel coordinates to obtain the projection residual.

[0187] Based on the error between the control point pixel coordinates to be processed after exposure timing correction and the actual observed control point pixel coordinates, the pose of the image acquisition device after exposure timing correction is determined by iteratively optimizing the parameters of the dynamic pose model.

[0188] As an optional implementation, a standard nonlinear least squares optimization algorithm is used to iteratively adjust the model parameters of the dynamic pose model, so as to minimize the sum of the projection residuals of all control points.

[0189] Through iterative operations, the final dynamic pose model is obtained. Based on the dynamic pose model, with the exposure time of the center row of the image as the reference, the pose corresponding to the exposure time model is used as the final device pose of the image after exposure timing correction.

[0190] See Figure 4 The diagram shown is a flowchart of a method for calculating the three-dimensional coordinates of a positioning feature point according to an embodiment of this disclosure, including:

[0191] Based on the first image and the second image, the location feature points of the target to be mapped are identified respectively;

[0192] The control point is determined within a preset pixel range, centered on the location feature point.

[0193] Calculate the pixel offset between the positioning feature point and the control point.

[0194] In practice, based on a preset interval distance, the outline of the target to be mapped is identified through target detection, and the positioning feature points of the target to be mapped are generated.

[0195] For example, the preset interval distance is set according to the actual size of the target to be measured and the required measurement accuracy.

[0196] On the outline of the target, a sufficient number of feature points are generated at certain spatial intervals to ensure that its geometry can be fully described.

[0197] The interval is set in the range of centimeters to decimeters. Smaller intervals are used for areas with complex shapes to capture details, while larger intervals can be used for flat or regular areas to improve processing efficiency.

[0198] Centered on the positioning feature point, control points corresponding to the positioning feature point are determined in the first image and the second image respectively according to preset rules, and the control points include at least two.

[0199] The pixel offset of the positioning feature point relative to the control point is obtained by calculating the image coordinate difference.

[0200] The pixel offset is a two-dimensional vector, which includes the number of pixels offset in the horizontal direction and the number of pixels offset in the vertical direction.

[0201] For example, the first image and the second image are processed using an instance segmentation model trained based on deep learning;

[0202] The instance segmentation model can classify pixels in an image and identify the complete outline of the target to be mapped.

[0203] As an optional implementation, after extracting the binary contour mask of the target, a contour tracking algorithm is used to obtain its outer boundary polygon, and sampling is performed at equal intervals along this boundary polygon according to the aforementioned preset interval distance. The image coordinates of the sampling points are used as the localization feature points.

[0204] In practice, each positioning feature point is used as the center, and within a preset pixel neighborhood range (such as a circular or rectangular area with a radius of several dozen pixels centered on the point), all control points whose three-dimensional coordinates have been calculated through the aforementioned steps are searched.

[0205] The control points should not be collinear; select 2-4 non-collinear control points that are closest to the positioning feature point.

[0206] Based on the coordinate system, the pixel offset between the positioning feature point and the control point is calculated. This calculation is performed in the image pixel coordinate system. For each control point selected for a specific positioning feature point, the coordinate difference between the two points in the horizontal direction and the coordinate difference in the vertical direction are calculated respectively.

[0207] The two differences together form a two-dimensional vector, which serves as the pixel offset of the localized feature point relative to this control point.

[0208] In a specific implementation, based on the parameters of the image acquisition device, the pixel offset is converted into an actual distance offset. The parameters of the image acquisition device include interior orientation elements and pose.

[0209] The three-dimensional coordinates of the positioning feature point are calculated based on the three-dimensional coordinates of the control point and the actual distance offset.

[0210] As an optional implementation, the focal length and imaging distance of the image acquisition device are obtained, and the ratio coefficient between the pixel size and the actual size is determined;

[0211] For example, the imaging distance is obtained by inverse calculation of the camera pose using the spatial geometric relationship between the control point and the camera's optical center.

[0212] The imaging distance at the control point is calculated based on the camera pose. Combining the focal length and sensor size, the actual size of a single pixel at that imaging distance is calculated using the principle of similar triangles.

[0213] The farther the imaging distance, the larger the actual size represented by a single pixel, and the larger the scaling factor also becomes.

[0214] Based on the scaling factor, the pixel offset is converted into the actual distance offset to be processed in the world coordinate system. The pixel offset includes: horizontal pixel offset and vertical pixel offset.

[0215] The actual distance offset to be processed is a three-dimensional vector, including horizontal distance offset, vertical distance offset and depth distance offset;

[0216] As an optional implementation, through a transformation operation, the horizontal pixel offset is converted into a horizontal distance offset in the world coordinate system, and the vertical pixel offset is converted into a vertical distance offset in the world coordinate system;

[0217] Based on the pose of the image acquisition device, the pitch angle of the pose is obtained;

[0218] The depth direction distance offset is calculated using a trigonometric function of the vertical direction distance offset and the pitch angle.

[0219] For example, the pixel offsets in the horizontal and vertical directions are multiplied by the scaling factor to obtain the horizontal and vertical distance offsets on an imaginary plane perpendicular to the camera's optical axis.

[0220] The distance offset in the depth direction (i.e. along the optical axis) is calculated using trigonometric functions based on the camera's pitch angle during shooting and the already obtained vertical distance offset.

[0221] Each vector obtained from the location feature point to its associated control point is represented as a three-dimensional vector of the actual distance offset to be processed, containing horizontal, vertical, and depth components.

[0222] In practice, based on a preset weight allocation principle, the actual distance offset of the positioning feature point relative to the control point is obtained by performing a weighted average calculation on all the actual distance offsets to be processed corresponding to the control point.

[0223] As an optional implementation, the preset weight allocation principle is based on the spatial distance between the control point and the positioning feature point, and the weight value is inversely proportional to the spatial distance.

[0224] In practice, the spatial distance in the image coordinate system is obtained by calculating the square root of the sum of the squares of the differences between each control point and the positioning feature point in the horizontal and vertical directions.

[0225] Initial weights are assigned to each control point based on an inverse proportional relationship. The smaller the spatial distance between a control point and a feature point in the image, the larger the weight value assigned to it.

[0226] For example, to ensure computational stability, a small constant is introduced based on the distance value to prevent division by zero errors.

[0227] Normalize all initial weights. Divide each initial weight by the sum of all weights, so that the sum of all weights equals one;

[0228] After weighting, a weighted average is calculated for each of the three coordinate components of the actual distance offset. The final offset in each direction is equal to the sum of the products of the offset of each control point in that direction and its normalized weight.

[0229] By taking a weighted average of all the components of all vectors, a final value representing the actual distance offset of the positioning feature point relative to the control point is obtained.

[0230] In a specific implementation, based on the three-dimensional coordinates of the control point and the actual distance offset, the first and second three-dimensional coordinates of the positioning feature point are obtained by spatial vector addition in the first and second images, respectively.

[0231] For example, the known three-dimensional coordinates of the selected control point are used as the starting point of the vector, and the calculated actual distance offset is used as the vector. The three-dimensional coordinates of the positioning feature point can be obtained by vector addition in three-dimensional space.

[0232] The vector addition operation is performed on the data corresponding to the first image and the second image respectively, to generate a first three-dimensional coordinate to be processed and a second three-dimensional coordinate to be processed.

[0233] In specific implementation, a first weight and a second weight are assigned to the first three-dimensional coordinates to be processed and the second three-dimensional coordinates to be processed, respectively;

[0234] For example, the first weight and the second weight are set to be equal weights, and the arithmetic mean of the horizontal, vertical and coordinate components of the first and second three-dimensional coordinates to be processed corresponding to the positioning feature point is calculated to obtain the three-dimensional coordinates of the positioning feature point of the target to be mapped.

[0235] Those skilled in the art will understand that, in the methods described above in the specific embodiments, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic. It should be understood that determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information.

[0236] In the description of this specification, the terms "exemplary," "for example," "specifically," etc., refer to a specific feature, structure, material, or characteristic described in connection with that embodiment or example, which is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

Claims

1. A spatial positioning method for surveying, characterized in that, The method includes: The optical coding grating of structured light projection projects grid light onto the area to be mapped, forming a positioning grid; A first image and a second image are acquired using an image acquisition device. The first image is acquired when the grid light is projected, and the second image is acquired when the grid light is turned off. Both the first image and the second image include the area to be mapped and the positioning grid. Based on the first image and the second image, meridians and parallels are extracted by color recognition, and the coordinates of the intersection of the meridians and parallels are calculated as the pixel coordinates of the control points. Based on a preset measurement light projection point, the distance between the projection point of the measurement light and the control point is obtained by emitting measurement light towards the control point; Based on the distance measurement values, spatial distance intersection calculations are performed to generate the three-dimensional coordinates of the control points; Based on the pixel coordinates and three-dimensional coordinates of the control points, the pose of the image acquisition device is obtained through spatial resection calculation. Based on the first image and the second image, the location feature points of the target to be mapped are identified respectively; The control point is determined within a preset pixel range, centered on the location feature point; Calculate the pixel offset between the positioning feature point and the control point; Based on the parameters of the image acquisition device, the pixel offset is converted into an actual distance offset. The parameters of the image acquisition device include interior orientation elements and pose. The three-dimensional coordinates of the positioning feature point are calculated based on the three-dimensional coordinates of the control point and the actual distance offset.

2. The spatial positioning method for surveying and mapping according to claim 1, characterized in that, The process of projecting grid light onto the area to be mapped using an optically encoded grating via structured light projection to form a positioning grid includes: The grid light generates longitude and latitude lines with corresponding color combinations through a preset encoding rule. The corresponding color combinations include: a first color combination and a second color combination. The color combination includes at least three different colors, and the first color combination and the second color combination do not overlap. The meridians and parallels are projected through a grating according to their corresponding color combinations to generate the positioning grid.

3. The spatial positioning method for surveying according to claim 1, characterized in that, The acquisition of the first and second images via the image acquisition device includes: The parameters of the image acquisition device include interior orientation elements, which include at least: principal point coordinates, focal length, and distortion coefficients; Based on a preset shooting interval, the image acquisition device is used to acquire a first image and a second image; A coordinate system is constructed on a plane, with the meridians as the vertical axis and the parallels as the horizontal axis. Based on the coordinate system, the intersections of the meridians and the parallels of latitude constitute control points, and the pixel coordinates of the control points are calculated using an image processing algorithm.

4. A spatial positioning method for surveying according to claim 1, characterized in that, The method of obtaining the distance measurement value between the projection point of the measurement light and the control point by emitting measurement light towards the control point based on a preset measurement light projection point includes: At least three measurement light projection points are set up at a preset height, and measurement light beams are emitted to the control points respectively, and the measurement light reflection signals of the control points are received. The flight time of the measurement light is calculated by modulating the wave based on the reflected signal of the measurement light received at each measurement light projection point. Based on the speed of light and the flight time, the spatial straight-line distance from each measurement light projection point to the control point is calculated.

5. A spatial positioning method for surveying according to claim 4, characterized in that, The spatial straight-line distance from each measurement light projection point to the control point, calculated based on the speed of light and the time of flight, includes: Based on the spatial straight-line distance from each measurement light projection point to the control point, a spatial distance intersection equation system is established by calculating the spatial distance intersection, with the three-dimensional coordinates of the control point as unknowns and the known three-dimensional coordinates of each measurement light projection point and the spatial straight-line distance to the control point as known conditions. The system of equations is solved using numerical calculation methods to generate the three-dimensional coordinates of the control points.

6. A spatial positioning method for surveying according to claim 1, characterized in that, The pose of the image acquisition device is obtained by spatial resection calculation based on the pixel coordinates and three-dimensional coordinates of the control points. Obtain the pixel coordinates and three-dimensional coordinates of each control point in the first image and the second image; Based on the color encoding and decoding results, color crosstalk, and exposure differences, the pixel observation confidence parameters of each control point are determined; and based on the signal-to-noise ratio of the distance measurements of multiple light projection points and their spatial geometric weakness, the confidence parameters of the three-dimensional coordinates of each control point are determined. Based on the pixel observation confidence parameters and the three-dimensional coordinate confidence parameters, a heteroscedastic total least squares pose inverse model is constructed that simultaneously considers pixel observation errors and three-dimensional coordinate errors. Robust weights with heavy-tailed distribution are introduced for the reprojection residuals to obtain the joint optimization objective for pose solving. By employing a decomposed alternating update and damped iterative strategy to minimize the joint optimization objective, based on the small corrections to the six-DOF pose parameters including position coordinates and rotation angles, as well as the three-dimensional coordinates of each control point, an intermediate solution for the pose parameters of the image acquisition device is obtained.

7. A spatial positioning method for surveying according to claim 6, characterized in that, When the image acquisition device is a rolling shutter imaging device, the step of obtaining the pose of the image acquisition device through spatial resection calculation includes: Based on the line-by-line exposure characteristics of the rolling shutter imaging device, a dynamic pose model of the image acquisition device that continuously changes with exposure time during image acquisition is established. Based on the exposure time of the image row where each control point is located, a dynamic pose projection corresponding to the exposure time is performed on the three-dimensional coordinates of each control point to obtain the control point pixel coordinates to be processed after exposure timing correction. Based on the error between the control point pixel coordinates to be processed after exposure timing correction and the actual observed control point pixel coordinates, the pose of the image acquisition device after exposure timing correction is determined by iteratively optimizing the parameters of the dynamic pose model.

8. A spatial positioning method for surveying according to claim 1, characterized in that, The process of converting the pixel offset into an actual distance offset based on the parameters of the image acquisition device includes: Obtain the focal length and imaging distance of the image acquisition device, and determine the ratio coefficient between the pixel size and the actual size; Based on the scaling factor, the pixel offset is converted into the actual distance offset to be processed in the world coordinate system. The pixel offset includes: horizontal pixel offset and vertical pixel offset. The actual distance offset to be processed is a three-dimensional vector, including horizontal distance offset, vertical distance offset and depth distance offset; Through the transformation operation, the horizontal pixel offset is converted into the horizontal distance offset of the world coordinate system, and the vertical pixel offset is converted into the vertical distance offset of the world coordinate system. Based on the pose of the image acquisition device, the pitch angle of the pose is obtained; The depth direction distance offset is calculated using a trigonometric function of the vertical direction distance offset and the pitch angle; Based on a preset weighting principle, the actual distance offset of the positioning feature point relative to the control point is obtained by performing a weighted average calculation on all the actual distance offsets to be processed corresponding to the control point.

9. A spatial positioning method for surveying according to claim 1, characterized in that, The calculation of the three-dimensional coordinates of the positioning feature point based on the three-dimensional coordinates of the control point and the actual distance offset includes: Based on the three-dimensional coordinates of the control point and the actual distance offset, the first and second three-dimensional coordinates of the positioning feature point are obtained by spatial vector addition in the first and second images, respectively.

10. A spatial positioning method for surveying according to claim 9, characterized in that, The calculation of the three-dimensional coordinates of the positioning feature point based on the three-dimensional coordinates of the control point and the actual distance offset further includes: A first weight and a second weight are assigned to the first and second three-dimensional coordinates to be processed, respectively. Based on the first weight and the second weight, the first and second three-dimensional coordinates to be processed corresponding to the positioning feature points are respectively calculated by averaging the horizontal, vertical and coordinate components to obtain the three-dimensional coordinates of the positioning feature points of the target to be mapped.

Images