Ranging method and device based on nonlinear camera model, and storage medium
Through the method based on the nonlinear camera model, the problem of low ranging efficiency and accuracy in power grid inspection is solved, and the efficient tree barrier warning of monocular images in power grid inspection is realized, which improves the safety of power grid operation and maintenance reliability.
Patent Information
- Application Number
- CN202410973743.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-19
- Publication Date
- 2024-11-08
AI Technical Summary
The existing ranging method has low ranging efficiency and low accuracy in daily operation and maintenance inspection of the power grid. Especially when using monocular image recognition technology, it is difficult to match feature points, resulting in low tree barrier warning efficiency and accuracy.
Using the ranging method based on the nonlinear camera model, the visible light image and three-dimensional point cloud data of the target scene are obtained, and the pre-trained nonlinear camera model is used to perform linear and nonlinear transformation processing to establish the image pixel coordinates and point cloud coordinates. The corresponding relationship between the objects to be measured is determined.
The ranging efficiency and accuracy are improved, and the accurate ranging of the objects to be measured in the scene is achieved through monocular images, which improves the efficiency and accuracy of grid tree barrier warning.
Smart Images

Figure CN118915038A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of power equipment safety detection, and in particular to a distance measurement method, device and storage medium based on a nonlinear camera model. Background Art
[0002] External force damage and tree obstacles are one of the major hidden dangers to the safe and stable operation of field transmission line systems. They are key factors affecting the power supply quality and reliability of power grid systems. They are also the key tasks of daily operation and maintenance inspections of power systems. The key technology for early warning of external force damage and tree obstacles is to measure the distance between power equipment and external hidden dangers.
[0003] At present, the methods for distance measurement in daily power grid operation and maintenance inspection are as follows: 1. Scan the power system equipment components through the laser radar equipment carried by the helicopter to obtain the three-dimensional point cloud data near the components and channels of the transmission line. Classify the equipment area and obstacle area in the point cloud by manual or automatic methods, and then search for the closest distance between the equipment point cloud and the obstacle point to give a hidden danger warning; 2. Through the binocular image stereo vision algorithm, the transmission line and wire point cloud are reconstructed using the parallax of the left and right views, and the wire point cloud and tree point cloud are classified according to the image recognition technology, and the closest distance between the wire point cloud and the tree point is searched to give a hidden danger warning. In the method of using laser point cloud data for hidden danger warning, it is necessary to use drones and other equipment equipped with laser radar to scan the transmission line channel and then classify the point cloud. The point cloud classification process requires human intervention, and it is difficult to achieve automatic and accurate classification of point clouds. Laser point cloud data cannot be reused multiple times. In the binocular image stereo vision tree obstacle warning method, it is necessary to generate a disparity map through global dense matching of binocular images. When the baseline distance of the binocular camera is too long or the ranging range is too large, the feature points of the left and right view images are subject to many interference factors, making it difficult to complete the matching of feature points and the disparity map generation. This results in low ranging efficiency and accuracy. Summary of the invention
[0004] The present invention provides a distance measurement method, device and storage medium based on a nonlinear camera model to solve the problems of low distance measurement efficiency and low accuracy of the existing distance measurement methods.
[0005] According to one aspect of the present invention, a distance measurement method based on a nonlinear camera model is provided, the method comprising:
[0006] Acquire a visible light image and three-dimensional point cloud data corresponding to a target scene, wherein the three-dimensional point cloud data includes a plurality of point cloud coordinate points in a world coordinate system;
[0007] Performing linear transformation and nonlinear transformation on the point cloud coordinate points through a pre-trained nonlinear camera model to obtain pixel coordinate points in an image pixel coordinate system corresponding to the point cloud coordinate points;
[0008] The pixel coordinate points are projected into the visible light image to obtain a coordinate point correspondence relationship between the point cloud coordinate points and the pixel coordinate points corresponding to them, and the distance data between at least two objects to be measured are determined based on the coordinate point correspondence relationship and the pixel coordinate points corresponding to at least two objects to be measured.
[0009] According to another aspect of the present invention, a distance measuring device based on a nonlinear camera model is provided, the device comprising:
[0010] A data acquisition module, used to acquire a visible light image and three-dimensional point cloud data corresponding to a target scene, wherein the three-dimensional point cloud data includes a plurality of point cloud coordinate points in a world coordinate system;
[0011] A coordinate conversion module, used for performing linear transformation processing and nonlinear transformation processing on the point cloud coordinate points through a pre-trained nonlinear camera model to obtain pixel coordinate points in an image pixel coordinate system corresponding to the point cloud coordinate points;
[0012] The ranging module is used to project the pixel coordinate points into the visible light image to obtain a coordinate point correspondence relationship between the point cloud coordinate points and the pixel coordinate points corresponding to them, and determine the distance data between at least two objects to be measured based on the coordinate point correspondence relationship and the pixel coordinate points corresponding to at least two objects to be measured.
[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:
[0014] at least one processor; and
[0015] a memory communicatively connected to the at least one processor; wherein,
[0016] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the ranging method based on the nonlinear camera model described in any embodiment of the present invention.
[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the ranging method based on the nonlinear camera model described in any embodiment of the present invention when executed.
[0018] The technical solution of the embodiment of the present invention is to obtain a visible light image and three-dimensional point cloud data corresponding to a target scene, wherein the three-dimensional point cloud data includes a plurality of point cloud coordinate points in a world coordinate system; a corresponding relationship between the visible light image and the three-dimensional point cloud data of the same scene can be established; then, the point cloud coordinate points are subjected to linear transformation processing and nonlinear transformation processing by a pre-trained nonlinear camera model to obtain pixel coordinate points in an image pixel coordinate system corresponding to the point cloud coordinate points; the pixel coordinate points of the point cloud coordinate points in the image pixel coordinate system are accurately determined; finally, the pixel coordinate points are projected into the visible light image to obtain a coordinate point correspondence relationship between the point cloud coordinate points and the pixel coordinate points corresponding to them, and the distance data between at least two objects to be measured are determined based on the coordinate point correspondence relationship and the pixel coordinate points corresponding to at least two objects to be measured, thereby solving the problem of low ranging efficiency and low accuracy of the existing ranging method, realizing ranging of at least two objects to be measured in a scene by using a monocular image, and achieving the beneficial effect of improving ranging efficiency and accuracy.
[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0021] Figure 1 is a flow chart of a distance measurement method based on a nonlinear camera model provided according to Embodiment 1 of the present invention;
[0022] Figure 2a is a flow chart of a distance measurement method based on a nonlinear camera model provided according to Embodiment 2 of the present invention;
[0023] Figure 2b is a flow chart of an optional example of a distance measurement method based on a nonlinear camera model provided according to Embodiment 2 of the present invention;
[0024] Figure 2c is a sample schematic diagram of an object to be measured in an optional example of a distance measurement method based on a nonlinear camera model provided in accordance with Embodiment 2 of the present invention;
[0025] Figure 2dis a sample schematic diagram after the wire and ground point cloud are separated according to an optional example of a distance measurement method based on a nonlinear camera model provided in Embodiment 2 of the present invention;
[0026] Figure 2e is a sample schematic diagram of point distance measurement of an optional example of a distance measurement method based on a nonlinear camera model provided in Embodiment 2 of the present invention;
[0027] Figure 3 is a structural schematic diagram of a distance measurement device based on a nonlinear camera model provided according to Embodiment 3 of the present invention;
[0028] Figure 4 It is a structural schematic diagram of an electronic device for implementing the distance measurement method based on a nonlinear camera model according to an embodiment of the present invention. DETAILED DESCRIPTION
[0029] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0031] Embodiment 1
[0032] Figure 1 A flowchart of a distance measurement method based on a nonlinear camera model is provided for the first embodiment of the present invention. This embodiment is applicable to obstacle distance measurement in industrial scenarios. The method can be performed by a distance measurement device based on a nonlinear camera model. The distance measurement device based on a nonlinear camera model can be implemented in the form of hardware and / or software. The distance measurement device based on a nonlinear camera model can be configured in an electronic device. Figure 1 As shown, the method includes:
[0033] S110, acquiring a visible light image and three-dimensional point cloud data corresponding to a target scene, wherein the three-dimensional point cloud data includes a plurality of point cloud coordinate points in a world coordinate system.
[0034] The target scene can be understood as the target shooting scene, for example, the scene where the power distribution equipment is located. The visible light image can be understood as a two-dimensional image. The point cloud coordinate point can be understood as a three-dimensional coordinate point.
[0035] Optionally, acquiring the visible light image and three-dimensional point cloud data corresponding to the target scene includes: acquiring the visible light image corresponding to the target scene based on a visible light camera; and acquiring the three-dimensional point cloud data corresponding to the target scene based on a lidar.
[0036] Specifically, the inspection drone is equipped with a visible light camera to obtain real-time image data on the flight route. The camera's shooting angle and speed are controlled by the gimbal, and the image transmission device on the aircraft is used to transmit the image back to the base station for analysis. The image can also be processed in real time by the embedded device on the aircraft. The laser radar is used to obtain the point cloud coordinates of the scene.
[0037] S120, performing linear transformation processing and nonlinear transformation processing on the point cloud coordinate points through a pre-trained nonlinear camera model to obtain pixel coordinate points in an image pixel coordinate system corresponding to the point cloud coordinate points.
[0038] The pixel coordinate point can be understood as a coordinate point in a visible light image. The nonlinear camera model includes a camera model and a distortion model.
[0039] Specifically, the point cloud coordinate points are input into a pre-trained nonlinear camera model to transform the point cloud coordinate points, and the pixel coordinate points corresponding to the point cloud coordinate points are output.
[0040] Optionally, before the point cloud coordinate points are subjected to linear transformation processing and nonlinear transformation processing by a pre-trained nonlinear camera model, the method further includes: obtaining camera-associated parameters and a preset number of sample coordinate points, wherein the camera-associated parameters include camera intrinsic parameters, distortion parameters and camera extrinsic parameters, and the sample coordinate points include sample pixel coordinate points and sample point cloud coordinate points; defining an initial nonlinear camera model based on the camera intrinsic parameters, the distortion parameters and the camera extrinsic parameters, and aligning the sample pixel coordinate points with the sample point cloud coordinate points to obtain multiple groups of sample point pairs; iteratively optimizing the initial nonlinear camera model based on a nonlinear optimization algorithm and the sample point pairs; and completing model optimization when the model loss function converges to obtain a trained nonlinear camera model.
[0041] Specifically, the key points in the sample image are calculated by a feature point detection algorithm, a preset number of sample pixel coordinate points are obtained by an image target detection algorithm, and a preset number of sample point cloud coordinate points are obtained by a three-dimensional key point detection algorithm. The sample pixel coordinate points and the sample point cloud coordinate points are manually aligned to obtain multiple groups of sample point pairs. The camera intrinsic parameters, distortion parameters and camera extrinsic parameters of the camera are obtained; an initial nonlinear camera model is defined based on the camera intrinsic parameters, the distortion parameters and the camera extrinsic parameters. The initial nonlinear camera model is iteratively optimized based on a nonlinear optimization algorithm and the sample point pairs until the model function converges to complete the model optimization, and the optimized model is used as a trained nonlinear camera model.
[0042] In the embodiment of the present invention, the three-dimensional point cloud coordinate points can be accurately converted into two-dimensional pixel coordinate points by training the nonlinear camera model, so as to facilitate the subsequent determination of the distance between the objects to be measured based on the pixel coordinates, thereby improving the ranging efficiency and accuracy.
[0043] S130, projecting the pixel coordinate points into the visible light image to obtain a coordinate point correspondence relationship between the point cloud coordinate points and the pixel coordinate points corresponding to them, and determining distance data between at least two objects to be measured based on the coordinate point correspondence relationship and the pixel coordinate points corresponding to at least two objects to be measured.
[0044] Specifically, the pixel coordinate points are projected into the visible light image to obtain multiple groups of coordinate point correspondences between each point cloud coordinate point and its corresponding pixel coordinate point. Using image target detection technology such as target detection algorithm, potential external breach objects in the scene image are detected, and the potential external breach objects are used as objects to be measured. The distance data between any two objects to be measured can be determined based on the coordinate point correspondence and the pixel coordinate points corresponding to at least two objects to be measured. It can be understood that the objects to be measured may include ground objects, such as cars and trees, and may also include aerial objects, such as wires.
[0045] Optionally, determining the distance data between at least two objects to be measured based on the coordinate point correspondence and pixel coordinate points corresponding to at least two objects to be measured includes: determining point cloud coordinate points corresponding to the pixel coordinate points of the objects to be measured based on the pixel coordinate points of the objects to be measured and the coordinate point correspondence; and calculating the distance data between at least two objects to be measured based on the point cloud coordinate points of at least two objects to be measured.
[0046] Specifically, the pixel coordinate points of the object to be measured are obtained, and based on the pixel coordinate points of the object to be measured, the point cloud coordinate points corresponding to the pixel coordinate points of the object to be measured are searched in the coordinate point correspondence relationship; and the distance data between at least two objects to be measured are calculated based on the point cloud coordinate points of at least two objects to be measured.
[0047] The technical solution of the embodiment of the present invention is to obtain a visible light image and three-dimensional point cloud data corresponding to a target scene, wherein the three-dimensional point cloud data includes a plurality of point cloud coordinate points in a world coordinate system; a corresponding relationship between the visible light image and the three-dimensional point cloud data of the same scene can be established; then, the point cloud coordinate points are subjected to linear transformation processing and nonlinear transformation processing by a pre-trained nonlinear camera model to obtain pixel coordinate points in an image pixel coordinate system corresponding to the point cloud coordinate points; the pixel coordinate points of the point cloud coordinate points in the image pixel coordinate system are accurately determined; finally, the pixel coordinate points are projected into the visible light image to obtain a coordinate point correspondence relationship between the point cloud coordinate points and the pixel coordinate points corresponding to them, and the distance data between at least two objects to be measured are determined based on the coordinate point correspondence relationship and the pixel coordinate points corresponding to at least two objects to be measured, thereby solving the problem of low ranging efficiency and low accuracy of the existing ranging method, realizing ranging of at least two objects to be measured in a scene by using a monocular image, and achieving the beneficial effect of improving ranging efficiency and accuracy.
[0048] Embodiment 2
[0049] Figure 2a A flowchart of a distance measurement method based on a nonlinear camera model is provided for the second embodiment of the present invention. This embodiment is a further optimization of how to perform linear transformation processing and nonlinear transformation processing on the point cloud coordinate points through the nonlinear camera model in the above embodiment to obtain pixel coordinate points in the image pixel coordinate system corresponding to the point cloud coordinate points. Optionally, the nonlinear camera model includes a camera model and a distortion model; the linear transformation processing and nonlinear transformation processing on the point cloud coordinate points through the nonlinear camera model to obtain pixel coordinate points in the image pixel coordinate system corresponding to the point cloud coordinate points includes: performing linear transformation processing on the point cloud coordinate points through the camera model to convert the point cloud coordinate points into image coordinate points; performing nonlinear transformation processing on the image coordinate points through the distortion model to convert the image coordinate points into pixel coordinate points in the image pixel coordinate system.
[0050] like Figure 2a As shown, the method includes:
[0051] S210, acquiring a visible light image and three-dimensional point cloud data corresponding to a target scene, wherein the three-dimensional point cloud data includes a plurality of point cloud coordinate points in a world coordinate system.
[0052] S220 , performing linear transformation processing on the point cloud coordinate points through a camera model to convert the point cloud coordinate points into image coordinate points.
[0053] Optionally, performing linear transformation on the point cloud coordinate points through the camera model to convert the point cloud coordinate points into image coordinate points includes: performing rotation and translation processing on the point cloud coordinate points through the camera model to convert the point cloud coordinate points into camera coordinate points in a camera coordinate system, and projecting the camera coordinate points to the image coordinate system to obtain image coordinate points corresponding to the point cloud coordinate points.
[0054] Specifically, the point cloud coordinate points are first converted into camera coordinate points in the camera coordinate system through the camera ink, and then the camera coordinate points are converted into image coordinate points.
[0055] For example, the undistorted coordinates (U, V) in the image pixel coordinate system (UOV coordinate system) fall on (Ud, Vd) in the UOV coordinate system after radial distortion and tangential distortion. That is, the transformation relationship between the real image imgR and the distorted image imgD is as follows:
[0056] imgR(U,V)=imgD(Ud,Vd)
[0057] Among them, imgR(U,V) is the pixel coordinate of the real image, and imgD(Ud,Vd) is the pixel coordinate of the distorted image.
[0058] Since the transformation process from the imaging plane of the camera to the image pixel coordinates is a linear transformation, that is, a transformation process involving an intrinsic parameter matrix, the camera model used in this embodiment assumes that the distortion of the camera occurs on a normalized plane.
[0059] The following is the process of projecting a point A in the world coordinate system to the image pixel coordinate system through a pinhole camera model containing a multi-layer perceptron. First, point A is transformed into the camera coordinate system by rotation and translation, and the formula is as follows:
[0060]
[0061] in, is the camera coordinate point, Represents a three-dimensional coordinate point, is the camera external parameter matrix, R is the rotation matrix, T is the translation vector, 0 T is the zero vector.
[0062] The transformation from the camera coordinate system to the image coordinate system (i.e., from the camera coordinate point to the image coordinate point) is expressed as follows:
[0063]
[0064] in, is the camera internal parameter matrix, f is the camera focal length, is the image coordinate point.
[0065] S230: Perform nonlinear transformation processing on the image coordinate points through a distortion model to convert the image coordinate points into pixel coordinate points in an image pixel coordinate system.
[0066] Optionally, the image coordinate points are subjected to nonlinear transformation processing through the distortion model to convert the image coordinate points into pixel coordinate points in an image pixel coordinate system, including: scaling and translating the image coordinate points through the distortion model to convert the image coordinate points into initial pixel coordinate points in the image pixel coordinate system, and offset correcting the initial pixel coordinate points to obtain pixel coordinate points in the image pixel coordinate system corresponding to the point cloud coordinate points.
[0067] Optionally, the distortion model includes two linear layers and an activation function layer; the distortion model performs scaling and translation processing on the image coordinate points through the following formula to convert the image coordinate points into initial pixel coordinate points in the image pixel coordinate system:
[0068]
[0069] Among them, w1 is the transformation parameter of the first linear layer, b1 is the bias of the first linear layer, w2 is the transformation parameter of the second linear layer, b1 is the bias of the second linear layer, is the initial pixel coordinate to which the distortion parameters are added.
[0070] In an embodiment of the present invention, two linear layers plus a sigmoid activation function are used to describe the nonlinear distortion of the camera, and the radial distortion and tangential distortion factors of the camera are uniformly considered. With the goal of minimizing the camera projection error, the external parameters, internal parameters and distortion parameters in the camera model are uniformly optimized to achieve the adjustment of the parameters of each part in the camera projection process.
[0071] Furthermore, by performing offset correction on the initial pixel coordinate point, a pixel coordinate point in the image pixel coordinate system corresponding to the point cloud coordinate point is obtained, which is expressed by the following formula:
[0072]
[0073] Among them, _u and _v are the pixel coordinates after distortion, f x and f y is the focal length of the camera, c x and c y It is the vertical and horizontal offset of the image origin relative to the optical center imaging point.
[0074] In the embodiment of the present invention, the initial pixel coordinates are corrected based on the vertical and horizontal offsets of the origin relative to the optical center imaging point to obtain the pixel coordinates restored from the actual camera shooting, thereby improving the accuracy of the pixel coordinate points.
[0075] S240, projecting the pixel coordinate points into the visible light image to obtain a coordinate point correspondence relationship between the point cloud coordinate points and the pixel coordinate points corresponding to them, and determining distance data between at least two objects to be measured based on the coordinate point correspondence relationship and the pixel coordinate points corresponding to at least two objects to be measured.
[0076] The technical solution of the embodiment of the present invention is to convert the point cloud coordinate points into image coordinate points by performing linear transformation processing on the point cloud coordinate points through a camera model, and convert the image coordinate points into pixel coordinate points in the image pixel coordinate system by performing nonlinear transformation processing on the image coordinate points through a distortion model. The point cloud coordinate points are accurately converted into pixel coordinate points, thereby improving the conversion accuracy of the pixel coordinate points.
[0077] Figure 2b A flowchart of an optional example of a distance measurement method based on a nonlinear camera model is provided. Figure 2b As shown, the distance measurement method based on the nonlinear camera model specifically includes the following steps:
[0078] Step 1. Obtain visible light images and point cloud data
[0079] The inspection drone is equipped with a visible light camera to obtain real-time image data on the flight route. The camera's shooting angle and speed are controlled by the gimbal, and the image transmission device on the aircraft is used to transmit the image back to the base station for analysis. The image can also be processed in real time by the embedded device on the aircraft. The point cloud coordinates of the scene are obtained using a laser radar.
[0080] Step 2. Train the nonlinear camera model
[0081] The key points in the sample image are calculated by the feature point detection algorithm, a preset number of sample pixel coordinate points are obtained by the image target detection algorithm, and a preset number of sample point cloud coordinate points are obtained by the three-dimensional key point detection algorithm. The sample pixel coordinate points and the sample point cloud coordinate points are manually aligned to obtain multiple groups of sample point pairs. The camera intrinsic parameters, distortion parameters and camera extrinsic parameters of the camera are obtained; an initial nonlinear camera model is defined based on the camera intrinsic parameters, the distortion parameters and the camera extrinsic parameters. The initial nonlinear camera model is iteratively optimized based on the nonlinear optimization algorithm and the sample point pairs until the model function converges to complete the model optimization, and the optimized model is used as the trained nonlinear camera model.
[0082] Exemplarily, several pairs of two-dimensional and three-dimensional points are manually selected to identify the two-dimensional information of potential external damage factors (such as excavators and trees) in the image, and the three-dimensional coordinates of the points to be measured are obtained by combining the intrinsic and extrinsic parameters of the camera using the mapping relationship from the image coordinates to the three-dimensional point cloud coordinates, and then the distance between the external damage factors and the tree obstacle points and the power equipment is calculated.
[0083] Step 2.1 Selection of registration points
[0084] In order to improve the accuracy of the registration result and the real-time performance of the algorithm, the sample pixel coordinate points and the sample point cloud coordinate points are manually registered to obtain multiple groups of sample point pairs, as shown in Table 1. The first two columns in Table 1 are the registered sample pixel coordinate points manually selected in the visible light picture of a building in the scene. In order to simplify the process of registration point detection and calculation, we selected the corner points with texture mutation in the image; the right side of Table 1 (the last three columns) are the point cloud coordinates of the building corresponding to the sample pixel coordinate points.
[0085]
[0086]
[0087] Table 1
[0088] Step 2.2 Nonlinear camera model
[0089] In computer vision, this embodiment starts from the perspective of photogrammetric adjustment theory, comprehensively considers the linear transformation and nonlinear transformation in the camera projection model, uses two linear layers plus an activation function to describe the nonlinear distortion of the camera, and uniformly considers the radial distortion and tangential distortion factors of the camera. With the goal of minimizing the camera projection error, the external parameters, internal parameters and distortion parameters in the camera model are uniformly optimized to achieve the adjustment between the parameters of each part in the camera projection process.
[0090] Step 2.2.1, Camera projection principle
[0091] The imaging process is essentially a conversion between four coordinate systems. First, a point in the world coordinate system is converted to the camera coordinate system, then projected to the imaging plane (image physical coordinate system), and finally the coordinate data on the imaging plane is converted to the image plane (image pixel coordinate system).
[0092] For example, the undistorted coordinates (U, V) in the image pixel coordinate system (UOV coordinate system) fall on (Ud, Vd) in the UOV coordinate system after radial distortion and tangential distortion. That is, the transformation relationship between the real image imgR and the distorted image imgD is as follows:
[0093] imgR(U,V)=imgD(Ud,Vd)
[0094] Among them, imgR is the real image, imgD is the distorted image, (U, V) is the pixel coordinate of the real image, and (Ud, Vd) is the pixel coordinate of the distorted image.
[0095] Since the transformation process from the imaging plane of the camera to the image pixel coordinates is a linear transformation, that is, a transformation process involving an intrinsic parameter matrix, the camera model used in this embodiment assumes that the distortion of the camera occurs on a normalized plane.
[0096] The following is the process of projecting a point A in the world coordinate system to the image pixel coordinate system through a pinhole camera model containing a multi-layer perceptron. First, point A is transformed into the camera coordinate system by rotation and translation, and the formula is as follows:
[0097]
[0098] in, is the camera coordinate point, Represents a three-dimensional coordinate point, is the camera external parameter matrix, R is the rotation matrix, T is the translation vector, 0 T is the zero vector.
[0099] The transformation from the camera coordinate system to the image coordinate system (i.e., from the camera coordinate point to the image coordinate point) is expressed as follows:
[0100]
[0101] in, is the camera internal parameter matrix, f is the camera focal length, is the image coordinate point.
[0102] S230: Perform nonlinear transformation processing on the image coordinate points through a distortion model to convert the image coordinate points into pixel coordinate points in an image pixel coordinate system.
[0103] Optionally, the image coordinate points are subjected to nonlinear transformation processing through the distortion model to convert the image coordinate points into pixel coordinate points in an image pixel coordinate system, including: scaling and translating the image coordinate points through the distortion model to convert the image coordinate points into initial pixel coordinate points in the image pixel coordinate system, and offset correcting the initial pixel coordinate points to obtain pixel coordinate points in the image pixel coordinate system corresponding to the point cloud coordinate points.
[0104] In this embodiment, it is assumed that the camera distortion process occurs on the camera normalization plane, and the radial distortion and tangential distortion are modeled by a layer including two linear layers plus a sigmoid activation function. The neural network structure and parameters for modeling the distortion part are shown in Table 2:
[0105]
[0106]
[0107] Table 2
[0108] Optionally, the distortion model includes two linear layers and an activation function layer; the distortion model performs scaling and translation processing on the image coordinate points through the following formula to convert the image coordinate points into initial pixel coordinate points in the image pixel coordinate system:
[0109]
[0110] Among them, w1 is the transformation parameter of the first linear layer, b1 is the bias of the first linear layer, w2 is the transformation parameter of the second linear layer, b1 is the bias of the second linear layer, is the initial pixel coordinate to which the distortion parameters are added.
[0111] In an embodiment of the present invention, two linear layers plus a sigmoid activation function are used to describe the nonlinear distortion of the camera, and the radial distortion and tangential distortion factors of the camera are uniformly considered. With the goal of minimizing the camera projection error, the external parameters, internal parameters and distortion parameters in the camera model are uniformly optimized to achieve the adjustment of the parameters of each part in the camera projection process.
[0112] Furthermore, by performing offset correction on the initial pixel coordinate point, a pixel coordinate point in the image pixel coordinate system corresponding to the point cloud coordinate point is obtained, which is expressed by the following formula:
[0113]
[0114] Among them, _u and _v are the pixel coordinates after distortion, f x and fy is the focal length of the camera, c x and c y It is the vertical and horizontal offset of the image origin relative to the optical center imaging point.
[0115] In the embodiment of the present invention, the initial pixel coordinates are corrected based on the vertical and horizontal offsets of the origin relative to the optical center imaging point to obtain the pixel coordinates restored from the actual camera shooting, thereby improving the accuracy of the pixel coordinate points.
[0116] Step 2.2.2: Camera parameter optimization method
[0117] In step 2.2.1, the camera external parameters (6), distortion coefficients (17), and internal parameters (4) are modeled at the same time. The model contains a total of 27 parameters. Assuming that the real three-dimensional coordinates (world coordinates) of the space point A are (X, Y, Z), the corresponding real observed image pixel coordinates are (u, v), and the projection coordinates of the camera model are (_u, _v), the difference between the real observed value and the projection value can be calculated, and the square of the error in the u and v directions is calculated and then summed, which can be expressed as follows:
[0118]
[0119] Among them, loss is the model loss value, (u i , v i ) is the real observed image pixel coordinate, (_u i ,_v i ) is the pixel coordinate of the projection of the camera model.
[0120] The optimization variables are 27 parameters of the camera model. After constructing the loss function, the nonlinear least squares optimization tool ceres is used to implement the algorithm.
[0121] The preprocessed data is used as the input data of the optimization program developed based on the ceres library. In order to make the optimization algorithm converge quickly to the global optimal solution and prevent overfitting, only 27 parameters of the camera are modeled in this scheme, that is, 6 external parameters and 17 distortion parameters are randomly initialized using the standard normal distribution, and the coordinates of the two principal points of the camera and the focal lengths in the x and y directions are initialized by the length and width of the photos taken by the camera.
[0122] Step 3. Distance measurement
[0123] The pixel coordinate points are projected into the visible light image using a nonlinear camera model to obtain a coordinate point correspondence relationship between the point cloud coordinate points and their corresponding pixel coordinate points, that is, the coordinate point correspondence relationship can be used to achieve the conversion from pixel coordinate points to three-dimensional world coordinate points.
[0124] Using image target detection technology such as target detection algorithm, potential external objects such as excavators and engineering vehicles in the scene image are detected. Figure 2c A sample schematic diagram of an object to be measured is provided as an optional example of a distance measurement method based on a nonlinear camera model. Figure 2c As shown in the figure, a total of 3 coordinate points of the target objects to be measured are detected. Figure 2d A sample schematic diagram of a wire and ground point cloud separation is provided for an optional example of a distance measurement method based on a nonlinear camera model, such as Figure 2d As shown, the upper part of the figure is the point cloud data of the wire, and the lower part of the figure is the ground point cloud data. Figure 2e A sample schematic diagram of point distance measurement is provided, which is an optional example of a distance measurement method based on a nonlinear camera model. Figure 2e As shown, the distance between the object to be measured and the wire can be determined according to the point cloud coordinates of the object to be measured and the wire.
[0125] The technical solution of the embodiment of the present invention uses a neural network of a multi-layer perceptron to model the nonlinear part of the projection process, and uses an optimized algorithm to solve the nonlinear camera model parameters, avoiding the process of calibrating the camera internal parameters using a calibration plate. The gradient descent method is used to solve the transformation relationship between image coordinates and point cloud coordinates (nonlinear camera model parameters), which reduces the requirements for the number of feature point pairs and also reduces the complexity of the optimization algorithm, effectively improving the algorithm's computational efficiency. By combining monocular images with point cloud data, it is possible to achieve distance measurement between two points in a scene through a monocular image, thereby improving the distance measurement efficiency and accuracy.
[0126] Embodiment 3
[0127] Figure 3 This is a schematic diagram of the structure of a distance measurement device based on a nonlinear camera model provided in the third embodiment of the present invention. Figure 3 As shown, the device includes: a data acquisition module 310, a coordinate conversion module 320, and a distance measurement module 330.
[0128] Among them, the data acquisition module 310 is used to acquire the visible light image and three-dimensional point cloud data corresponding to the target scene, wherein the three-dimensional point cloud data includes multiple point cloud coordinate points in the world coordinate system; the coordinate conversion module 320 is used to perform linear transformation processing and nonlinear transformation processing on the point cloud coordinate points through a pre-trained nonlinear camera model to obtain pixel coordinate points in the image pixel coordinate system corresponding to the point cloud coordinate points; the ranging module 330 is used to project the pixel coordinate points into the visible light image to obtain the coordinate point correspondence relationship between the point cloud coordinate points and the pixel coordinate points corresponding to them, and determine the distance data between at least two objects to be measured based on the coordinate point correspondence relationship and the pixel coordinate points corresponding to the at least two objects to be measured.
[0129] The technical solution of the embodiment of the present invention is to obtain a visible light image and three-dimensional point cloud data corresponding to a target scene through a data acquisition module, wherein the three-dimensional point cloud data includes a plurality of point cloud coordinate points in a world coordinate system; a corresponding relationship between the visible light image and the three-dimensional point cloud data of the same scene can be established; then, through a coordinate conversion module, a pre-trained nonlinear camera model is used to perform linear transformation processing and nonlinear transformation processing on the point cloud coordinate points to obtain pixel coordinate points in an image pixel coordinate system corresponding to the point cloud coordinate points; so as to accurately determine the pixel coordinate points of the point cloud coordinate points in the image pixel coordinate system; finally, through a ranging module, the pixel coordinate points are projected into the visible light image to obtain a coordinate point correspondence relationship between the point cloud coordinate points and the pixel coordinate points corresponding to them, and the distance data between at least two objects to be measured is determined based on the coordinate point correspondence relationship and the pixel coordinate points corresponding to at least two objects to be measured, so as to solve the problem of low ranging efficiency and low accuracy of the existing ranging method, realize ranging of at least two objects to be measured in a scene through a monocular image, and achieve the beneficial effect of improving ranging efficiency and accuracy.
[0130] Optionally, the nonlinear camera model includes a camera model and a distortion model; accordingly, the coordinate conversion module includes:
[0131] A linear transformation unit, used for performing linear transformation processing on the point cloud coordinate points through the camera model to convert the point cloud coordinate points into image coordinate points;
[0132] The nonlinear transformation unit is used to perform nonlinear transformation processing on the image coordinate points through the distortion model, and convert the image coordinate points into pixel coordinate points in the image pixel coordinate system.
[0133] Optionally, the linear transformation unit is used to:
[0134] The point cloud coordinate points are rotated and translated by the camera model, the point cloud coordinate points are converted into camera coordinate points in a camera coordinate system, and the camera coordinate points are projected into an image coordinate system to obtain image coordinate points corresponding to the point cloud coordinate points.
[0135] Optionally, the nonlinear transformation unit is used to:
[0136] The image coordinate points are scaled and translated by the distortion model to be converted into initial pixel coordinate points in an image pixel coordinate system, and offset correction is performed on the initial pixel coordinate points to obtain pixel coordinate points in the image pixel coordinate system corresponding to the point cloud coordinate points.
[0137] Optionally, the distortion model includes two linear layers and an activation function layer; accordingly, the nonlinear transformation unit is specifically used for:
[0138] The distortion model performs scaling and translation processing on the image coordinate points through the following formula to convert the image coordinate points into initial pixel coordinate points in the image pixel coordinate system:
[0139]
[0140] Among them, w1 is the transformation parameter of the first linear layer, b1 is the bias of the first linear layer, w2 is the transformation parameter of the second linear layer, b1 is the bias of the second linear layer, is the initial pixel coordinate to which the distortion parameters are added.
[0141] Optionally, the device further includes a sample point acquisition module, a model construction and sample point registration module, a model optimization module and a nonlinear camera model acquisition module.
[0142] The sample point acquisition module is used to obtain camera-related parameters and a preset number of sample coordinate points before performing linear transformation processing and nonlinear transformation processing on the point cloud coordinate points through the pre-trained nonlinear camera model, wherein the camera-related parameters include camera intrinsic parameters, distortion parameters and camera extrinsic parameters, and the sample coordinate points include sample pixel coordinate points and sample point cloud coordinate points;
[0143] The model building and sample point registration module is used to define an initial nonlinear camera model based on the camera intrinsic parameters, the distortion parameters and the camera extrinsic parameters, and to register the sample pixel coordinate points with the sample point cloud coordinate points to obtain multiple groups of sample point pairs;
[0144] The model optimization module is used to iteratively optimize the initial nonlinear camera model based on a nonlinear optimization algorithm and the sample point pairs;
[0145] The nonlinear camera model acquisition module is used to complete model optimization when the model loss function converges to obtain a trained nonlinear camera model.
[0146] Optionally, the ranging module includes:
[0147] A point cloud coordinate point determination unit, configured to determine a point cloud coordinate point corresponding to the pixel coordinate point of the object to be measured based on the pixel coordinate point of the object to be measured and the coordinate point correspondence relationship;
[0148] The distance measuring unit is used to calculate the distance data between at least two objects to be measured based on the point cloud coordinate points of at least two objects to be measured.
[0149] Optionally, the data acquisition module includes:
[0150] An image acquisition unit, used for acquiring a visible light image corresponding to a target scene based on a visible light camera;
[0151] The point cloud acquisition unit is used to acquire three-dimensional point cloud data corresponding to the target scene based on the laser radar.
[0152] The distance measuring device based on the nonlinear camera model provided in the embodiment of the present invention can execute the distance measuring method based on the nonlinear camera model provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0153] Embodiment 4
[0154] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.
[0155] like Figure 4As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.
[0156] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0157] The processor 11 may be a variety of general and / or dedicated processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the ranging method based on a nonlinear camera model.
[0158] In some embodiments, the method of nonlinear camera model-based ranging can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method of nonlinear camera model-based ranging described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the method of nonlinear camera model-based ranging in any other appropriate manner (for example, by means of firmware).
[0159] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0160] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0161] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0162] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).
[0163] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.
[0164] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.
[0165] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.
[0166] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A distance measurement method based on a nonlinear camera model, characterized in that: include: Acquire a visible light image and three-dimensional point cloud data corresponding to a target scene, wherein the three-dimensional point cloud data includes a plurality of point cloud coordinate points in a world coordinate system; Performing linear transformation and nonlinear transformation on the point cloud coordinate points through a pre-trained nonlinear camera model to obtain pixel coordinate points in an image pixel coordinate system corresponding to the point cloud coordinate points; The pixel coordinate points are projected into the visible light image to obtain a coordinate point correspondence relationship between the point cloud coordinate points and the pixel coordinate points corresponding to them, and the distance data between at least two objects to be measured are determined based on the coordinate point correspondence relationship and the pixel coordinate points corresponding to at least two objects to be measured.
2. The method according to claim 1, characterized in that The nonlinear camera model includes a camera model and a distortion model; the linear transformation processing and nonlinear transformation processing are performed on the point cloud coordinate points by the nonlinear camera model to obtain pixel coordinate points in an image pixel coordinate system corresponding to the point cloud coordinate points, including: Performing linear transformation processing on the point cloud coordinate points through the camera model to convert the point cloud coordinate points into image coordinate points; The image coordinate points are subjected to nonlinear transformation processing by using the distortion model, so as to convert the image coordinate points into pixel coordinate points in an image pixel coordinate system.
3. The method according to claim 2, characterized in that The linear transformation of the point cloud coordinate points by the camera model to convert the point cloud coordinate points into image coordinate points includes: The point cloud coordinate points are rotated and translated by the camera model, the point cloud coordinate points are converted into camera coordinate points in a camera coordinate system, and the camera coordinate points are projected into an image coordinate system to obtain image coordinate points corresponding to the point cloud coordinate points.
4. The method according to claim 2, characterized in that: The performing nonlinear transformation processing on the image coordinate points by using the distortion model to convert the image coordinate points into pixel coordinate points in an image pixel coordinate system includes: The image coordinate points are scaled and translated by the distortion model to be converted into initial pixel coordinate points in an image pixel coordinate system, and offset correction is performed on the initial pixel coordinate points to obtain pixel coordinate points in the image pixel coordinate system corresponding to the point cloud coordinate points.
5. The method according to claim 4, characterized in that The distortion model includes two linear layers and an activation function layer; the distortion model performs scaling and translation processing on the image coordinate points through the following formula to convert the image coordinate points into initial pixel coordinate points in the image pixel coordinate system: Among them, w1 is the transformation parameter of the first linear layer, b1 is the bias of the first linear layer, w2 is the transformation parameter of the second linear layer, b1 is the bias of the second linear layer, is the initial pixel coordinate to which the distortion parameters are added.
6. The method according to claim 1, characterized in that Before the linear transformation and nonlinear transformation of the point cloud coordinate points are performed by the pre-trained nonlinear camera model, the method further includes: Acquire camera-related parameters and a preset number of sample coordinate points, wherein the camera-related parameters include camera intrinsic parameters, distortion parameters, and camera extrinsic parameters, and the sample coordinate points include sample pixel coordinate points and sample point cloud coordinate points; An initial nonlinear camera model is defined based on the camera intrinsic parameters, the distortion parameters and the camera extrinsic parameters, and the sample pixel coordinate points are aligned with the sample point cloud coordinate points to obtain multiple groups of sample point pairs; Iteratively optimizing the initial nonlinear camera model based on a nonlinear optimization algorithm and the sample point pairs; When the model loss function converges, the model optimization is completed to obtain a trained nonlinear camera model.
7. The method according to claim 1, characterized in that The determining the distance data between at least two objects to be measured based on the coordinate point correspondence relationship and the pixel coordinate points corresponding to at least two objects to be measured includes: Determine a point cloud coordinate point corresponding to the pixel coordinate point of the object to be measured based on the pixel coordinate point of the object to be measured and the coordinate point correspondence relationship; The distance data between at least two objects to be measured is calculated based on the point cloud coordinate points of at least two objects to be measured.
8. The method according to claim 1, characterized in that The step of acquiring the visible light image and three-dimensional point cloud data corresponding to the target scene includes: Acquire a visible light image corresponding to the target scene based on a visible light camera; Acquire three-dimensional point cloud data corresponding to the target scene based on lidar.
9. A distance measuring device based on a nonlinear camera model, characterized in that: include: A data acquisition module, used to acquire a visible light image and three-dimensional point cloud data corresponding to a target scene, wherein the three-dimensional point cloud data includes a plurality of point cloud coordinate points in a world coordinate system; A coordinate conversion module, used for performing linear transformation processing and nonlinear transformation processing on the point cloud coordinate points through a pre-trained nonlinear camera model to obtain pixel coordinate points in an image pixel coordinate system corresponding to the point cloud coordinate points; The ranging module is used to project the pixel coordinate points into the visible light image to obtain a coordinate point correspondence relationship between the point cloud coordinate points and the pixel coordinate points corresponding to them, and determine the distance data between at least two objects to be measured based on the coordinate point correspondence relationship and the pixel coordinate points corresponding to at least two objects to be measured.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the ranging method based on a nonlinear camera model according to any one of claims 1 to 8 when executed.
