Positioning method and device based on binocular vision, equipment and medium

Abstract

The invention relates to the technical field of visual localization, and provides a binocular vision-based localization method and device, equipment and a medium, and the method comprises the steps: obtaining a calibration image of a calibration plate and an original image of a target object through a binocular camera, carrying out the angular point extraction of the calibration image, obtaining a calibration angular point, and carrying out the angular point extraction; calculating internal and external parameters of the binocular camera according to the calibration angular points; performing distortion correction processing on the original image according to the internal and external parameters to obtain a corrected image, and obtaining an overlapped effective region image of the original image and the corrected image based on the corrected image; based on the overlapped effective area image, carrying out defogging processing on the image pixels through a boundary limitation method to obtain a restored image; and performing stereo matching on the restored image through a region growing algorithm to obtain the positioning information of the target object, thereby not only ensuring the accuracy of the image, but also performing stereo matching through the region growing algorithm to enable the object positioning to be more accurate.

Description

Technical Field

[0001] The present invention relates to the field of visual positioning technology, and in particular to a positioning method, device, equipment and medium based on binocular vision. Background Art

[0002] Visual positioning technology does not require an external signal source and has little dependence on equipment. It can locate the target object by obtaining information about the surrounding environment through visual sensors. The camera is low-cost and suitable for large-scale deployment. With the development of computer vision, visual positioning technology can effectively perceive the environment while positioning, which is difficult to achieve with other positioning methods.

[0003] However, when the positioning task environment is complex and full of uncertainty, the positioning technology based on binocular vision is very difficult. Visual positioning needs to determine the object to be located through the target detection algorithm, so it has higher complexity than other positioning methods. The current visual positioning technology still has problems such as image detection distortion, inaccurate object positioning, and low positioning efficiency. Summary of the invention

[0004] The present invention aims at solving the technical problems in the prior art and provides a binocular vision-based positioning method, system, device and medium, including:

[0005] Acquire a calibration image of the calibration plate and an original image of the target object through a binocular camera, extract corner points of the calibration image, obtain calibration corner points, and calculate the internal and external parameters of the binocular camera according to the calibration corner points;

[0006] According to the internal and external parameters, the original image is subjected to distortion correction processing to obtain a corrected image, and based on the corrected image, an image of an overlapping effective area of ​​the original image and the corrected image is obtained;

[0007] Based on the overlapping effective area image, defogging the image pixels by a boundary restriction method to obtain a restored image;

[0008] The restored images are stereo matched using a region growing algorithm to obtain the location information of the target object.

[0009] Preferably, the step of acquiring a calibration image of a calibration plate and an original image of a target object through a binocular camera, extracting corner points of the calibration image to acquire calibration corner points, and calculating the internal and external parameters of the binocular camera according to the calibration corner points includes:

[0010] Acquire a calibration image of the calibration plate through a preset binocular camera, and extract corner points of the calibration image to obtain calibration corner points;

[0011] Acquire the initial coordinates of the calibrated corner point based on the world coordinate system, perform coordinate transformation on the initial coordinates to obtain the transformed coordinates corresponding to the camera imaging plane, and determine the transformation mapping relationship according to the initial coordinates and the transformed coordinates;

[0012] According to the conversion mapping relationship, a homography matrix is ​​obtained, a symmetric matrix is ​​calculated according to the homography matrix, and a constraint equation is obtained according to the symmetric matrix and a unit orthogonal matrix;

[0013] According to the constraint equation and the six-dimensional vector in the symmetric matrix, the internal and external parameters of the binocular camera are calculated.

[0014] Preferably, performing distortion correction processing on the original image according to the internal and external parameters to obtain a corrected image, and obtaining an overlapping effective area image of the original image and the corrected image based on the corrected image, comprises:

[0015] According to the internal and external parameters, a unit transformation relationship is obtained, and the original coordinates of the original image of the target object are obtained through a binocular camera;

[0016] According to the unit transformation relationship and the original coordinates, obtaining the corrected coordinates of the original image after distortion correction through the distortion model;

[0017] Wherein, the distortion model includes: a radial distortion model and a tangential distortion model;

[0018] Substituting the corrected coordinates into the established likelihood function, and performing logarithmic transformation on the likelihood function to obtain a logarithmic likelihood function;

[0019] Solving the partial derivative of the log-likelihood function to obtain a maximum likelihood estimate of the distortion coefficient, and eliminating the distortion of the original image according to the maximum likelihood estimate to obtain a corrected image;

[0020] The corrected image is mapped to a corresponding position in the original image space to obtain an overlapping effective area image of the original image and the corrected image.

[0021] Preferably, the maximum likelihood estimate is calculated as follows:

[0022]

[0023] in,

[0024] in,

[0025] Where G(x,y)=w<M(x,y)+Y(x,y)>

[0026] In the formula, F represents the maximum likelihood estimate; ε represents the distortion coefficient; Represents partial derivative; M(x,y) represents radial distortion model; Y(x,y) represents tangential distortion model; k1 represents the first unit transformation parameter; k2 represents the second unit transformation parameter; R represents the total number of all original coordinates in the original image; v(x,y) represents the original coordinate point in the original image; ρ represents radial distortion error; σ represents radial distortion parameter; μ represents tangential distortion parameter; τ represents tangential distortion error; b represents radial distortion factor; h represents probability distribution of radial distortion error; e represents probability distribution of tangential distortion error; q represents tangential distortion factor; G(x,y) represents correction coordinate; w represents correction parameter.

[0027] Preferably, the defogging process is performed on the image pixels based on the overlapped effective area image by limiting the boundaries to obtain the restored image, including:

[0028] Based on the overlapping effective area image, the grayscale image of the overlapping effective area image is divided into four blocks, and the mean square error and mean value of each sub-block are calculated;

[0029] Select the sub-block with the smallest mean square error in turn to continue the four-segmentation. When the segmented area is smaller than the set threshold, search for pixels with values ​​within the preset value range in the segmented image block, and the RGB values ​​corresponding to the pixels within the preset value can be considered as the atmospheric ambient light value, otherwise continue the segmentation;

[0030] Calculate and obtain a first transmittance of an image pixel according to the atmospheric ambient light value and a pixel limit in an image radiation cube;

[0031] Based on the first transmittance, adding a guided filter in the image block to obtain an optimized transmittance of the image pixel;

[0032] According to the atmospheric ambient light value and the optimized transmittance, the overlapping effective area image is defogged to obtain a restored image.

[0033] The restored image expression is as follows:

[0034]

[0035] in,

[0036] in,

[0037] Where p(x) represents the restored image; Z(x) represents the overlapped effective area image; x represents the pixel in the grayscale image; I(x) represents the atmospheric ambient light value corresponding to the pixel x; B represents the optimized transmittance; t(x) represents the scene transmittance; ζ represents the ratio of the ambient transmittance; θ represents the conversion factor; represents the guided filter parameter; t k Indicates pixel limit; mint k represents the minimum boundary limit; s represents the upper limit of grayscale image segmentation; α represents the mean square error of the grayscale image sub-block; β0 represents the mean of the grayscale image sub-block; η represents the atmospheric scattering coefficient.

[0038] Preferably, performing stereo matching on the restored image by using a region growing algorithm to obtain the positioning information of the target object includes:

[0039] Step R1: grayscale the restored image to obtain a grayscale image, and perform pixel point detection on the grayscale image using an ORB algorithm to determine feature points of the grayscale image;

[0040] Step R2: input the feature point into a preset matching model to obtain a matching point set, select a group of pixels from the matching point set as a starting point, and use the starting point as the center point of the first area;

[0041] Step R3: Calculate the similarity between the surrounding pixels of the center point and the center point pixel by using the ZNCC method. When the obtained similarity is greater than a preset threshold, add the current surrounding pixels to the current area, and use the surrounding pixels of the center point as the new center point. Repeat step R3 to iterate sequentially.

[0042] When the obtained similarity is less than or equal to the preset threshold, the iteration is stopped to obtain pixels that meet the conditions;

[0043] Step R4: Mark the region composed of all pixels that meet the conditions, treat the marked region as an object, and return to R3 to continue to obtain a new center point. When all pixels in the entire image are marked as an object, obtain the regional growth disparity map;

[0044] Step R5: calculating the integral image of the region growth disparity map for subsequent mean filtering;

[0045] Step R6: obtaining the largest hole in the region growing disparity map, and calculating the area ratio of the hole to the region growing disparity map, selecting an initial filtering window of a preset size according to the area ratio, and filling the hole by mean filtering according to the integral image of the initial window and the region growing disparity map;

[0046] Step R7: reducing the initial window at a fixed reduction ratio to obtain a second filter window, and performing secondary hole filling by mean filtering according to the second filter window and the integral image;

[0047] Step R8: Repeat step R7, and when the filter window size reaches the preset specification, stop filtering to obtain the positioning information of the target object.

[0048] Preferably, the steps of constructing the matching model are as follows:

[0049] Step S1: sampling feature points of the grayscale image by random sampling method to obtain sample points, and setting initial thresholds and initial model parameters according to the sample points;

[0050] Wherein, the initial thresholds include: an inlier error threshold, an inlier number threshold and a maximum number of iterations;

[0051] Step S2: when the number of iterations is less than or equal to the maximum number of iterations, the error values ​​of the initial model are calculated according to the non-sample points in the feature points, and the error values ​​are sorted in ascending order to obtain an error point set;

[0052] Step S3: Based on the error point set, select according to a preset error range to obtain a target error point set, calculate the inlier point set of the error point set and the target error point set under the current model, and obtain the number of inliers and the error range of the current model;

[0053] Step S4: Calculate the sampling probability of the feature points of the grayscale image according to the weight of the number of inliers and the error range, and randomly select a feature point from the sample points based on the sampling probability, and add it to the inlier set to update the number of inliers;

[0054] Step S5: Calculate the current model error and current model parameters based on the number of inliers, the error point set and the target error point set;

[0055] Step S6: When the number of inliers is less than or equal to the inlier number threshold, return to step 1 to continue iterating; when the number of inliers is greater than the inlier number threshold, obtain the current model parameters and the current inlier set.

[0056] Based on the same inventive concept, the present invention also provides a positioning device based on binocular vision, comprising:

[0057] A parameter calculation module, which obtains a calibration image of the calibration plate and an original image of the target object through a binocular camera, extracts corner points of the calibration image, obtains calibration corner points, and calculates the internal and external parameters of the binocular camera according to the calibration corner points;

[0058] An image correction module, performing distortion correction processing on the original image according to the internal and external parameters to obtain a corrected image, and obtaining an overlapping effective area image of the original image and the corrected image based on the corrected image;

[0059] An image restoration module, based on the overlapped effective area image, performs defogging processing on the image pixels by a boundary restriction method to obtain a restored image;

[0060] The object positioning module performs stereo matching on the restored image through a region growing algorithm to obtain positioning information of the target object.

[0061] Preferably, the parameter calculation module is specifically used for:

[0062] Acquire a calibration image of the calibration plate through a preset binocular camera, and extract corner points of the calibration image to obtain calibration corner points;

[0063] Acquire the initial coordinates of the calibrated corner point based on the world coordinate system, perform coordinate transformation on the initial coordinates to obtain the transformed coordinates corresponding to the camera imaging plane, and determine the transformation mapping relationship according to the initial coordinates and the transformed coordinates;

[0064] According to the conversion mapping relationship, a homography matrix is ​​obtained, a symmetric matrix is ​​calculated according to the homography matrix, and a constraint equation is obtained according to the symmetric matrix and a unit orthogonal matrix;

[0065] According to the constraint equation and the six-dimensional vector in the symmetric matrix, the internal and external parameters of the binocular camera are calculated.

[0066] Preferably, the image correction module is specifically used for:

[0067] According to the internal and external parameters, a unit transformation relationship is obtained, and the original coordinates of the original image of the target object are obtained through a binocular camera;

[0068] According to the unit transformation relationship and the original coordinates, obtaining the corrected coordinates of the original image after distortion correction through the distortion model;

[0069] Wherein, the distortion model includes: a radial distortion model and a tangential distortion model;

[0070] Substituting the corrected coordinates into the established likelihood function, and performing logarithmic transformation on the likelihood function to obtain a logarithmic likelihood function;

[0071] Solving the partial derivative of the log-likelihood function to obtain a maximum likelihood estimate of the distortion coefficient, and eliminating the distortion of the original image according to the maximum likelihood estimate to obtain a corrected image;

[0072] The corrected image is mapped to a corresponding position in the original image space to obtain an overlapping effective area image of the original image and the corrected image.

[0073] Preferably, the maximum likelihood estimation value calculation formula in the image correction module is as follows:

[0074]

[0075] in,

[0076] in,

[0077] Where G(x,y)=w<M(x,y)+Y(x,y)>

[0078] In the formula, F represents the maximum likelihood estimate; ε represents the distortion coefficient; Represents partial derivative; M(x,y) represents radial distortion model; Y(x,y) represents tangential distortion model; k1 represents the first unit transformation parameter; k2 represents the second unit transformation parameter; R represents the total number of all original coordinates in the original image; v(x,y) represents the original coordinate point in the original image; ρ represents radial distortion error; σ represents radial distortion parameter; μ represents tangential distortion parameter; τ represents tangential distortion error; b represents radial distortion factor; h represents probability distribution of radial distortion error; e represents probability distribution of tangential distortion error; q represents tangential distortion factor; G(x,y) represents correction coordinate; w represents correction parameter.

[0079] Preferably, the image restoration module is specifically used for:

[0080] Based on the overlapping effective area image, the grayscale image of the overlapping effective area image is divided into four blocks, and the mean square error and mean value of each sub-block are calculated;

[0081] Select the sub-block with the smallest mean square error in turn to continue the four-segmentation. When the segmented area is smaller than the set threshold, search for pixels with values ​​within the preset value range in the segmented image block, and the RGB values ​​corresponding to the pixels within the preset value can be considered as the atmospheric ambient light value, otherwise continue the segmentation;

[0082] Calculate and obtain a first transmittance of an image pixel according to the atmospheric ambient light value and a pixel limit in an image radiation cube;

[0083] Based on the first transmittance, adding a guided filter in the image block to obtain an optimized transmittance of the image pixel;

[0084] According to the atmospheric ambient light value and the optimized transmittance, the overlapping effective area image is defogged to obtain a restored image.

[0085] The restored image expression is as follows:

[0086]

[0087] in,

[0088] in,

[0089] Where p(x) represents the restored image; Z(x) represents the overlapped effective area image; x represents the pixel in the grayscale image; I(x) represents the atmospheric ambient light value corresponding to the pixel x; B represents the optimized transmittance; t(x) represents the scene transmittance; ζ represents the ratio of the ambient transmittance; θ represents the conversion factor; represents the guided filter parameter; t k Indicates pixel limit; mint k represents the minimum boundary limit; s represents the upper limit of grayscale image segmentation; α represents the mean square error of the grayscale image sub-block; β0 represents the mean of the grayscale image sub-block; η represents the atmospheric scattering coefficient.

[0090] Preferably, the object positioning module is specifically used for:

[0091] Step R1: grayscale the restored image to obtain a grayscale image, and perform pixel point detection on the grayscale image using an ORB algorithm to determine feature points of the grayscale image;

[0092] Step R2: input the feature point into a preset matching model to obtain a matching point set, select a group of pixels from the matching point set as a starting point, and use the starting point as the center point of the first area;

[0093] Step R3: Calculate the similarity between the surrounding pixels of the center point and the center point pixel by using the ZNCC method. When the obtained similarity is greater than a preset threshold, add the current surrounding pixels to the current area, and use the surrounding pixels of the center point as the new center point. Repeat step R3 to iterate sequentially.

[0094] When the obtained similarity is less than or equal to the preset threshold, the iteration is stopped to obtain pixels that meet the conditions;

[0095] Step R4: Mark the region composed of all pixels that meet the conditions, treat the marked region as an object, and return to R3 to continue to obtain a new center point. When all pixels in the entire image are marked as an object, obtain the regional growth disparity map;

[0096] Step R5: calculating the integral image of the region growth disparity map for subsequent mean filtering;

[0097] Step R6: obtaining the largest hole in the region growing disparity map, and calculating the area ratio of the hole to the region growing disparity map, selecting an initial filtering window of a preset size according to the area ratio, and filling the hole by mean filtering according to the integral image of the initial window and the region growing disparity map;

[0098] Step R7: reducing the initial window at a fixed reduction ratio to obtain a second filter window, and performing secondary hole filling by mean filtering according to the second filter window and the integral image;

[0099] Step R8: Repeat step R7, and when the filter window size reaches the preset specification, stop filtering to obtain the positioning information of the target object.

[0100] Preferably, the steps of constructing the matching model in the object positioning module are as follows:

[0101] Step S1: sampling feature points of the grayscale image by random sampling method to obtain sample points, and setting initial thresholds and initial model parameters according to the sample points;

[0102] Wherein, the initial thresholds include: an inlier error threshold, an inlier number threshold and a maximum number of iterations;

[0103] Step S2: when the number of iterations is less than or equal to the maximum number of iterations, the error values ​​of the initial model are calculated according to the non-sample points in the feature points, and the error values ​​are sorted in ascending order to obtain an error point set;

[0104] Step S3: Based on the error point set, select according to a preset error range to obtain a target error point set, calculate the inlier point set of the error point set and the target error point set under the current model, and obtain the number of inliers and the error range of the current model;

[0105] Step S4: Calculate the sampling probability of the feature points of the grayscale image according to the weight of the number of inliers and the error range, and randomly select a feature point from the sample points based on the sampling probability, and add it to the inlier set to update the number of inliers;

[0106] Step S5: Calculate the current model error and current model parameters based on the number of inliers, the error point set and the target error point set;

[0107] Step S6: When the number of inliers is less than or equal to the inlier number threshold, return to step 1 to continue iterating; when the number of inliers is greater than the inlier number threshold, obtain the current model parameters and the current inlier set.

[0108] Based on the same inventive concept, the present invention also provides an electronic device, including:

[0109] Memory for storing computer software programs;

[0110] The processor is used to read and execute the computer software program, thereby realizing the positioning method based on binocular vision.

[0111] Based on the same inventive concept, the present invention also provides a non-transitory computer-readable storage medium, in which a computer software program is stored. When the computer software program is executed by a processor, the positioning method based on binocular vision is implemented.

[0112] The beneficial effects of the present invention are as follows: 1. The present invention provides a positioning method, device, equipment and medium based on binocular vision, including: obtaining a calibration image of a calibration plate and an original image of a target object through a binocular camera, extracting corner points of the calibration image to obtain calibration corner points, and calculating the internal and external parameters of the binocular camera according to the calibration corner points; performing distortion correction processing on the original image according to the internal and external parameters to obtain a corrected image, and obtaining an overlapping effective area image of the original image and the corrected image based on the corrected image; based on the overlapping effective area image, performing defogging processing on the image pixels through a boundary restriction method to obtain a restored image; performing stereo matching on the restored image through a region growing algorithm to obtain the positioning information of the target object. The present invention corrects image distortion according to the camera internal and external parameters and the distortion correction parameters, which can ensure the accuracy of the image, and performs stereo matching through image defogging processing and region growing algorithm, which can make the object positioning more accurate;

[0113] 2. In the present invention, the image is pre-processed into grayscale to reduce the amount of calculation, and then the matching points are optimized by the method to improve the matching accuracy. Then, the matching point pairs are used as initial seed points to perform region growing to generate a dense disparity map, and the holes in the disparity map are filled to improve the integrity of the disparity map, thereby quickly performing object recognition and positioning;

[0114] 3. The present invention can improve the accuracy of fitting results when processing data containing noise and outliers while ensuring computational efficiency through matching models, and improve the accuracy and robustness of the model by introducing "priority" to screen candidate models. BRIEF DESCRIPTION OF THE DRAWINGS

[0115] Figure 1 The present invention provides a schematic flow chart of a positioning method based on binocular vision;

[0116] Figure 2 This is a schematic diagram of the connection of modules of a positioning device based on binocular vision in the present invention.

[0117] Figure 3 This is a schematic diagram of the structure of an electronic device of the present invention;

[0118] Figure 4 A schematic diagram of the structure of a non-transitory computer-readable storage medium according to the present invention;

[0119] In the accompanying drawings, the components represented by the reference numerals are listed as follows:

[0120] 500 - electronic device, 510 - memory, 511 - computer program, 520 - processor, 600 - computer readable storage medium, 611 - computer program. DETAILED DESCRIPTION

[0121] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0122] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, "plurality" means two or more, unless otherwise clearly and specifically defined.

[0123] In the description of the present application, the term "for example" is used to mean "used as an example, illustration or description". Any embodiment described as "for example" in the present application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any technician in the field to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid unnecessary details to obscure the description of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.

[0124] Embodiment 1:

[0125] A flowchart of a binocular vision-based positioning method provided by the present invention is shown in FIG. Figure 1 As shown, including:

[0126] Step 1: Obtain a calibration image of the calibration plate and an original image of the target object through a binocular camera, extract corner points of the calibration image, obtain calibration corner points, and calculate the internal and external parameters of the binocular camera according to the calibration corner points;

[0127] Step 2: performing distortion correction processing on the original image according to the internal and external parameters to obtain a corrected image, and obtaining an overlapping effective area image of the original image and the corrected image based on the corrected image;

[0128] Step 3: Based on the overlapping effective area image, the image pixels are defogged by a boundary restriction method to obtain a restored image;

[0129] Step 4: Perform stereo matching on the restored image through a region growing algorithm to obtain the positioning information of the target object.

[0130] Specifically, the step 1 includes:

[0131] Acquire a calibration image of the calibration plate through a preset binocular camera, and extract corner points of the calibration image to obtain calibration corner points;

[0132] The camera calibration experiment uses a 12x9 checkerboard calibration plate with a single square length of 10 mm. The position of the calibration plate is changed, and calibration images at different angles are captured by the binocular camera module.

[0133] Then, the corner points are detected to remove the low-quality calibration images to ensure that the corner points in the calibration plate can be detected. Two camera modules are used to construct a binocular stereo camera, i.e., the binocular camera in the present invention, to perform image acquisition and calibration, and further perform corner point extraction;

[0134] In order to improve the accuracy of calibration, the present invention collects 16 calibration plate images at different angles, which can more accurately reflect the internal and external parameters of the binocular camera;

[0135] Acquire the initial coordinates of the calibrated corner point based on the world coordinate system, perform coordinate transformation on the initial coordinates to obtain the transformed coordinates corresponding to the camera imaging plane, and determine the transformation mapping relationship according to the initial coordinates and the transformed coordinates;

[0136] Wherein, the conversion mapping relationship is: dv = W[RT]Q

[0137] Where d represents the scale factor; v represents the initial coordinates; W represents the intrinsic parameter matrix; R represents the rotation matrix; T represents the translation matrix; Q represents the conversion ratio;

[0138] According to the conversion mapping relationship, a homography matrix is ​​obtained, a symmetric matrix is ​​calculated according to the homography matrix, and a constraint equation is obtained according to the symmetric matrix and a unit orthogonal matrix;

[0139] The homography matrix is: H = [h1h2h3] = λW[r1r2T];

[0140] In the formula, H represents the homography matrix; h1 represents the first vector in the homography matrix; h2 represents the second vector in the homography matrix; h3 represents the third vector in the homography matrix; W represents the intrinsic parameter matrix; r1 represents the first unit orthogonal matrix; r2 represents the second unit orthogonal matrix; λ represents the eigenvalue of the rotation matrix; T represents the translation matrix.

[0141] Wherein, the unit orthogonal matrix includes: a first unit orthogonal matrix and a second unit orthogonal matrix;

[0142] Among them, the homography matrix is ​​a matrix used to describe the relationship between the image and the checkerboard calibration plate, and the upper left corner of the calibration plate is selected as the origin of the world coordinate system;

[0143] After determining the coordinates of the calibration corner points in three-dimensional space, the coordinates of the corresponding points on the camera imaging plane are then determined through coordinate transformation;

[0144] The homography matrix H is a 3*3 homogeneous matrix, in which one equation is in homogeneous form, so the matrix has 8 unknown variables. The symmetric matrix is ​​calculated based on the homography matrix, and two equations need to be listed when obtaining the constraint equation based on the symmetric matrix and the unit orthogonal matrix. Therefore, only 4 pairs of corresponding point coordinates are needed to solve the equation;

[0145] According to the constraint equation and the six-dimensional vector in the symmetric matrix, the internal and external parameters of the binocular camera are calculated.

[0146] The camera projects a three-dimensional object onto a two-dimensional imaging plane. The imaging process can be seen as the process of a ray from the object to the camera being projected onto the imaging plane at the intersection point at the pinhole.

[0147] However, due to various errors and limitations in camera imaging, such as limitations of the optical system, limitations of photosensitive elements, and environmental influences, the ideal perspective model cannot be fully realized;

[0148] Therefore, there is distortion between the actual imaging results and the ideal imaging. The types of distortion mainly include geometric distortion and photometric distortion. Geometric distortion can be divided into radial distortion and tangential distortion. The distortion can be corrected according to the camera's internal and external parameters and distortion correction parameters. Therefore, the calculation of the internal and external parameters of the binocular camera can ensure the accuracy of the image.

[0149] Specifically, the step 2 includes:

[0150] According to the internal and external parameters, a unit transformation relationship is obtained, and the original coordinates of the original image of the target object are obtained through a binocular camera;

[0151] According to the unit transformation relationship and the original coordinates, obtaining the corrected coordinates of the original image after distortion correction through the distortion model;

[0152] Wherein, the distortion model includes: a radial distortion model and a tangential distortion model;

[0153] Among them, when building the distortion model, it is necessary to assume that the distortion error obeys the Gaussian distribution, and the pixel coordinates of the original image before distortion correction are v(x,y);

[0154] Then the radial distortion model can be obtained:

[0155] Tangential distortion model:

[0156] Wherein, M(x,y) represents the radial distortion model; Y(x,y) represents the tangential distortion model; R represents the total number of all original coordinates in the original image; v(x,y) represents the original coordinate point in the original image; ρ represents the radial distortion error; σ represents the radial distortion parameter; μ represents the tangential distortion parameter; τ represents the tangential distortion error; b represents the radial distortion factor; h represents the probability distribution of the radial distortion error; e represents the probability distribution of the tangential distortion error; q represents the tangential distortion factor.

[0157] The correction coordinates obtained according to the radial distortion model and the tangential distortion model are:

[0158] G(x,y)=w<M(x,y)+Y(x,y)>

[0159] Where G(x,y) represents the correction coordinates; w represents the correction parameters; M(x,y) represents the radial distortion model; Y(x,y) represents the tangential distortion model;

[0160] Substituting the corrected coordinates into the established likelihood function, and performing logarithmic transformation on the likelihood function to obtain a log-likelihood function;

[0161] Solving the partial derivative of the log-likelihood function to obtain a maximum likelihood estimate of the distortion coefficient, and eliminating the distortion of the original image according to the maximum likelihood estimate to obtain a corrected image;

[0162] The corrected image is mapped to a corresponding position in the original image space to obtain an overlapping effective area image of the original image and the corrected image.

[0163] The maximum likelihood estimate is calculated as follows:

[0164]

[0165] In the formula, F represents the maximum likelihood estimate; ε represents the distortion coefficient; represents partial derivative; k1 represents the first unit transformation parameter; k2 represents the second unit transformation parameter; G(x,y) represents the correction coordinate.

[0166] Specifically, the step 3 includes:

[0167] Based on the overlapping effective area image, the grayscale image of the overlapping effective area image is divided into four blocks, and the mean square error and mean value of each sub-block are calculated;

[0168] Select the sub-block with the smallest mean square error in turn to continue the four-segmentation. When the segmented area is smaller than the set threshold, search for pixels with values ​​within the preset value range in the segmented image block, and the RGB values ​​corresponding to the pixels within the preset value can be considered as the atmospheric ambient light value, otherwise continue the segmentation;

[0169] Among them, for images containing sky areas, the atmospheric light estimation value should fall in the sky area as much as possible; for images without sky areas, the atmospheric light value should be selected from the brighter pixel area;

[0170] Considering that the variance of the pixel values ​​in the atmospheric light estimation area is usually low, for images with sky areas, the present invention eliminates the interference caused by overly bright pixels through the above-mentioned quadtree segmentation traversal to more accurately estimate the atmospheric light value;

[0171] Calculate and obtain a first transmittance of an image pixel according to the atmospheric ambient light value and a pixel limit in an image radiation cube;

[0172] Based on the first transmittance, adding a guided filter in the image block to obtain an optimized transmittance of the image pixel;

[0173] The rough transmittance estimated by the boundary constraint has blurred edge information and cannot be directly used to restore the haze-free image, so the transmittance needs to be further refined;

[0174] The present invention refines the transmittance by adding guided filtering, which has the characteristics of smoothing image edges, restoring details and reducing noise, can better refine the transmittance and has a fast running speed;

[0175] Guided filtering divides the image into two parts: a base layer containing the essential part of the image and edge information, and a detail layer containing the texture details and noise information of the image. The base layer is considered to be a local linear transformation of the guided image. The base layer and the guided image have the following linear relationship;

[0176] P=Cψ x +ξ x

[0177] Where P represents the base layer; C represents the guide image; ψ i represents the first linear function centered at pixel x; ξi represents the second linear function centered at pixel x;

[0178] According to the atmospheric ambient light value and the optimized transmittance, the overlapping effective area image is defogged to obtain a restored image.

[0179] The restored image expression is as follows:

[0180]

[0181] in,

[0182] in,

[0183] Where p(x) represents the restored image; Z(x) represents the overlapped effective area image; x represents the pixel in the grayscale image; I(x) represents the atmospheric ambient light value corresponding to the pixel x; B represents the optimized transmittance; t(x) represents the scene transmittance; ζ represents the ratio of the ambient transmittance; θ represents the conversion factor; represents the guided filter parameter; t k Indicates pixel limit; mint k represents the minimum boundary limit; s represents the upper limit of grayscale image segmentation; α represents the mean square error of the grayscale image sub-block; β0 represents the mean of the grayscale image sub-block; η represents the atmospheric scattering coefficient.

[0184] Specifically, the step 4 includes:

[0185] Step R1: grayscale the restored image to obtain a grayscale image, and perform pixel point detection on the grayscale image using an ORB algorithm to determine feature points of the grayscale image;

[0186] Among them, the ORB (Oriented FAST and Rotated BRIEF) algorithm has the advantages of fast calculation speed, high matching accuracy, scale invariance and rotation invariance. The present invention uses this algorithm to extract feature points of grayscale images;

[0187] Step R2: input the feature point into a preset matching model to obtain a matching point set, select a group of pixels from the matching point set as a starting point, and use the starting point as the center point of the first area;

[0188] Step R3: Calculate the similarity between the surrounding pixels of the center point and the center point pixel by using the ZNCC method. When the obtained similarity is greater than a preset threshold, add the current surrounding pixels to the current area, and use the surrounding pixels of the center point as the new center point. Repeat step R3 to iterate sequentially.

[0189] When the obtained similarity is less than or equal to the preset threshold, the iteration is stopped to obtain pixels that meet the conditions;

[0190] Step R4: Mark the region composed of all pixels that meet the conditions, treat the marked region as an object, and return to R3 to continue to obtain a new center point. When all pixels in the entire image are marked as an object, obtain the regional growth disparity map;

[0191] Among them, the region growing algorithm is an image segmentation method based on pixel similarity. Its principle is to select a group of seed pixels as the starting point, and then gradually add pixels similar to the seed pixels into the same region by defining the similarity standard of adjacent pixels until all pixels are classified into a certain region.

[0192] Step R5: calculating the integral image of the region growth disparity map for subsequent mean filtering;

[0193] Step R6: obtaining the largest hole in the region growing disparity map, and calculating the area ratio of the hole to the region growing disparity map, selecting an initial filtering window of a preset size according to the area ratio, and filling the hole by mean filtering according to the integral image of the initial window and the region growing disparity map;

[0194] Among them, after region growing, a dense disparity map is obtained, but due to factors such as unclear image texture or uneven lighting, effective feature points cannot be extracted and matching cannot be successful, resulting in holes in the disparity map;

[0195] In order to improve the quality of subsequent stereo images, the present invention smoothes the pixels around the holes by using mean filtering, thereby filling the holes;

[0196] Step R7: reducing the initial window at a fixed reduction ratio to obtain a second filter window, and performing secondary hole filling by mean filtering according to the second filter window and the integral image;

[0197] Step R8: Repeat step R7, and when the filter window size reaches the preset specification, stop filtering to obtain the positioning information of the target object.

[0198] The steps for constructing the matching model are as follows:

[0199] Step S1: sampling feature points of the grayscale image by random sampling method to obtain sample points, and setting initial thresholds and initial model parameters according to the sample points;

[0200] Wherein, the initial thresholds include: an inlier error threshold, an inlier number threshold and a maximum number of iterations;

[0201] Step S2: when the number of iterations is less than or equal to the maximum number of iterations, the error values ​​of the initial model are calculated according to the non-sample points in the feature points, and the error values ​​are sorted in ascending order to obtain an error point set;

[0202] Step S3: Based on the error point set, select according to a preset error range to obtain a target error point set, calculate the inlier point set of the error point set and the target error point set under the current model, and obtain the number of inliers and the error range of the current model;

[0203] Step S4: Calculate the sampling probability of the feature points of the grayscale image according to the weight of the number of inliers and the error range, and randomly select a feature point from the sample points based on the sampling probability, and add it to the inlier set to update the number of inliers;

[0204] Step S5: Calculate the current model error and current model parameters based on the number of inliers, the error point set and the target error point set;

[0205] Step S6: When the number of inliers is less than or equal to the inlier number threshold, return to step 1 to continue iterating; when the number of inliers is greater than the inlier number threshold, obtain the current model parameters and the current inlier set.

[0206] Among them, the matching model can improve the accuracy of the fitting results when processing data containing noise and outliers while ensuring computational efficiency; its principle is to introduce "priority" to screen candidate models on the basis of random sampling, thereby improving the accuracy and robustness of the model.

[0207] In the present invention, the above method first performs grayscale preprocessing on the image to reduce the amount of calculation, and then uses the progressive consistent sampling method to optimize the matching points to improve the matching accuracy.

[0208] Then, the matching point pairs are used as initial seed points to perform region growing, generate a dense disparity map, and fill the holes in the disparity map to improve the integrity of the disparity map, thereby quickly performing object recognition and positioning.

[0209] Embodiment 2:

[0210] The present invention provides a schematic diagram of the connection of modules of a positioning system device based on binocular vision. Figure 2 As shown, including:

[0211] A parameter calculation module, which obtains a calibration image of the calibration plate and an original image of the target object through a binocular camera, extracts corner points of the calibration image, obtains calibration corner points, and calculates the internal and external parameters of the binocular camera according to the calibration corner points;

[0212] An image correction module, performing distortion correction processing on the original image according to the internal and external parameters to obtain a corrected image, and obtaining an overlapping effective area image of the original image and the corrected image based on the corrected image;

[0213] An image restoration module, based on the overlapped effective area image, performs defogging processing on the image pixels by a boundary restriction method to obtain a restored image;

[0214] The object positioning module performs stereo matching on the restored image through a region growing algorithm to obtain positioning information of the target object.

[0215] Specifically, the parameter calculation module is specifically used for:

[0216] Acquire a calibration image of the calibration plate through a preset binocular camera, and extract corner points of the calibration image to obtain calibration corner points;

[0217] Acquire the initial coordinates of the calibrated corner point based on the world coordinate system, perform coordinate transformation on the initial coordinates to obtain the transformed coordinates corresponding to the camera imaging plane, and determine the transformation mapping relationship according to the initial coordinates and the transformed coordinates;

[0218] According to the conversion mapping relationship, a homography matrix is ​​obtained, a symmetric matrix is ​​calculated according to the homography matrix, and a constraint equation is obtained according to the symmetric matrix and a unit orthogonal matrix;

[0219] According to the constraint equation and the six-dimensional vector in the symmetric matrix, the internal and external parameters of the binocular camera are calculated.

[0220] Specifically, the image correction module is specifically used for:

[0221] According to the internal and external parameters, a unit transformation relationship is obtained, and the original coordinates of the original image of the target object are obtained through a binocular camera;

[0222] According to the unit transformation relationship and the original coordinates, obtaining the corrected coordinates of the original image after distortion correction through the distortion model;

[0223] Wherein, the distortion model includes: a radial distortion model and a tangential distortion model;

[0224] Substituting the corrected coordinates into the established likelihood function, and performing logarithmic transformation on the likelihood function to obtain a log-likelihood function;

[0225] Solving the partial derivative of the log-likelihood function to obtain a maximum likelihood estimate of the distortion coefficient, and eliminating the distortion of the original image according to the maximum likelihood estimate to obtain a corrected image;

[0226] The corrected image is mapped to a corresponding position in the original image space to obtain an overlapping effective area image of the original image and the corrected image.

[0227] The maximum likelihood estimate is calculated as follows:

[0228]

[0229] in,

[0230] in,

[0231] Where G(x,y)=w<M(x,y)+Y(x,y)>

[0232] In the formula, F represents the maximum likelihood estimate; ε represents the distortion coefficient; Represents partial derivative; M(x,y) represents radial distortion model; Y(x,y) represents tangential distortion model; k1 represents the first unit transformation parameter; k2 represents the second unit transformation parameter; R represents the total number of all original coordinates in the original image; v(x,y) represents the original coordinate point in the original image; ρ represents radial distortion error; σ represents radial distortion parameter; μ represents tangential distortion parameter; τ represents tangential distortion error; b represents radial distortion factor; h represents probability distribution of radial distortion error; e represents probability distribution of tangential distortion error; q represents tangential distortion factor; G(x,y) represents correction coordinate; w represents correction parameter.

[0233] Specifically, the image restoration module is specifically used for:

[0234] Based on the overlapping effective area image, the grayscale image of the overlapping effective area image is divided into four blocks, and the mean square error and mean value of each sub-block are calculated;

[0235] Select the sub-block with the smallest mean square error in turn to continue the four-segmentation. When the segmented area is smaller than the set threshold, search for pixels with values ​​within the preset value range in the segmented image block, and the RGB values ​​corresponding to the pixels within the preset value can be considered as the atmospheric ambient light value, otherwise continue the segmentation;

[0236] Calculate and obtain a first transmittance of an image pixel according to the atmospheric ambient light value and a pixel limit in an image radiation cube;

[0237] Based on the first transmittance, adding a guided filter in the image block to obtain an optimized transmittance of the image pixel;

[0238] According to the atmospheric ambient light value and the optimized transmittance, the overlapping effective area image is defogged to obtain a restored image.

[0239] The restored image expression is as follows:

[0240]

[0241] in,

[0242] in,

[0243] Where p(x) represents the restored image; Z(x) represents the overlapped effective area image; x represents the pixel in the grayscale image; I(x) represents the atmospheric ambient light value corresponding to the pixel x; B represents the optimized transmittance; t(x) represents the scene transmittance; ζ represents the ratio of the ambient transmittance; θ represents the conversion factor; represents the guided filter parameter; t k Indicates pixel limit; mint k represents the minimum boundary limit; s represents the upper limit of grayscale image segmentation; α represents the mean square error of the grayscale image sub-block; β0 represents the mean of the grayscale image sub-block; η represents the atmospheric scattering coefficient.

[0244] Specifically, the object positioning module is specifically used for:

[0245] Step R1: grayscale the restored image to obtain a grayscale image, and perform pixel point detection on the grayscale image using an ORB algorithm to determine feature points of the grayscale image;

[0246] Step R2: input the feature point into a preset matching model to obtain a matching point set, select a group of pixels from the matching point set as a starting point, and use the starting point as the center point of the first area;

[0247] Step R3: Calculate the similarity between the surrounding pixels of the center point and the center point pixel by using the ZNCC method. When the obtained similarity is greater than a preset threshold, add the current surrounding pixels to the current area, and use the surrounding pixels of the center point as the new center point. Repeat step R3 to iterate sequentially.

[0248] When the obtained similarity is less than or equal to the preset threshold, the iteration is stopped to obtain pixels that meet the conditions;

[0249] Step R4: Mark the region composed of all pixels that meet the conditions, treat the marked region as an object, and return to R3 to continue to obtain a new center point. When all pixels in the entire image are marked as an object, obtain the regional growth disparity map;

[0250] Step R5: calculating the integral image of the region growth disparity map for subsequent mean filtering;

[0251] Step R6: obtaining the largest hole in the region growing disparity map, and calculating the area ratio of the hole to the region growing disparity map, selecting an initial filtering window of a preset size according to the area ratio, and filling the hole by mean filtering according to the integral image of the initial window and the region growing disparity map;

[0252] Step R7: reducing the initial window at a fixed reduction ratio to obtain a second filter window, and performing secondary hole filling by mean filtering according to the second filter window and the integral image;

[0253] Step R8: Repeat step R7, and when the filter window size reaches the preset specification, stop filtering to obtain the positioning information of the target object.

[0254] The steps for constructing the matching model are as follows:

[0255] Step S1: sampling feature points of the grayscale image by random sampling method to obtain sample points, and setting initial thresholds and initial model parameters according to the sample points;

[0256] Wherein, the initial thresholds include: an inlier error threshold, an inlier number threshold and a maximum number of iterations;

[0257] Step S2: when the number of iterations is less than or equal to the maximum number of iterations, the error values ​​of the initial model are calculated according to the non-sample points in the feature points, and the error values ​​are sorted in ascending order to obtain an error point set;

[0258] Step S3: Based on the error point set, select according to a preset error range to obtain a target error point set, calculate the inlier point set of the error point set and the target error point set under the current model, and obtain the number of inliers and the error range of the current model;

[0259] Step S4: Calculate the sampling probability of the feature points of the grayscale image according to the weight of the number of inliers and the error range, and randomly select a feature point from the sample points based on the sampling probability, and add it to the inlier set to update the number of inliers;

[0260] Step S5: Calculate the current model error and current model parameters based on the number of inliers, the error point set and the target error point set;

[0261] Step S6: When the number of inliers is less than or equal to the inlier number threshold, return to step 1 to continue iterating; when the number of inliers is greater than the inlier number threshold, obtain the current model parameters and the current inlier set.

[0262] See also Figure 3 , Figure 3 Schematic diagram of an electronic device provided by an embodiment of the present invention. Figure 3As shown, an embodiment of the present invention provides an electronic device 500, including a memory 510, a processor 520, and a computer program 511 stored in the memory 510 and executable on the processor 520. When the processor 520 executes the computer program 511, the following steps are implemented:

[0263] Memory for storing computer software programs;

[0264] The processor is used to read and execute the computer software program, thereby implementing the positioning method based on laser radar and inertial navigation.

[0265] See also Figure 4 , Figure 4 A schematic diagram of an embodiment of a computer-readable storage medium provided in an embodiment of the present invention. Figure 4 As shown, this embodiment provides a computer-readable storage medium 600, on which a computer program 611 is stored. When the computer program 611 is executed by a processor, the following steps are implemented:

[0266] The storage medium stores a computer software program, which, when executed by a processor, implements the positioning method based on laser radar and inertial navigation.

[0267] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0268] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0269] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0270] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0271] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0272] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0273] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A positioning method based on binocular vision, characterized in that: include: Acquire a calibration image of the calibration plate and an original image of the target object through a binocular camera, extract corner points of the calibration image, obtain calibration corner points, and calculate the internal and external parameters of the binocular camera according to the calibration corner points; According to the internal and external parameters, the original image is subjected to distortion correction processing to obtain a corrected image, and based on the corrected image, an image of an overlapping effective area of ​​the original image and the corrected image is obtained; Based on the overlapping effective area image, defogging the image pixels by a boundary restriction method to obtain a restored image; The restored images are stereo matched using a region growing algorithm to obtain the location information of the target object.

2. A binocular vision-based positioning method according to claim 1, characterized in that: The method of obtaining a calibration image of the calibration plate and an original image of the target object through a binocular camera, extracting corner points of the calibration image, obtaining calibration corner points, and calculating the internal and external parameters of the binocular camera according to the calibration corner points includes: Acquire a calibration image of the calibration plate through a preset binocular camera, and extract corner points of the calibration image to obtain calibration corner points; Acquire the initial coordinates of the calibrated corner point based on the world coordinate system, perform coordinate transformation on the initial coordinates to obtain the transformed coordinates corresponding to the camera imaging plane, and determine the transformation mapping relationship according to the initial coordinates and the transformed coordinates; According to the conversion mapping relationship, a homography matrix is ​​obtained, a symmetric matrix is ​​calculated according to the homography matrix, and a constraint equation is obtained according to the symmetric matrix and a unit orthogonal matrix; According to the constraint equation and the six-dimensional vector in the symmetric matrix, the internal and external parameters of the binocular camera are calculated.

3. A binocular vision-based positioning method according to claim 1, characterized in that: The method of performing distortion correction processing on the original image according to the internal and external parameters to obtain a corrected image, and obtaining an overlapping effective area image of the original image and the corrected image based on the corrected image, includes: According to the internal and external parameters, a unit transformation relationship is obtained, and the original coordinates of the original image of the target object are obtained through a binocular camera; According to the unit transformation relationship and the original coordinates, obtaining the corrected coordinates of the original image after distortion correction through the distortion model; Wherein, the distortion model includes: a radial distortion model and a tangential distortion model; Substituting the corrected coordinates into the established likelihood function, and performing logarithmic transformation on the likelihood function to obtain a log-likelihood function; Solving the partial derivative of the log-likelihood function to obtain a maximum likelihood estimate of the distortion coefficient, and eliminating the distortion of the original image according to the maximum likelihood estimate to obtain a corrected image; The corrected image is mapped to a corresponding position in the original image space to obtain an overlapping effective area image of the original image and the corrected image.

4. A binocular vision-based positioning method according to claim 3, characterized in that: The maximum likelihood estimate is calculated as follows: in, in, Among them, G(x, y)=w(M(x, y)+Y(x, y)> In the formula, F represents the maximum likelihood estimate; ε represents the distortion coefficient; Represents partial derivative; M(x, y) represents radial distortion model; Y(x, y) represents tangential distortion model; k1 represents the first unit transformation parameter; k2 represents the second unit transformation parameter; R represents the total number of all original coordinates in the original image; v(x, y) represents the original coordinate point in the original image; ρ represents radial distortion error; σ represents radial distortion parameter; μ represents tangential distortion parameter; τ represents tangential distortion error; b represents radial distortion factor; h represents probability distribution of radial distortion error; e represents probability distribution of tangential distortion error; q represents tangential distortion factor; G(x, y) represents correction coordinate; w represents correction parameter.

5. The binocular vision-based positioning method according to claim 1, characterized in that: The step of performing defogging processing on the image pixels based on the overlapped effective area image by limiting the boundaries to obtain a restored image includes: Based on the overlapping effective area image, the grayscale image of the overlapping effective area image is divided into four blocks, and the mean square error and mean value of each sub-block are calculated; Select the sub-block with the smallest mean square error in turn to continue the four-segmentation. When the segmented area is smaller than the set threshold, search for pixels with values ​​within the preset value range in the segmented image block, and the RGB values ​​corresponding to the pixels within the preset value can be considered as the atmospheric ambient light value, otherwise continue the segmentation; Calculate and obtain a first transmittance of an image pixel according to the atmospheric ambient light value and a pixel limit in an image radiation cube; Based on the first transmittance, adding a guided filter in the image block to obtain an optimized transmittance of the image pixel; According to the atmospheric ambient light value and the optimized transmittance, the overlapping effective area image is defogged to obtain a restored image. The restored image expression is as follows: in, in, Where p(x) represents the restored image; Z(x) represents the overlapped effective area image; x represents the pixel in the grayscale image; I(x) represents the atmospheric ambient light value corresponding to the pixel x; B represents the optimized transmittance; t(x) represents the scene transmittance; ζ represents the ratio of the ambient transmittance; θ represents the conversion factor; represents the guided filter parameter; t k Indicates pixel limit; mint k represents the minimum boundary limit; s represents the upper limit of grayscale image segmentation; α represents the mean square error of the grayscale image sub-block; β0 represents the mean of the grayscale image sub-block; η represents the atmospheric scattering coefficient.

6. A binocular vision-based positioning method according to claim 1, characterized in that: The performing stereo matching on the restored image by using a region growing algorithm to obtain the positioning information of the target object includes: Step R1: grayscale the restored image to obtain a grayscale image, and perform pixel detection on the grayscale image using an ORB algorithm to determine feature points of the grayscale image; Step R2: input the feature point into a preset matching model to obtain a matching point set, select a group of pixels from the matching point set as a starting point, and use the starting point as the center point of the first area; Step R3: Calculate the similarity between the surrounding pixels of the center point and the center point pixel by using the ZNCC method. When the obtained similarity is greater than a preset threshold, add the current surrounding pixels to the current area, and use the surrounding pixels of the center point as the new center point. Repeat step R3 to iterate sequentially. When the obtained similarity is less than or equal to the preset threshold, the iteration is stopped to obtain pixels that meet the conditions; Step R4: Mark the region composed of all pixels that meet the conditions, treat the marked region as an object, and return to R3 to continue to obtain a new center point. When all pixels in the entire image are marked as an object, obtain the regional growth disparity map; Step R5: calculating the integral image of the region growth disparity map for subsequent mean filtering; Step R6: obtaining the largest hole in the region growing disparity map, and calculating the area ratio of the hole to the region growing disparity map, selecting an initial filtering window of a preset size according to the area ratio, and filling the hole by mean filtering according to the integral image of the initial window and the region growing disparity map; Step R7: reducing the initial window at a fixed reduction ratio to obtain a second filter window, and performing secondary hole filling by mean filtering according to the second filter window and the integral image; Step R8: Repeat step R7, and when the filter window size reaches the preset specification, stop filtering to obtain the positioning information of the target object.

7. A binocular vision-based positioning method according to claim 6, characterized in that: The steps for constructing the matching model are as follows: Step S1: sampling feature points of the grayscale image by random sampling method to obtain sample points, and setting initial thresholds and initial model parameters according to the sample points; Wherein, the initial thresholds include: an inlier error threshold, an inlier number threshold and a maximum number of iterations; Step S2: when the number of iterations is less than or equal to the maximum number of iterations, the error values ​​of the initial model are calculated according to the non-sample points in the feature points, and the error values ​​are sorted in ascending order to obtain an error point set; Step S3: Based on the error point set, select according to a preset error range to obtain a target error point set, calculate the inlier point set of the error point set and the target error point set under the current model, and obtain the number of inliers and the error range of the current model; Step S4: Calculate the sampling probability of the feature points of the grayscale image according to the weight of the number of inliers and the error range, and randomly select a feature point from the sample points based on the sampling probability, and add it to the inlier set to update the number of inliers; Step S5: Calculate the current model error and current model parameters based on the number of inliers, the error point set and the target error point set; Step S6: When the number of inliers is less than or equal to the inlier number threshold, return to step 1 to continue iterating; when the number of inliers is greater than the inlier number threshold, obtain the current model parameters and the current inlier set.

8. A positioning device based on binocular vision, characterized in that: include: A parameter calculation module, which obtains a calibration image of the calibration plate and an original image of the target object through a binocular camera, extracts corner points of the calibration image, obtains calibration corner points, and calculates the internal and external parameters of the binocular camera according to the calibration corner points; An image correction module, performing distortion correction processing on the original image according to the internal and external parameters to obtain a corrected image, and obtaining an overlapping effective area image of the original image and the corrected image based on the corrected image; An image restoration module, based on the overlapped effective area image, performs defogging processing on the image pixels by a boundary restriction method to obtain a restored image; The object positioning module performs stereo matching on the restored image through a region growing algorithm to obtain positioning information of the target object.

9. An electronic device, characterized in that: include: Memory for storing computer software programs; A processor is used to read and execute the computer software program, thereby implementing a binocular vision-based positioning method as described in any one of claims 1-8.

10. A non-transitory computer-readable storage medium, characterized in that: The storage medium stores a computer software program, and when the computer software program is executed by the processor, it implements a binocular vision-based positioning method as described in any one of claims 1 to 7.

Images