Target positioning method and system based on dynamic color threshold and laser depth cooperation

Abstract

The invention discloses a target positioning method and system based on coordination of a dynamic color threshold value and laser depth, and the method comprises the steps: collecting a left view and a right view of a detection environment through a binocular camera, and taking an object which accords with a set threshold value color and accords with a set area and a length-width ratio threshold value in the left view and the right view as a target; calculating a center coordinate of a target in a pixel coordinate system under the binocular camera; obtaining the depth of the target according to the internal reference matrix of the camera, and when the depth is normal, converting the center coordinate of the target under the pixel coordinate system into a polar coordinate by adopting matrix transformation to obtain a coarse positioning result of the target; subscribing laser radar data, performing time registration on the laser radar data and the left and right views, extracting target distance information in a laser array, and calculating a target position after laser calibration when a matching point cloud exists; and dynamically updating a target coordinate estimated value by using the optimized Kalman filtering gain to complete target positioning. According to the invention, robust sensing and high-precision positioning in different illumination environments are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition and target positioning, and in particular to a target positioning method and system based on dynamic color threshold and laser depth collaboration. Background Art

[0002] With the rapid development of fields such as autonomous driving and intelligent transportation, people's daily living standards are constantly improving, and the demand for target detection and high-precision positioning technology is becoming increasingly urgent. However, the existing target detection and positioning systems still have defects to varying degrees: the Chinese patent with the publication number CN109634279B and the invention name of the object positioning method based on laser radar and monocular vision uses a deep learning algorithm to identify the target and locate the target through laser radar. This method requires a large amount of data to be labeled and trained, with high deployment costs, and cannot achieve target positioning in different lighting environments; the Chinese patent with the publication number CN118392151B and the invention name of the indoor and outdoor positioning method based on multi-sensor fusion algorithm fuses and corrects the reference positioning information with the multi-sensor positioning data to achieve target positioning. This method lacks feedback verification of target recognition, and once the erroneous data enters the fusion process, it will cause irreversible positioning drift. Summary of the invention

[0003] The purpose of the present invention is to provide a target positioning method and system based on the collaboration of dynamic color threshold and laser depth, to achieve robust perception and high-precision positioning in different lighting environments through laser point cloud, real-time feedback verification of visual depth and adaptive threshold recalibration. The present invention has achieved significant improvements in positioning accuracy and environmental adaptability, and is particularly suitable for target recognition and positioning of drones, unmanned vehicles, etc. in uncertain lighting environments.

[0004] The present invention adopts the following technical solution: a target positioning method based on dynamic color threshold and laser depth coordination, comprising the following steps:

[0005] S1. The left and right views of the detection environment are collected by the binocular camera. The objects in the left and right views that meet the set threshold color and the set area and aspect ratio thresholds are taken as targets. The color features of the targets are extracted using the HSV color space. The target bounding box is extracted using area / aspect ratio filtering analysis. The center coordinates of the target in the pixel coordinate system under the binocular camera are calculated.

[0006] S2. Obtain the depth of the target according to the camera intrinsic parameter matrix. If the depth is abnormal, perform threshold recalibration. If the depth is normal, use matrix transformation to convert the center coordinates of the target in the pixel coordinate system into camera coordinates, convert the camera coordinates into polar coordinates, and use the polar coordinates as the coarse positioning result of the target.

[0007] S3. Subscribe to the lidar data, align the lidar data with the left and right views in time, extract the target distance information in the laser array according to the rough positioning result of the target, use the anomaly assessment factor to perform spatial consistency check, if there is no matching point cloud, perform threshold recalibration, if there is a matching point cloud, calculate the target position after laser calibration.

[0008] S4. Use bimodal confidence driving to optimize the Kalman filter gain, dynamically update the target coordinate estimate according to the optimized Kalman filter gain, and complete target positioning.

[0009] Furthermore, in step S1, calculating the center coordinates of the target in the pixel coordinate system includes the following:

[0010] S101. Use ROS to collect the left and right views of the detection environment through a binocular camera, use cv_bridge to convert the view into an OpenCV BGR image, convert the BGR image into an HSV color space, use the hue, saturation, and brightness channels to separate the color features of the target, dynamically load the threshold ranges of the five colors of red, blue, green, brown, and white through the ROS parameter server, generate the corresponding binary mask, and complete the extraction of color features.

[0011] S102, using an opening operation method of first corroding and then dilating to eliminate isolated noise points of the binary mask, and then superimposing the mask and performing a dilation operation to obtain a fused mask.

[0012] S103, using findContours to detect the contour of the fused mask, retaining the outermost contour, traversing the outermost contour, and using a double filtering mechanism to eliminate contours that are smaller than an area threshold and are not within an aspect ratio threshold range, to obtain a target bounding box.

[0013] S104, using the priority strategy of red, blue, green, brown, and white from high to low priority to classify the overlapping color areas of the target bounding box, obtain the corresponding color block color, and calculate the pixel coordinates of the center point of the target under the binocular camera. The specific formula is:

[0014] left_xzhong=(left_self.x+left_self.x+left_self.w) / 2;

[0015] left_yzhong=(left_self.y+left_self.y-left_self.h) / 2;

[0016] right_xzhong=(right_self.x+right_self.x+right_self.w) / 2;

[0017] right_yzhong=(right_self.y+right_self.y-right_self.h) / 2;

[0018] Among them, left_xzhong and left_yzhong represent the horizontal and vertical coordinates of the center pixel of the target under the left camera respectively; right_xzhong and right_yzhong represent the horizontal and vertical coordinates of the center pixel of the target under the right camera respectively; left_self.x and left_self.y represent the horizontal and vertical coordinates of the upper left corner vertex of the target bounding box of the left camera respectively, left_self.w and left_self.h represent the width and height of the target bounding box of the left camera respectively, right_self.x and right_self.y represent the horizontal and vertical coordinates of the upper left corner vertex of the target bounding box of the right camera respectively, right_self.w and right_self.h represent the width and height of the target bounding box of the right camera respectively.

[0019] Furthermore, in step S2, the rough positioning result of the target is obtained, including the following contents:

[0020] S201, according to the horizontal difference of the target center of the left and right views, combined with the baseline distance, the depth Z of the target is obtained, and the specific formula is:

[0021] ;

[0022] Where b is the distance between the optical centers of the two cameras, f is the focal length of the camera, and X L Indicates the pixel coordinates of the target on the left camera, X R Indicates the pixel coordinates of the target on the right camera.

[0023] If Z is not between 0.5m and 20m, it indicates that the depth is abnormal and the threshold should be recalibrated; if Z is between 0.5m and 20m, it indicates that the depth is normal.

[0024] S202, when the depth is normal, the center point coordinates of the target under the binocular camera are converted from the pixel coordinate system to the camera coordinate system by using the principle of proportional magnification of similar triangles, and the preliminary coordinates of the target are obtained. The specific formula is:

[0025] ;

[0026] in, represents the mean value of the horizontal coordinate of the target pixel under the binocular camera, ; Represents the mean value of the ordinate of the target pixel under the binocular camera, ;X C Indicates the horizontal coordinate of the target in the camera coordinate system; YC Indicates the target vertical coordinate in the camera coordinate system; Z C Indicates the vertical distance between the camera and the target plane, Z C =Z;f x 、f y Both represent the focal length of the camera.

[0027] S203, converting the preliminary coordinates of the target into polar coordinates to obtain a rough positioning result of the target. The specific formula is:

[0028] A=atan(X C / Z);

[0029] S = Z / cos(A);

[0030] Wherein, A represents the first horizontal deflection angle of the target relative to the front direction of the laser radar; S represents the horizontal distance between the target and the laser radar.

[0031] Further, in step S3, obtaining the target position after laser calibration includes the following contents:

[0032] S301, linearly interpolate the point cloud data in the laser radar data to obtain new point cloud data. The specific formula is:

[0033] ;

[0034] Among them, P virtual represents new point cloud data; P0 represents the point cloud data of the previous frame; P1 represents the point cloud data of the current frame; t m Indicates inserting a new time. , t0 represents the time of the previous frame, and t1 represents the time of the current frame.

[0035] The new point cloud data and the corresponding ROS timestamp are stored in the scan array, and the ROS timestamp is compared with the timestamp corresponding to the target coarse positioning result, and the point cloud data closest to the target coarse positioning result is selected to complete the time alignment.

[0036] S302, according to the first horizontal deflection angle of the target relative to the front direction of the laser radar, the corresponding laser radar scanning index is calculated, and the specific formula is:

[0037] ;

[0038] Among them, zhong represents the index value of the lidar data matching A; represents the angle increment, , length represents the length of the array storing the new point cloud data.

[0039] S303, calculating the polar coordinates of the target relative to the laser radar according to the laser radar scanning index, the specific formula is:

[0040] ; ; ;

[0041] Wherein, number represents the laser radar target index value, median represents the median function, error represents the measurement error range of S set according to experience, N represents the laser radar index array after time alignment, i represents the laser index corresponding to the laser distance in the range of S-error and S+error, M represents the laser radar distance array after time alignment, B represents the second horizontal deflection angle of the target relative to the front direction of the laser radar, and distance represents the target distance information in the laser radar distance array.

[0042] If the difference between distance and S is within 1m, the distance is normal, indicating that there is a matching point cloud; otherwise, the number of abnormalities is recorded and the abnormal situation is further judged. The specific formula is:

[0043] ;

[0044] in, represents the anomaly assessment factor, Q represents the number of times the distance anomaly occurs at least twice in a row, and k represents the total number of scans.

[0045] like Greater than the empirical threshold , indicating that there is no point cloud matching the target bounding box, confirming an abnormality and performing threshold recalibration; if Not greater than the empirical threshold , indicating that there is a point cloud that matches the target bounding box.

[0046] S304. When there is a matching point cloud, the polar coordinates obtained in step S303 are converted into rectangular coordinates to obtain the laser point cloud target positioning coordinates in the camera coordinate system. The specific formula is:

[0047] x 1=distance*cos(B);

[0048] y1=distance*sin(B);

[0049] Among them, x1 represents the horizontal coordinate of the laser point cloud target positioning in the camera coordinate system, y1 represents the vertical coordinate of the laser point cloud target positioning in the camera coordinate system, cos represents the cosine function, and sin represents the sine function.

[0050] S305, converting the laser point cloud target positioning coordinates in the camera coordinate system into the laser point cloud target positioning coordinates in the global coordinate system to obtain the target position after laser calibration. The specific formula is:

[0051] ;

[0052] Among them, X and Y represent the horizontal and vertical coordinates of the target in the global coordinate system; R represents the rotation matrix, , , , They represent the yaw angle, pitch angle, and roll angle of the robot respectively.

[0053] Furthermore, in step S4, completing target positioning includes the following:

[0054] S401, using the weight distribution mechanism to dynamically calculate the Kalman gain, using the dual-mode confidence drive to assist in calculating the Kalman gain, the calculation formula is:

[0055] ;

[0056] K_x=K adaptive *Eesti_x / (Emea_x+Eesti_x);

[0057] K_y=K adaptive *Eesti_y / (Emea_y+Eesti_y);

[0058] Among them, K adaptive represents the Kalman gain correction coefficient, C color Represents binocular vision color confidence, C laster Indicates the confidence level of laser calibration, Represents the visual-laser positioning difference attenuation coefficient, represents the Euclidean distance difference between binocular vision positioning and laser positioning, K_x represents the Kalman gain of the target horizontal coordinate, K_y represents the Kalman gain of the target vertical coordinate, Eesti_x represents the estimated error of the target horizontal coordinate, Eesti_y represents the estimated error of the target vertical coordinate, Emea_x represents the measurement error of the target horizontal coordinate, Emea_y represents the measurement error of the target vertical coordinate, represents the modal dominant factor.

[0059] S402, according to the Kalman gain, the predicted value and the measured value are continuously weighted averaged through iteration to obtain the optimal estimated value after filtering, which is the estimated value of the target coordinates. The specific formula is as follows:

[0060] ;

[0061] ;

[0062] in, Indicates The optimal target coordinate estimate after iterations is: Indicates The optimal target horizontal coordinate estimate after iterations is: Indicates The optimal target ordinate estimate after iterations is: Indicates The optimal target horizontal coordinate estimate after iterations.

[0063] S403, dynamically update the estimated error, the formula is as follows:

[0064] Eesti_xg=(1.0-K_x)*Eesti_x;

[0065] Eesti_yg=(1.0-K_y)*Eesti_y;

[0066] Among them, Eesti_xg represents the estimated error of the updated target horizontal coordinate, and Eesti_yg represents the estimated error of the updated target vertical coordinate.

[0067] S404. Based on the updated estimation error, repeat steps S401-S403 to complete iteration and realize dynamic optimization.

[0068] Furthermore, the threshold recalibration includes the following:

[0069] Convert the BGR image to HSV color space and extract the brightness channel. The specific formula is:

[0070] ;

[0071] ;

[0072] ;

[0073] in, represents the predicted brightness change, Represents the arithmetic mean of all pixels in the V channel, , All represent adjustable parameters. Represents the absolute value of the brightness gradient, Indicates the decrease value of the brightness threshold, represents the local contrast, Indicates the increase in brightness threshold, Indicates the average brightness of the brightest 5% area in the view.

[0074] Will and Linearly mapped into the range of 0-255 and , Indicates the pixel brightness threshold reduction amount, Indicates the pixel brightness threshold increase.

[0075] Calculate the arithmetic mean of all pixels in the brightness channel and normalize the result to the range of 0-2 to obtain the global brightness; when the global brightness is lower than 0.7, reduce the pixel brightness threshold ; When the global brightness is higher than 1.3, increase the pixel brightness threshold .

[0076] Furthermore, the present invention also proposes a target positioning system based on dynamic color threshold and laser depth coordination, comprising:

[0077] The target center pixel coordinate acquisition module is used to collect the left and right views of the detection environment through the binocular camera, take the objects in the left and right views that meet the set threshold color and meet the set area and aspect ratio thresholds as the target, use the HSV color space to extract the color features of the target, use the area / aspect ratio filtering analysis to extract the target bounding box, and calculate the center coordinates of the target in the pixel coordinate system under the binocular camera.

[0078] The module for obtaining the coarse positioning result of the target is used to obtain the depth of the target according to the camera intrinsic parameter matrix. If the depth is abnormal, the threshold is recalibrated. If the depth is normal, the matrix transformation is used to convert the center coordinates of the target in the pixel coordinate system into camera coordinates, and the camera coordinates are transformed into polar coordinates, which are used as the coarse positioning result of the target.

[0079] The target position acquisition module after laser calibration is used to subscribe to the laser radar data, time-align the laser radar data with the left and right views, extract the target distance information in the laser array according to the rough positioning result of the target, and use the anomaly assessment factor to perform spatial consistency verification. If there is no matching point cloud, the threshold is recalibrated. If there is a matching point cloud, the target position after laser calibration is calculated.

[0080] The target positioning module is used to optimize the Kalman filter gain using dual-modal confidence driving, dynamically update the target coordinate estimation value according to the optimized Kalman filter gain, and complete the target positioning.

[0081] Furthermore, the present invention also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the target positioning method based on the coordination of dynamic color threshold and laser depth are implemented.

[0082] Furthermore, the present invention also proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is run by a processor, the target positioning method based on the coordination of dynamic color threshold and laser depth is executed.

[0083] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0084] The present invention realizes dynamic loading of color thresholds based on the ROS parameter server, can complete robust recognition of targets under different lighting conditions, and greatly improves the accuracy of target recognition through laser-depth collaborative verification.

[0085] The present invention adopts dual-mode confidence driving and reconstructs the Kalman gain calculation formula to achieve adaptive suppression of measurement noise, thereby significantly improving the target positioning accuracy.

[0086] The target recognition accuracy of the present invention is stable at more than 95.7% under different lighting conditions, and the positioning accuracy of the target is improved to the quasi-millimeter level (the positioning error is within 2 cm). At the same time, the modular design of the present invention supports the rapid deployment of mobile platforms such as drones. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 It is an overall implementation flow chart of the present invention.

[0088] Figure 2 It is the target image recognized by the binocular camera through the double filtering mechanism in the embodiment of the present invention.

[0089] Figure 3 It is a pixel coordinate diagram obtained by binocular camera target detection in an embodiment of the present invention.

[0090] Figure 4 It is a schematic diagram of the stereoscopic vision depth calculation principle in an embodiment of the present invention.

[0091] Figure 5 It is a depth map obtained for the same target for 20 consecutive seconds in an embodiment of the present invention.

[0092] Figure 6 It is a schematic diagram of the principle of transforming the target from the pixel coordinate system to the camera coordinate system in an embodiment of the present invention.

[0093] Figure 7 It is a polar coordinate diagram of the rough positioning of the target in the embodiment of the present invention.

[0094] Figure 8 It is a horizontal and vertical coordinate diagram of the target obtained by combining laser data in an embodiment of the present invention.

[0095] Fig. 9It is a target horizontal and vertical coordinate diagram obtained after Kalman filtering in an embodiment of the present invention.

[0096] Fig.10 It is a graph of the horizontal and vertical coordinate errors of positioning 100 different targets in an embodiment of the present invention.

[0097] Fig.11 It is a schematic diagram of the UAV in an embodiment of the present invention equipped with the laser radar and binocular camera required in the patent. DETAILED DESCRIPTION

[0098] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0099] To achieve the above objectives, the present invention proposes a target positioning method based on the collaboration of dynamic color threshold and laser depth. First, the binocular camera image is calibrated with a color threshold, and the target candidate area is calibrated through morphological denoising. The target contour is obtained using the geometric constraints of aspect ratio and area to complete preliminary target identification. The target is initially positioned based on the pixel coordinates of the target in the contour in combination with the binocular target depth formula. After aligning the laser radar with the visual data timestamp, the laser point cloud is used to verify the preliminary recognition result of the binocular vision to obtain the final high-precision positioning result. The present invention significantly improves the recognition accuracy and robustness of the system, ensuring that the system can efficiently and reliably complete target recognition and precise positioning in different lighting environments. Figure 1 As shown, the specific steps are as follows:

[0100] S1. The left and right views of the detection environment are collected by the binocular camera. The objects in the left and right views that meet the set threshold color and the set area and aspect ratio threshold are taken as targets. The color features of the targets are extracted using the HSV (Hue-Saturation-Value) color space. The fused mask is obtained using morphological denoising. The target bounding box is extracted using area / aspect ratio filtering analysis. The center coordinates of the target in the pixel coordinate system under the binocular camera are calculated. The specific contents are as follows:

[0101] S101. Use ROS (Robot Operating System) to collect left and right views of the detection environment through a binocular camera, use cv_bridge (OpenCV ROS Bridge, a library for converting image data in the robot operating system) to convert the view into a BGR (Blue-Green-Red) image of OpenCV (Open Source Computer Vision Library), convert the BGR image into an HSV color space, use the three channels of hue, saturation, and brightness to separate the color features of the target, dynamically load the threshold ranges of the five colors of red, blue, green, brown, and white through the ROS parameter server, generate the corresponding binary mask, and complete the extraction of color features.

[0102] S102, using an opening operation of first corroding and then dilating to eliminate isolated noise points of the binary mask, and then superimposing the mask and performing a dilation operation to obtain a fused mask, thereby enhancing the connectivity of the target area and avoiding mask breakage caused by uneven illumination.

[0103] S103, use findContours (a function in Opencv used to detect the contour of an object) to detect the contour of the fused mask, retain the outermost contour, traverse the outermost contour, and use a double filtering mechanism to remove contours that are smaller than the area threshold and are not within the aspect ratio threshold range, which is used to exclude small area noise and non-target shapes such as slender or flat, ensure that the target shape is close to a rectangle, and obtain a rectangular target bounding box, such as Figure 2 As shown, Figure 2 (a) is the target bounding box recognized by the left camera. Figure 2 (b) is the target bounding box recognized by the right camera. The red box represents the recognized target, and the green words represent the recognized target color.

[0104] S104, such as Figure 3 As shown in the figure, the overlapping color areas of the target bounding box are classified using a priority strategy of red, blue, green, brown, and white from high to low to avoid conflicts between multi-color targets, obtain the corresponding color blocks, and calculate the pixel coordinates of the center point of the target under the binocular camera. The specific formula is:

[0105] left_xzhong=(left_self.x+left_self.x+left_self.w) / 2;

[0106] left_yzhong=(left_self.y+left_self.y-left_self.h) / 2;

[0107] right_xzhong=(right_self.x+right_self.x+right_self.w) / 2;

[0108] right_yzhong=(right_self.y+right_self.y-right_self.h) / 2;

[0109] Among them, left_xzhong and left_yzhong represent the horizontal and vertical coordinates of the center pixel of the target under the left camera respectively; right_xzhong and right_yzhong represent the horizontal and vertical coordinates of the center pixel of the target under the right camera respectively; left_self.x and left_self.y represent the horizontal and vertical coordinates of the upper left corner vertex of the target bounding box of the left camera respectively, left_self.w and left_self.h represent the width and height of the target bounding box of the left camera respectively, right_self.x and right_self.y represent the horizontal and vertical coordinates of the upper left corner vertex of the target bounding box of the right camera respectively, right_self.w and right_self.h represent the width and height of the target bounding box of the right camera respectively.

[0110] Among them, the threshold colors are set as follows: the upper and lower thresholds of red HSV are set to [0, 240, 70] and [10, 255,190], the upper and lower thresholds of blue HSV are set to [110, 210, 50] and [120, 240, 90], the upper and lower thresholds of green HSV are set to [50, 170, 40] and [60, 190, 90], the upper and lower thresholds of brown HSV are set to [5, 190, 30] and [10, 220,70], and the upper and lower thresholds of white HSV are set to [100, 10, 90] and [110, 30, 220].

[0111] The area threshold is 500 pixels, and the aspect ratio threshold ranges from 0.3 to 1.6.

[0112] Figure 3 (a) is the horizontal coordinate diagram of the target center pixel of the left camera obtained for the same target for 20 consecutive seconds. It can be seen from the figure that the horizontal coordinate of the target center pixel of the left camera is stable at around 152 pixels. Figure 3 (b) is the relationship between the horizontal coordinate and the vertical coordinate of the target center pixel under the left camera obtained for the same target for 20 consecutive seconds. It can be seen from the figure that when the horizontal coordinate of the target center pixel of the left camera is 152, the corresponding vertical coordinate is 179. Figure 3 (c) is the horizontal coordinate diagram of the target center pixel of the right camera obtained for the same target within 20 seconds. It can be seen from the figure that the horizontal coordinate of the target center pixel of the right camera is stable at around 143 pixels. Figure 3 (d) is the relationship between the horizontal coordinate and vertical coordinate of the target center pixel under the right camera obtained for the same target for 20 consecutive seconds. It can be seen from the figure that when the horizontal coordinate of the target center pixel of the right camera is 143, the corresponding vertical coordinate is 176. Figure 3 It can be seen that the target recognition of the binocular camera is relatively stable. There is a 9-pixel difference in the horizontal coordinates of the target center pixels of the binocular camera, but the vertical coordinates of the target center pixels are almost the same.

[0113] S2. Obtain the depth of the target according to the camera intrinsic parameter matrix. If the depth is abnormal, perform threshold recalibration. If the depth is normal, use matrix transformation to convert the center coordinates of the target in the pixel coordinate system into camera coordinates, convert the camera coordinates into polar coordinates, and use the polar coordinates as the rough positioning result of the target. The specific contents are as follows:

[0114] S201, Stereoscopic Vision Depth Calculation: Figure 4 As shown in the figure, the imaging point of the same target in the binocular camera will have a certain deviation. According to the horizontal difference of the target center in the left and right views, combined with the baseline distance, the depth Z of the target is obtained. The specific formula is:

[0115] ;

[0116] Where b represents the distance between the optical centers of the two cameras, also called the baseline, which is generally obtained by consulting the camera sensor parameter configuration; f represents the camera focal length, which can be obtained by consulting the camera intrinsic parameter matrix; X L Indicates the pixel coordinates of the target on the left camera, X R The pixel coordinates representing the target on the right camera are obtained through step S104.

[0117] In the figure, P represents the target, which is the imaging point of the target on the left camera; L represents the length of the horizontal axis of the camera pixel; P L represents the imaging point of the target on the left camera; P R represents the imaging point of the target on the right camera; O L Indicates the shooting point of the left camera; O R Indicates the shooting point of the right camera.

[0118] If Z is not between 0.5m and 20m, it indicates that the depth is abnormal and the threshold should be recalibrated; if Z is between 0.5m and 20m, it indicates that the depth is normal.

[0119] The calculation results are as follows Figure 5 As shown, Figure 5 The target depth map obtained for the same target for 20 consecutive seconds is shown. It can be seen from the figure that the calculated target depth fluctuates around 5m, with a large fluctuation range. Therefore, the target depth is sometimes calculated inaccurately, and it is necessary to combine the laser data in step S3 for precise positioning.

[0120] S202, when the depth is normal, the center point coordinates of the target under the binocular camera are converted from the pixel coordinate system to the camera coordinate system by using the principle of proportional magnification of similar triangles, and the preliminary coordinates of the target are obtained, such as Figure 6 As shown, the specific formula is:

[0121] ;

[0122] in, represents the mean value of the horizontal coordinate of the target pixel under the binocular camera, ; Represents the mean value of the ordinate of the target pixel under the binocular camera, ;X C Indicates the horizontal coordinate of the target in the camera coordinate system; Y C Indicates the target vertical coordinate in the camera coordinate system; Z C Indicates the vertical distance between the camera and the target plane, Z C =Z;f x 、f y Both represent the focal length of the camera.

[0123] In the figure, O C represents the midpoint of the line segment where the left and right camera shooting points are located, O represents the midpoint of the line segment where the target is located at the left camera imaging point and the right camera imaging point, and p represents the average pixel coordinate.

[0124] S203, converting the preliminary coordinates of the target into polar coordinates to obtain a rough positioning result of the target. The specific formula is:

[0125] A=atan(X C / Z);

[0126] S = Z / cos(A);

[0127] Wherein, A represents the first horizontal deflection angle of the target relative to the front direction of the laser radar; S represents the horizontal distance between the target and the laser radar.

[0128] Figure 7 This is the polar coordinate diagram of the target coarse positioning. It can be seen from the figure that the coarse positioning polar angle is stable at 0.546 radians, and the polar diameter fluctuates around 5.8m and the fluctuation is large. Therefore, the inaccurate calculated target depth affects the target coarse positioning accuracy, and it is necessary to combine the laser data in step S3 for precise positioning.

[0129] S3, subscribe to the laser radar data, perform time registration between the laser radar data and the left and right views, extract the target distance information in the laser array according to the rough positioning result of the target, filter the effective distance information through the median filter, use the abnormality assessment factor to perform spatial consistency check, if there is no matching point cloud, it is determined as recognition abnormality, and the threshold is recalibrated. If there is a matching point cloud, the target position after laser calibration is calculated; the specific content is:

[0130] S301, multi-sensor time registration: linearly interpolate the point cloud data in the lidar data to obtain new point cloud data with higher frequency. The specific formula is:

[0131] ;

[0132] Among them, P virtual represents new point cloud data; P0 represents the point cloud data of the previous frame; P1 represents the point cloud data of the current frame; t m Indicates inserting a new time. , t0 represents the time of the previous frame, and t1 represents the time of the current frame.

[0133] The new point cloud data and the corresponding ROS timestamp are stored in the scan array, and the ROS timestamp is compared with the timestamp corresponding to the target coarse positioning result, and the point cloud data closest to the target coarse positioning result is selected to complete the time alignment.

[0134] S302, the two-dimensional laser point cloud data is stored in a floating point array. The length of the array varies depending on the type of laser radar. The array stores the distance value measured by the two-dimensional laser radar at each scanning angle, which is used to describe the obstacle distribution in the surrounding environment. The array is defined, and the corresponding laser radar scanning index is calculated according to the first horizontal deflection angle of the target relative to the front direction of the laser radar. The specific formula is:

[0135] ;

[0136] Among them, zhong represents the index value of the lidar data matching A; represents the angle increment, , length represents the length of the array storing the new point cloud data.

[0137] S303. Since the target is usually a three-dimensional object, and the laser data corresponding to the calculated laser index value is a point, more points need to be collected for data processing to achieve precise positioning. Here, the index value and the five points before and after it are selected for scanning, and the measurement error of the distance S between the target and the laser radar is defined. The laser indexes with distances between S-error and S+error are recorded in a new array cun. The polar coordinates of the target relative to the laser radar are calculated based on the laser radar scanning index. The specific formula is:

[0138] ;

[0139] ;

[0140] ;

[0141] Wherein, number represents the laser radar target index value, median represents the median function, error represents the measurement error range of S set according to experience, N represents the laser radar index array after time alignment, i represents the laser index corresponding to the laser distance in the range of S-error and S+error, M represents the laser radar distance array after time alignment, B represents the second horizontal deflection angle of the target relative to the forward direction of the laser radar, and distance represents the target distance information in the laser radar distance array.

[0142] If the difference between distance and S is within 1m, the distance is normal, indicating that there is a matching point cloud; otherwise, the number of abnormalities is recorded and the abnormal situation is further judged. The specific formula is:

[0143] ;

[0144] in, represents the anomaly assessment factor, Q represents the number of times the distance anomaly occurs at least twice in a row, and k represents the total number of scans.

[0145] like Greater than the empirical threshold , indicating that there is no point cloud matching the target bounding box, confirming an abnormality and performing threshold recalibration; if Not greater than the empirical threshold , indicating that there is a point cloud that matches the target bounding box.

[0146] S304. When there is a matching point cloud, the polar coordinates obtained in step S303 are converted into rectangular coordinates to obtain the laser point cloud target positioning coordinates in the camera coordinate system. The specific formula is:

[0147] x1=distance*cos(B);

[0148] y1=distance*sin(B);

[0149] Among them, x1 represents the horizontal coordinate of the laser point cloud target positioning in the camera coordinate system, y1 represents the vertical coordinate of the laser point cloud target positioning in the camera coordinate system, cos represents the cosine function, and sin represents the sine function.

[0150] S305. Since the laser radar is usually installed on the robot, and the robot usually has posture changes, and since the laser radar sensor and the robot are solid-state connected, it is necessary to consider the problem caused by the misalignment of the coordinate system. The two-dimensional robot mainly considers the influence of the yaw angle, but the three-dimensional robot needs to fully consider the positioning errors caused by the pitch angle, roll angle and yaw angle. Considering the situation where the algorithm is deployed on a drone, the laser point cloud target positioning coordinates in the camera coordinate system are converted to the laser point cloud target positioning coordinates in the global coordinate system to obtain the target position after laser calibration. The specific formula is:

[0151] ;

[0152] Among them, X and Y represent the horizontal and vertical coordinates of the target in the global coordinate system; R represents the rotation matrix, , , , They represent the yaw angle, pitch angle, and roll angle of the robot respectively.

[0153] Figure 8 The target horizontal and vertical coordinates are calculated in combination with the laser data. It can be seen from the figure that the calculated target horizontal coordinate is stable at 5m, with an up and down fluctuation of less than 0.3m, and the target vertical coordinate is stable at 3m, with an up and down fluctuation of less than 0.15m. This shows that the positioning error of the target coarse positioning is greatly reduced by combining the laser data in step S3, and the target precise positioning is achieved.

[0154] S4. Use dual-mode confidence driving to optimize the traditional Kalman filter gain, dynamically update the target coordinate estimation value according to the optimized Kalman filter gain, and complete the target positioning; the specific content is:

[0155] S401, using the weight distribution mechanism to dynamically calculate the Kalman gain, using the dual-mode confidence drive to assist in calculating the Kalman gain, the calculation formula is:

[0156] ;

[0157] K_x=K adaptive *Eesti_x / (Emea_x+Eesti_x);

[0158] K_y=K adaptive*Eesti_y / (Emea_y+Eesti_y);

[0159] Among them, K adaptive represents the Kalman gain correction coefficient, C color Represents binocular vision color confidence, C laster Indicates the confidence level of laser calibration, Represents the visual-laser positioning difference attenuation coefficient, represents the Euclidean distance difference between binocular vision positioning and laser positioning, K_x represents the Kalman gain of the target horizontal coordinate, K_y represents the Kalman gain of the target vertical coordinate, Eesti_x represents the estimated error of the target horizontal coordinate, Eesti_y represents the estimated error of the target vertical coordinate, Emea_x represents the measurement error of the target horizontal coordinate, Emea_y represents the measurement error of the target vertical coordinate, represents the modal dominant factor.

[0160] S402, according to the Kalman gain, the predicted value and the measured value are continuously weighted averaged through iteration to obtain the optimal estimated value after filtering, which is the estimated value of the target coordinates. The specific formula is as follows:

[0161] ;

[0162] ;

[0163] in, Indicates The optimal target coordinate estimate after iterations is: Indicates The optimal target horizontal coordinate estimate after iterations is: Indicates The optimal target ordinate estimate after iterations is: Indicates The optimal target horizontal coordinate estimate after iterations.

[0164] S403, dynamically update the estimated error, the formula is as follows:

[0165] Eesti_xg=(1.0-K_x)*Eesti_x;

[0166] Eesti_yg=(1.0-K_y)*Eesti_y;

[0167] Among them, Eesti_xg represents the estimated error of the updated target horizontal coordinate, and Eesti_yg represents the estimated error of the updated target vertical coordinate.

[0168] S404. Based on the updated estimation error, repeat steps S401-S403 to complete iteration and realize dynamic optimization.

[0169] Threshold recalibration includes the following:

[0170] Convert the BGR image to HSV color space and extract the brightness channel. The specific formula is:

[0171] ;

[0172] ;

[0173] ;

[0174] in, represents the predicted brightness change, Represents the arithmetic mean of all pixels in the V channel, , All represent adjustable parameters. Represents the absolute value of the brightness gradient, Indicates the decrease value of the brightness threshold, represents the local contrast, Indicates the increase in brightness threshold, Indicates the average brightness of the brightest 5% area in the view.

[0175] Will and Linearly mapped into the range of 0-255 and , Indicates the pixel brightness threshold reduction amount, Indicates the pixel brightness threshold increase.

[0176] Calculate the arithmetic mean of all pixels in the brightness channel, take 128 as the base value, and normalize the result to the range of 0-2 to obtain the global brightness; when the global brightness is lower than 0.7, reduce the pixel brightness threshold To capture dark targets; when the global brightness is higher than 1.3, increase the pixel brightness threshold Avoid false alarms in overexposed areas.

[0177] Fig. 9 This is the target horizontal and vertical coordinate diagram calculated after Kalman filtering. It can be seen from the figure that the calculated target horizontal coordinate is stable at 4.967m, with an up and down fluctuation of no more than 0.002m, and the calculated target vertical coordinate is stable at 3.02m, with an up and down fluctuation of no more than 0.002m, indicating that the target positioning accuracy is further improved through the Kalman filtering in step S4.

[0178] Fig.10The horizontal and vertical coordinate error diagrams of target positioning performed 100 times using the method proposed in the present invention are shown. From the diagrams, it can be seen that the horizontal coordinate positioning error of the target proposed in the present invention is less than 0.946 cm, and the vertical coordinate positioning error of the target is less than 0.73 cm. Therefore, the method proposed in the present invention can achieve high-precision positioning of the target.

[0179] Fig.11 A schematic diagram showing the drone equipped with the laser radar and binocular camera required in the patent is shown. Fig.11 (a) is the main view of the drone after loading. Fig.11 (b) is a top view of the drone after being mounted. As can be seen from the figure, the laser radar and the binocular camera can be easily mounted on the drone, and the method proposed in the present invention can be deployed very conveniently.

[0180] The embodiment of the present invention also proposes a target positioning system based on dynamic color threshold and laser depth coordination, including a target center point pixel coordinate acquisition module, a target coarse positioning result acquisition module, a laser calibrated target position acquisition module, a target positioning module and a computer program that can be run on a processor. It should be noted that each module in the above system corresponds to the specific steps of the method provided in the embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not described in detail in this embodiment, please refer to the method provided in the embodiment of the present invention.

[0181] The embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. It should be noted that when the processor executes the computer program, the specific steps of the method provided in the embodiment of the present invention are corresponding to the specific steps of the method provided in the embodiment of the present invention, and the processor has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the method provided in the embodiment of the present invention.

[0182] The embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. It should be noted that when the computer program is executed by the processor, it corresponds to the specific steps of the method provided in the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the method provided in the embodiment of the present invention.

[0183] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A target positioning method based on dynamic color threshold and laser depth coordination, characterized in that: include: S1. The left and right views of the detection environment are collected by a binocular camera. Objects that meet the set threshold color and the set area and aspect ratio thresholds in the left and right views are taken as targets. The color features of the targets are extracted using the HSV color space. The target bounding box is extracted using the area / aspect ratio filtering analysis. The center coordinates of the target in the pixel coordinate system under the binocular camera are calculated. S2. Obtain the depth of the target according to the camera intrinsic parameter matrix. If the depth is abnormal, perform threshold recalibration. If the depth is normal, use matrix transformation to convert the center coordinates of the target in the pixel coordinate system into camera coordinates, convert the camera coordinates into polar coordinates, and use the polar coordinates as the rough positioning result of the target. S3, subscribe to the laser radar data, perform temporal registration of the laser radar data with the left and right views, extract the target distance information in the laser array according to the rough positioning result of the target, use the anomaly assessment factor to perform spatial consistency check, if there is no matching point cloud, perform threshold recalibration, if there is a matching point cloud, calculate the target position after laser calibration; S4. Use bimodal confidence driving to optimize the Kalman filter gain, dynamically update the target coordinate estimate according to the optimized Kalman filter gain, and complete target positioning.

2. The target positioning method based on dynamic color threshold and laser depth coordination according to claim 1 is characterized in that: In step S1, calculating the center coordinates of the target in the pixel coordinate system includes the following: S101, using ROS to obtain the left and right views of the detection environment collected by the binocular camera, using cv_bridge to convert the view into a BGR image of OpenCV, converting the BGR image into an HSV color space, using the three channels of hue, saturation, and brightness to separate the color features of the target, dynamically loading the set red, blue, green, brown, and white threshold ranges through the ROS parameter server, generating a corresponding binary mask, and completing the extraction of color features; S102, using an opening operation method of first corroding and then dilating to eliminate isolated noise points of the binary mask, and then superimposing the mask and performing a dilation operation to obtain a fused mask; S103, using findContours to detect the contour of the fused mask, retaining the outermost contour, traversing the outermost contour, and using a double filtering mechanism to remove contours that are smaller than an area threshold and are not within an aspect ratio threshold range, to obtain a target bounding box; S104, using the priority strategy of red, blue, green, brown, and white from high to low priority to classify the overlapping color areas of the target bounding box, obtain the corresponding color block color, and calculate the pixel coordinates of the center point of the target under the binocular camera. The specific formula is: left_xzhong=(left_self.x+left_self.x+left_self.w) / 2; left_yzhong=(left_self.y+left_self.y-left_self.h) / 2; right_xzhong=(right_self.x+right_self.x+right_self.w) / 2; right_yzhong=(right_self.y+right_self.y-right_self.h) / 2; Among them, left_xzhong and left_yzhong represent the horizontal and vertical coordinates of the center pixel of the target under the left camera respectively; right_xzhong and right_yzhong represent the horizontal and vertical coordinates of the center pixel of the target under the right camera respectively; left_self.x and left_self.y represent the horizontal and vertical coordinates of the upper left corner vertex of the target bounding box of the left camera respectively, left_self.w and left_self.h represent the width and height of the target bounding box of the left camera respectively, right_self.x and right_self.y represent the horizontal and vertical coordinates of the upper left corner vertex of the target bounding box of the right camera respectively, right_self.w and right_self.h represent the width and height of the target bounding box of the right camera respectively.

3. The target positioning method based on dynamic color threshold and laser depth coordination according to claim 1 is characterized in that: In step S2, the rough positioning result of the target is obtained, including the following contents: S201, according to the horizontal difference of the target center in the left and right views, combined with the baseline distance, the depth Z of the target is obtained. The specific formula is: ； Where b is the distance between the optical centers of the two cameras, f is the focal length of the camera, and X L Indicates the pixel coordinates of the target on the left camera, X R Indicates the pixel coordinates of the target on the right camera; If Z is not between 0.5m and 20m, it indicates that the depth is abnormal and the threshold should be recalibrated; if Z is between 0.5m and 20m, it indicates that the depth is normal; S202, when the depth is normal, the center point coordinates of the target under the binocular camera are converted from the pixel coordinate system to the camera coordinate system by using the principle of proportional magnification of similar triangles, and the preliminary coordinates of the target are obtained. The specific formula is: ； in, represents the mean value of the horizontal coordinate of the target pixel under the binocular camera, , left_xzhong and right_xzhong represent the horizontal coordinates of the center pixel of the target under the left and right cameras respectively; Represents the mean value of the ordinate of the target pixel under the binocular camera, , left_yzhong and right_yzhong represent the ordinates of the center pixels of the target under the left and right cameras respectively; X C Indicates the horizontal coordinate of the target in the camera coordinate system; Y C Indicates the target vertical coordinate in the camera coordinate system; Z C Indicates the vertical distance between the camera and the target plane, Z C =Z;f x 、f y Both represent the focal length of the camera; S203, converting the preliminary coordinates of the target into polar coordinates to obtain a rough positioning result of the target. The specific formula is: <h2 style=";text-align:left;direction:ltr">A=atan(X<h2 style=";text-align:left;direction:ltr"> C <h2 style=";text-align:left;direction:ltr"> / Z); S = Z / cos(A); Wherein, A represents the first horizontal deflection angle of the target relative to the front direction of the laser radar; S represents the horizontal distance between the target and the laser radar.

4. The target positioning method based on dynamic color threshold and laser depth coordination according to claim 1 is characterized in that: In step S3, obtaining the target position after laser calibration includes the following contents: S301, linearly interpolate the point cloud data in the laser radar data to obtain new point cloud data. The specific formula is: ； Among them, P virtual represents new point cloud data; P0 represents the point cloud data of the previous frame; P1 represents the point cloud data of the current frame; t m Indicates inserting a new time. , t0 represents the time of the previous frame, and t1 represents the time of the current frame; The new point cloud data and the corresponding ROS timestamp are stored in the scan array, and the ROS timestamp is compared with the timestamp corresponding to the target coarse positioning result, and the point cloud data closest to the target coarse positioning result is selected to complete the time alignment; S302, according to the first horizontal deflection angle of the target relative to the front direction of the laser radar, the corresponding laser radar scanning index is calculated, and the specific formula is: ； Where A represents the first horizontal deflection angle of the target relative to the front direction of the laser radar; zhong represents the laser radar data index value matching A; represents the angle increment, , length represents the length of the array storing the new point cloud data; S303, calculating the polar coordinates of the target relative to the laser radar according to the laser radar scanning index, the specific formula is: ； ； ； Wherein, number represents the laser radar target index value, median represents the median function, S represents the horizontal distance between the target and the laser radar, error represents the measurement error range of S set according to experience, N represents the laser radar index array after time registration, i represents the laser index corresponding to the laser distance in the range of S-error and S+error, M represents the laser radar distance array after time registration, B represents the second horizontal deflection angle of the target relative to the front direction of the laser radar, and distance represents the target distance information in the laser radar distance array; If the difference between distance and S is within 1m, the distance is normal, indicating that there is a matching point cloud; otherwise, the number of abnormalities is recorded and the abnormal situation is further judged. The specific formula is: ； in, represents the anomaly assessment factor, Q represents the number of times that distance anomalies occur at least twice in a row, and k represents the total number of scans; like Greater than the empirical threshold , indicating that there is no point cloud matching the target bounding box, confirming an abnormality and performing threshold recalibration; if Not greater than the empirical threshold , indicating that there is a point cloud that matches the target bounding box; S304. When there is a matching point cloud, the polar coordinates obtained in step S303 are converted into rectangular coordinates to obtain the laser point cloud target positioning coordinates in the camera coordinate system. The specific formula is: x1=distance*cos(B); y1=distance*sin(B); Among them, x1 represents the horizontal coordinate of the laser point cloud target positioning in the camera coordinate system, y1 represents the vertical coordinate of the laser point cloud target positioning in the camera coordinate system, cos represents the cosine function, and sin represents the sine function; S305, converting the laser point cloud target positioning coordinates in the camera coordinate system into the laser point cloud target positioning coordinates in the global coordinate system to obtain the target position after laser calibration. The specific formula is: ； Among them, X and Y represent the horizontal and vertical coordinates of the target in the global coordinate system; Z represents the depth of the target; R represents the rotation matrix, , , , They represent the yaw angle, pitch angle, and roll angle of the robot respectively.

5. The target positioning method based on dynamic color threshold and laser depth coordination according to claim 1 is characterized in that: In step S4, completing target positioning includes the following: S401, using the weight distribution mechanism to dynamically calculate the Kalman gain, using the dual-mode confidence drive to assist in calculating the Kalman gain, the calculation formula is: ； K_x=K adaptive *Estonia_x / (Emea_x+Estonia_x); K_y=K adaptive *Estonia_y / (Emea_y+Estonia_y); Among them, K adaptive represents the Kalman gain correction coefficient, C color Represents binocular vision color confidence, C laster Indicates the confidence level of laser calibration, Represents the visual-laser positioning difference attenuation coefficient, represents the Euclidean distance difference between binocular vision positioning and laser positioning, K_x represents the Kalman gain of the target horizontal coordinate, K_y represents the Kalman gain of the target vertical coordinate, Eesti_x represents the estimated error of the target horizontal coordinate, Eesti_y represents the estimated error of the target vertical coordinate, Emea_x represents the measurement error of the target horizontal coordinate, Emea_y represents the measurement error of the target vertical coordinate, represents the modal dominant factor; S402, according to the Kalman gain, the predicted value and the measured value are continuously weighted averaged through iteration to obtain the optimal estimated value after filtering, which is the estimated value of the target coordinates. The specific formula is as follows: ； ； in, Indicates The optimal target coordinate estimate after iterations is: Indicates The optimal target horizontal coordinate estimate after iterations is: Indicates The optimal target ordinate estimate after iterations is: Indicates The optimal target horizontal coordinate estimate after iteration, X, Y represent the target horizontal and vertical coordinates in the global coordinate system; S403, dynamically update the estimated error, the formula is as follows: Eesti_xg=(1.0-K_x)*Eesti_x; Eesti_yg=(1.0-K_y)*Eesti_y; Among them, Eesti_xg represents the estimated error of the updated target horizontal coordinate, and Eesti_yg represents the estimated error of the updated target vertical coordinate; S404. Based on the updated estimation error, repeat steps S401-S403 to complete iteration and realize dynamic optimization.

6. The target positioning method based on dynamic color threshold and laser depth coordination according to claim 2 is characterized in that: Threshold recalibration includes the following: Convert the BGR image to HSV color space and extract the brightness channel. The specific formula is: ； ； ； in, represents the predicted brightness change, Represents the arithmetic mean of all pixels in the V channel, , All represent adjustment parameters. Represents the absolute value of the brightness gradient, Indicates the decrease value of the brightness threshold, represents the local contrast, Indicates the increase in brightness threshold, Indicates the average brightness of the brightest 5% area of ​​the view; Will and Linearly mapped into the range of 0-255 and , Indicates the pixel brightness threshold reduction amount, Indicates the increase in pixel brightness threshold; Calculate the arithmetic mean of all pixels in the brightness channel and normalize the result to the range of 0-2 to obtain the global brightness; when the global brightness is lower than 0.7, reduce the pixel brightness threshold ; When the global brightness is higher than 1.3, increase the pixel brightness threshold .

7. A system for the target positioning method based on dynamic color threshold and laser depth coordination as claimed in claim 1, characterized in that: include: The target center pixel coordinate acquisition module is used to collect the left and right views of the detection environment through the binocular camera, take the objects that meet the set threshold color and the set area and aspect ratio thresholds in the left and right views as the target, use the HSV color space to extract the color features of the target, use the area / aspect ratio filtering analysis to extract the target bounding box, and calculate the center coordinates of the target in the pixel coordinate system under the binocular camera; The module for obtaining the rough positioning result of the target is used to obtain the depth of the target according to the camera intrinsic parameter matrix. If the depth is abnormal, the threshold is recalibrated. If the depth is normal, the center coordinates of the target in the pixel coordinate system are converted to camera coordinates by matrix transformation, and the camera coordinates are transformed into polar coordinates, which are used as the rough positioning result of the target. The target position acquisition module after laser calibration is used to subscribe to the laser radar data, perform time registration of the laser radar data with the left and right views, extract the target distance information in the laser array according to the rough positioning result of the target, use the anomaly assessment factor to perform spatial consistency verification, and perform threshold recalibration if there is no matching point cloud. If there is a matching point cloud, the target position after laser calibration is calculated; The target positioning module is used to optimize the Kalman filter gain using dual-modal confidence driving, dynamically update the target coordinate estimation value according to the optimized Kalman filter gain, and complete the target positioning.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the target positioning method based on dynamic color threshold and laser depth coordination as described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the target positioning method based on dynamic color threshold and laser depth coordination described in any one of claims 1 to 6 is executed.

Images