Park unmanned light truck positioning method, system and device based on visual perception and medium

Abstract

The invention provides a park unmanned light truck positioning method, system and device based on visual perception and a medium, and belongs to the technical field of light truck unmanned positioning, and the method comprises the steps: installing a multi-view vision system at the front part of an unmanned light truck, so as to collect a visual image of a park scene in real time; feature points in the visual image are extracted, and park scene objects in the visual image are identified and classified in combination with a pre-trained deep learning model; based on the extracted feature points and a park scene object identification result, calculating depth information of a park scene object, and generating a local sensing map of the position of the unmanned light truck; matching the local sensing map with a pre-constructed high-precision map, and determining the current position of the unmanned light truck; and correcting the current position in real time by adopting a Kalman filtering algorithm to obtain an optimized positioning result. The method provides high-precision and high-robustness positioning, adapts to a complex park environment, and improves the operation safety and efficiency of the unmanned light truck.

Description

Technical Field

[0001] This application belongs to the field of positioning technology for unmanned light trucks, specifically involving a positioning method, system, equipment, and medium for unmanned light trucks in industrial parks based on visual perception. Background Technology

[0002] With the widespread application of autonomous driving technology in fields such as park logistics and sanitation, autonomous vehicles based on light truck chassis have become important transportation tools due to their large cargo capacity and strong adaptability. However, achieving accurate positioning is crucial to ensuring the safe and efficient operation of these vehicles. Existing positioning technologies mainly rely on the Global Navigation Satellite System (GNSS) and the Inertial Navigation System (INS). GNSS can provide relatively accurate position information in open areas, but in park environments, due to issues such as building obstruction, multipath effects, and signal interference, GNSS signals are prone to attenuation or loss of lock, leading to decreased positioning accuracy or even complete failure. Although INS can provide relatively accurate attitude and position information in a short period of time, its errors accumulate with time and travel distance, making it difficult to meet the requirements for long-term, high-precision positioning. In addition, for autonomous vehicles based on light truck chassis, their own structure is relatively heavy and has greater inertia, which places higher demands on the real-time performance and accuracy of the positioning system.

[0003] To overcome these problems, existing technologies attempt to use visual perception-assisted localization, such as using monocular or binocular cameras for environmental feature extraction and matching. However, in complex park scenarios, a single visual sensor is susceptible to changes in lighting, occlusion, and interference from dynamic objects, resulting in insufficient stability of feature matching and a lack of effective fusion with high-precision maps, leading to large fluctuations and poor robustness in localization results.

[0004] Therefore, there is an urgent need for a positioning method that can integrate multi-source sensing data, achieve centimeter-level accuracy, and adapt to the dynamic environment of the park. Summary of the Invention

[0005] In a first aspect, embodiments of this application provide a method for locating unmanned light trucks in a park based on visual perception, comprising the following steps: S1. Install a multi-view vision system at the front of the driverless light truck to collect visual images of the park scene in real time; S2. Use computer vision algorithms to extract feature points from the visual image, and combine a pre-trained deep learning model to identify and classify objects in the park scene in the visual image; S3. Based on the extracted feature points and the object recognition results of the park scene, use stereo vision algorithms to calculate the depth information of the objects in the park scene and generate a local perception map of the location of the unmanned light truck. S4. Match the local perception map with the pre-built high-precision map to determine the current location of the unmanned light truck; S5. The current position is corrected in real time using the Kalman filter algorithm to obtain the optimized positioning result.

[0006] Furthermore, the specific steps of step S1 are as follows: S11. Deploy at least two wide-angle cameras at the front of the driverless light truck to form a multi-view vision system, and the baseline distance between each wide-angle camera is a fixed value; S12. Control all wide-angle cameras by synchronizing the clock to capture visual images of the park scene from a preset number of perspectives at the same time. S13. Perform denoising and distortion correction preprocessing on the acquired raw visual images; The denoising algorithm uses a Gaussian filtering algorithm, and the convolution kernel function is:

[0007] In the formula, Let σ be the pixel coordinates, and σ be the standard deviation of the Gaussian distribution. Distortion correction uses the Brownian distortion algorithm. The formulas for radial and tangential distortion correction are as follows:

[0008]

[0009] In the formula, The coordinates of the original visual image, For the corrected coordinates, , , The radial distortion coefficient is... , The tangential distortion coefficient is... .

[0010] Furthermore, the specific steps of step S2 are as follows: S21. The ORB algorithm is used to extract the first key feature point from the preprocessed visual image, and the descriptor of the first key feature point is calculated. Specifically, the ORB algorithm first uses the FAST corner detector to locate the first key feature point, then uses the intensity centroid method to assign an orientation to each first key feature point, and finally uses the rBRIEF descriptor generation algorithm to calculate the binary string form descriptor for each first key feature point. S22. Input the visual image into the pre-trained YOLOv5 model to identify and classify objects in the park scene, and output the category label of each object in the park scene and the bounding box coordinates of each object in the visual image. The YOLOv5 model consists of three parts: the main network, the neck network, and the head. The training loss function is composed of localization loss, confidence loss, and classification loss. The YOLOv5 model was obtained through supervised training on a labeled image dataset containing a set number of park scene elements. S23. Determine the spatial location of the extracted first key feature point, and associate the first key feature point with the bounding box coordinates of each park scene object in the visual image based on the spatial location; If the first key feature point is located within the bounding box of an object in a certain park scene, then the corresponding first key feature point is assigned the category label of the corresponding park scene object, and a second key feature point with semantic information is generated. All second key feature points constitute an enhanced feature point set.

[0011] Furthermore, the specific steps of step S3 are as follows: S31. Based on the SGBM algorithm, using the enhanced feature point set as a guide, the search range for matching the second key feature point is limited to the bounding box of the park scene object in the visual image and the spatial structure region between adjacent park scene objects, so as to calculate the disparity d of each second key feature point in the enhanced feature point set in the left and right views. S32. Based on the principle of triangulation, and using the disparity d, calculate the physical depth value corresponding to each second key feature point. :

[0012] in, This represents the depth distance of the second key feature point relative to the wide-angle camera. This indicates the baseline distance of the wide-angle camera in a multi-view vision system. Indicates the focal length of a wide-angle camera; S33. Integrate the two-dimensional image coordinates, semantic information, and physical depth values ​​of all second key feature points. By transforming the coordinates of the points in the image coordinate system and corresponding depth values Transformed to 3D points in camera coordinate system And based on semantic information, the three-dimensional points Clustering is performed to generate point cloud clusters based on semantic objects, and a local perception map containing 3D point clouds with semantic labels is constructed.

[0013] Furthermore, the specific steps of step S31 are as follows: S311. Calculate the cost of all possible parallaxes within the parallax range for each pixel, and construct a cost cube; S312. The cost value is smoothed in several predetermined directions by one-dimensional cost aggregation of multiple paths; S313. For each second key feature point in the enhanced feature point set, select the disparity value pair with the minimum aggregation cost corresponding to the location of the second key feature point as the reliable disparity d of the corresponding second key feature point, and perform regional voting on the disparity results of all second key feature points to generate a sparse disparity map. S314. For second key feature points belonging to the same semantic category, if the deviation between the disparity value and the median disparity value within the corresponding semantic category group exceeds a preset threshold, the corresponding second key feature point is determined to be a mismatch point and is removed.

[0014] Furthermore, the specific steps of step S4 are as follows: S41. Call up a high-precision map of the park that has been pre-built using LiDAR, the high-precision map of the park containing three-dimensional point clouds and feature description information of roads, factories and landmarks with centimeter-level precision; S42. Extract the second key feature point in the local perception map and match it with the feature point descriptor in the high-precision map. Use Hamming distance as a similarity measure to find the correspondence between the two sets of data. S43. Based on the successfully matched feature point pairs, the transformation matrix is ​​estimated using the random sampling consensus algorithm. By aligning and fusing the local perception map with the high-precision map, the current position of the autonomous light truck in the global coordinate system can be calculated. and the current position Output as observed values.

[0015] Furthermore, the specific steps of step S5 are as follows: S51. Based on the optimized state vector of the unmanned light truck at time k-1. The prediction equation using Kalman filtering is used to predict the prior state estimate at time k. and prior estimation of covariance matrix The prediction equation is: ; ; in, The state prior estimate vector at time k includes position and velocity; A is the prior estimate of the covariance matrix; A is the state transition matrix; B is the posterior state estimation vector at time k-1; B is the control input matrix. Let Q be the control vector at time k-1; Q is the process noise covariance matrix. S52. Solve for the current position As observed value Based on the prediction results, the Kalman gain is calculated. And update the state estimate and covariance, with the update equation being:

[0016]

[0017]

[0018] Where H is the observation matrix, R is the observation noise covariance matrix, and I is the identity matrix; S53. Update the state vector The location information in the data is output as the final positioning result after optimization at time k.

[0019] Secondly, embodiments of this application also provide a vision-based positioning system for unmanned light trucks in industrial parks, comprising: The data acquisition module is used to install a multi-view vision system at the front of the driverless light truck to acquire visual images of the park scene in real time. The visual processing module is used to extract feature points from the visual image using computer vision algorithms, and to identify and classify objects in the park scene in the visual image by combining a pre-trained deep learning model. The local map construction module is used to calculate the depth information of objects in the park scene based on the extracted feature points and the object recognition results of the park scene, and generate a local perception map of the location of the unmanned light truck. The global positioning module is used to match the local perception map with a pre-built high-precision map to determine the current location of the unmanned light truck. The filtering optimization module is used to perform real-time correction on the current position using the Kalman filtering algorithm to obtain an optimized positioning result.

[0020] Thirdly, embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the vision-based unmanned light truck positioning method for parks as described in the first aspect.

[0021] Fourthly, embodiments of this application also provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the vision-based perception-based unmanned light truck positioning method for industrial parks as described in the first aspect.

[0022] As can be seen from the above technical solutions, this application has the following advantages: The visual perception-based positioning method, system, equipment, and medium for unmanned light trucks in industrial parks provided in this application reduce reliance on GNSS signals by utilizing visual perception, thereby mitigating the impact of building occlusion and multipath effects and enhancing adaptability and stability in complex environments. Real-time correction ensures rapid updates to positioning results, meeting the real-time requirements of unmanned light trucks in dynamic environments. Deep learning models are used to identify scene objects and assign semantic information, enabling the positioning system not only to perceive the environment but also to understand it, thus improving its intelligence level. This application achieves accurate positioning in complex industrial park environments, improving the positioning accuracy of unmanned light trucks. Attached Figure Description

[0023] To more clearly illustrate the technical solution of this application, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating the vision-based positioning method for unmanned light trucks in industrial parks according to the present invention.

[0025] Figure 2 This is a schematic diagram of the vision-based positioning system for unmanned light trucks in industrial parks according to the present invention. Detailed Implementation

[0026] The various embodiments of this disclosure will be described more fully in the following detailed steps of the vision-based autonomous light truck positioning method for industrial parks. This disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of this disclosure to the specific embodiments disclosed herein, but rather this disclosure should be understood to cover all adjustments, equivalents, and / or alternatives falling within the spirit and scope of the various embodiments of this disclosure.

[0027] This embodiment provides a vision-based positioning method for unmanned light trucks in industrial parks. By combining a multi-view vision system, a deep learning model, a stereo vision algorithm, and a Kalman filter algorithm, it achieves centimeter-level precise positioning in complex park environments, thereby improving the positioning accuracy of unmanned light trucks.

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] Please see Figure 1 The diagram shows a flowchart of a vision-based positioning method for unmanned light trucks in a park, which includes the following steps: S1. Install a multi-view vision system at the front of the driverless light truck to collect visual images of the park scene in real time; It should be noted that by installing a multi-view vision system at the front of the driverless light truck, it is possible to collect multi-view visual images of the park scene in real time, providing a data foundation for feature extraction and environmental perception, and solving the problem of insufficient information from a single vision sensor. S2. Use computer vision algorithms to extract feature points from the visual image, and combine a pre-trained deep learning model to identify and classify objects in the park scene in the visual image; It should be noted that by combining computer vision algorithms and deep learning models, feature points in visual images are extracted and scene objects are identified, and semantic information is given to the feature points. This not only improves the accuracy and efficiency of feature extraction, but also enhances the system's ability to understand the environment, providing an important basis for depth information calculation and map construction. S3. Based on the extracted feature points and the object recognition results of the park scene, use stereo vision algorithms to calculate the depth information of the objects in the park scene and generate a local perception map of the location of the unmanned light truck. It should be noted that by calculating the depth information of objects in the scene through stereo vision algorithms and generating a local perception map with semantic labels, the transformation from two-dimensional image to three-dimensional space is realized, providing a high-precision local map for subsequent map matching and improving the accuracy and robustness of positioning. S4. Match the local perception map with the pre-built high-precision map to determine the current location of the unmanned light truck; It should be noted that by matching the local perception map with the high-precision map, the current position of the autonomous light truck is determined, which solves the problem of fusion between local perception and global positioning, achieves high-precision global positioning, and improves positioning capability and adaptability. S5. The current position is corrected in real time using the Kalman filter algorithm to obtain the optimized positioning result; It should be noted that the Kalman filtering algorithm is used to perform real-time correction on the current position. By combining prediction and observation updates, the positioning result is optimized, error accumulation is reduced, real-time performance and stability are improved, and the positioning accuracy of the driverless light truck in a dynamic environment is ensured.

[0030] In this embodiment, a multi-camera vision system is installed at the front of the light truck to collect images. Computer vision algorithms and deep learning models are used to extract feature points and identify scene objects. The stereo vision algorithm is combined to calculate depth information to generate a local perception map, which is then matched with a high-precision map to determine the position. Finally, the Kalman filter is used to optimize the positioning result, achieving high-precision and high-robustness positioning, reducing the dependence on GNSS, adapting to complex park environments, meeting the dynamic positioning requirements of driverless light trucks, and improving operation safety and efficiency.

[0031] Furthermore, as a refinement and extension of the specific implementation method of the above embodiment, in order to fully illustrate the specific implementation process in this embodiment, taking the driverless light truck transportation scenario in an industrial park (covering an area of about 500,000 square meters, including 12 factory buildings, 3 main roads, 8 cargo loading and unloading points, and several green isolation belts) as an example, the hardware selection of the multi-camera vision system is as follows: Three Hikvision MV-CA032-10GM Gigabit Ethernet wide-angle industrial cameras are selected as the cameras, with a resolution of 3200×2400, a lens focal length of 8mm, and a horizontal field of view angle of 65°, meeting the coverage requirements of short-distance (0-50m) scenarios in the park; The installation method is to fix the 3 cameras on the front bracket of the light truck cab, arranged in a "pin" shape. The baseline distance between the main camera (in the middle) and the left and right sub-cameras is set to 0.5m (a fixed value). The optical axis of the camera is consistent with the forward direction of the light truck, and the installation height is 2.2m from the ground to avoid perspective deviation caused by chassis vibration; For synchronous control, a certain model of synchronous clock module is used to uniformly set the acquisition frame rate of the 3 cameras to 15fps, ensuring that 3-channel perspective images (main perspective, 15° left-offset perspective, 15° right-offset perspective) are acquired at the same time point (error ≤ 1ms), covering a 120° field of view in front of the light truck; The high-precision map is pre-constructed as follows: A certain model of lidar (for example, the point cloud density is 200 points / ㎡, and the ranging accuracy is ±2cm) is used, combined with GNSS (Beidou + GPS dual-mode, static positioning accuracy ±5cm) and IMU (inertial measurement unit, sampling rate 200Hz) to perform a full-area scan of the park; This method generates a centimeter-level high-precision map containing information such as roads (e.g., lane width 0.15m, road edge accuracy ±3cm), factory corners (coordinate accuracy ±2cm), cargo loading / unloading point signs (e.g., diameter 0.8m, center point coordinate accuracy ±1cm), and streetlights (e.g., spacing 20m, pole coordinate accuracy ±4cm). The map is stored in PCD (point cloud data) + JSON (feature description information, including object category labels and coordinate ranges). It provides another vision-based method for locating unmanned light trucks in industrial parks, which includes the following steps: S1. Install a multi-view vision system at the front of the unmanned light truck to collect visual images of the park scene in real time; the specific steps of step S1 are as follows: S11. Deploy at least two wide-angle cameras at the front of the driverless light truck to form a multi-view vision system, and the baseline distance between each wide-angle camera is a fixed value; S12. Control all wide-angle cameras by synchronizing the clock to capture visual images of the park scene from a preset number of perspectives at the same time. For example, a light truck travels along the main road of the park from factory A (starting point) to loading / unloading point B (end point, distance 3.2km). The synchronous clock module triggers three cameras to collect scene images in real time. For example, when the truck reaches the intersection between factory 3 and factory 4, it collects three images: the main view image includes the stop and yield sign and the corner of the wall of factory 4 on the right; the left view image includes the edge of the green isolation belt on the left; and the right view image includes the speed limit sign (5km / h) at the intersection. S13. Perform denoising and distortion correction preprocessing on the acquired raw visual images; The denoising algorithm uses a Gaussian filtering algorithm, and the convolution kernel function is:

[0032] In the formula, Let σ be the pixel coordinates, and σ be the standard deviation of the Gaussian distribution. Distortion correction uses the Brownian distortion algorithm. The formulas for radial and tangential distortion correction are as follows:

[0033]

[0034] In the formula, The coordinates of the original visual image, For the corrected coordinates, , , The radial distortion coefficient is... , The tangential distortion coefficient is... ; For example, a Gaussian filtering algorithm was used, with a standard deviation σ=1.5 and a convolution kernel size of 3×3, to filter the acquired original image (which contained slight salt-and-pepper noise, originating from dust interference in the industrial park). Taking the four corner areas of the factory building in the main view image as an example, before filtering, the gray value fluctuation range of this area was 45-120, and after filtering, the fluctuation range was reduced to 60-105, and the edge contour clarity was improved by 30%. Distortion correction: The Brownian distortion algorithm is used, and the radial distortion coefficient is obtained through prior camera calibration (e.g., using Zhang Zhengyou calibration method, with a calibration board of 12×9 checkerboard and 20mm squares). =-0.032、 =0.015、 =-0.002, tangential distortion coefficient =0.001、 =-0.0005; Correct the distorted area (original coordinates (x=850, y=620)) at the edge of the greenbelt in the left-view image, and calculate the corrected coordinates ( =848, =617), eliminating edge curvature, reducing the straightness deviation between the isolation zone lines and the corresponding area in the high-precision map from 1.2° to 0.3°; S2. Use computer vision algorithms to extract feature points from the visual image, and combine a pre-trained deep learning model to identify and classify objects in the park scene within the visual image; the specific steps of step S2 are as follows: S21. The ORB algorithm is used to extract the first key feature point from the preprocessed visual image, and the descriptor of the first key feature point is calculated. Specifically, the ORB algorithm first uses the FAST corner detector to locate the first key feature point, then uses the intensity centroid method to assign an orientation to each first key feature point, and finally uses the rBRIEF descriptor generation algorithm to calculate the binary string form descriptor for each first key feature point. For example, the ORB algorithm is applied to the preprocessed main view image. The FAST corner detector is set to a threshold of 30, and a total of 286 first key feature points are extracted. Among them, 15 feature points are extracted from the stop-and-yield sign area at the intersection, 25 feature points are extracted from the four corners of the factory (i.e., the sign with the number 4-2), and 48 feature points are extracted from the road lane lines (including the right-hand drive arrow). The direction of each feature point is assigned by the intensity centroid method (e.g., the direction of the corner sign feature point is vertical, and the direction of the lane line arrow feature point is along the driving direction). Then, a 64-bit binary descriptor is generated by the rBRIEF algorithm. For example, the descriptor of the top triangle vertex of the stop-and-yield sign is "10110010...01101001" (64 bits in total). S22. Input the visual image into the pre-trained YOLOv5 model to identify and classify objects in the park scene, and output the category label of each object in the park scene and the bounding box coordinates of each object in the visual image. The YOLOv5 model consists of three parts: the main network, the neck network, and the head. The training loss function is composed of localization loss, confidence loss, and classification loss. The YOLOv5 model was obtained through supervised training on a labeled image dataset containing a set number of park scene elements. For example, three preprocessed images are input into a pre-trained YOLOv5 model (the model is trained on 100,000 labeled images of the park, with labeled object categories including factory buildings (including wall number markings), stop signs, loading / unloading point number signs (e.g., loading / unloading point-3), speed limit signs (uniform 5km / h within the park), roadside reflective posts, and other freight vehicles, totaling six categories. The training loss function has a localization loss weight of 0.3, a confidence loss weight of 0.4, and a classification loss weight of 0.3. Taking the main view image as an example, the model identifies four typical park objects, with no traffic light-related recognition results, which conforms to the actual traffic rules of the park). Category of parking yield sign: bounding box coordinates ( ), with a confidence level of 0.98 (the sign is an inverted triangle with a red background and white border, containing the words "Stop and Yield" in bold font inside, and is a core traffic control sign at the intersection of the park). Category of factory building (with number) (corner marker of factory building 4-2): bounding box coordinates ( ), confidence level 0.96 (the sign is a blue rectangle with white lettering, 0.6m long and 0.3m wide, fixed at a height of 1.5m on the outer wall of the factory building, and is a fixed landmark of the park). Category speed limit sign: bounding box coordinates ( ), confidence level 0.95 (circular mark, red border and white background, the number 5 inside represents a speed limit of 5km / h); Category of road edge reflector: bounding box coordinates ( ), confidence level 0.97 (cylindrical, 1.2m high, 0.1m in diameter, with reflective film on the surface, arranged at 5m intervals along the edge of the main road to define the road area). S23. Determine the spatial location of the extracted first key feature point, and associate the first key feature point with the bounding box coordinates of each park scene object in the visual image based on the spatial location; If the first key feature point is located within the bounding box of an object in a certain park scene, then the corresponding first key feature point is assigned the category label of the corresponding park scene object, and a second key feature point with semantic information is generated. All second key feature points constitute an enhanced feature point set. For example, the spatial location of the first key feature points is determined. For instance, all 15 feature points of the stop sign area are located within its bounding box, and it is assigned the category label "stop sign". The 25 feature points of the corner sign of factory 4-2 are located within the bounding box of the factory (with number), and it is assigned the label "factory-4-2". The 8 feature points of the road edge reflector are located within their corresponding bounding boxes, and it is assigned the label "reflector". Finally, 272 second key feature points are generated (only 14 isolated feature points without corresponding objects are removed, 4 more than the original instance are retained because the reflector feature points are more dense), forming an enhanced feature point set. S3. Based on the extracted feature points and the object recognition results of the park scene, use a stereo vision algorithm to calculate the depth information of the objects in the park scene and generate a local perception map of the location of the unmanned light truck; the specific steps of step S3 are as follows: S31. Based on the SGBM algorithm, guided by the enhanced feature point set, the search range for matching the second key feature points is limited to the bounding boxes of objects in the park scene in the visual image and the spatial structure region between adjacent park scene objects, so as to calculate the disparity d of each second key feature point in the enhanced feature point set in the left and right views; the specific steps of step S31 are as follows: S311. Calculate the cost of all possible parallaxes within the parallax range for each pixel, and construct a cost cube; S312. The cost value is smoothed in several predetermined directions by one-dimensional cost aggregation of multiple paths; S313. For each second key feature point in the enhanced feature point set, select the disparity value pair with the minimum aggregation cost corresponding to the location of the second key feature point as the reliable disparity d of the corresponding second key feature point, and perform regional voting on the disparity results of all second key feature points to generate a sparse disparity map. S314. For second key feature points belonging to the same semantic category, if the deviation between the disparity value and the median disparity value within the corresponding semantic category group exceeds a preset threshold, the corresponding second key feature point is determined to be a mismatch point and is removed. For example, based on the SGBM algorithm, guided by an enhanced feature point set, the feature point matching search range is limited to the bounding box of the corresponding object (e.g., feature points of a stop sign are only searched for within its inverted triangle bounding box to avoid confusion with feature points of adjacent reflectors). Specifically, the following steps are performed: Construct the cost cube: For each pixel in the stop sign area in the main view and left view images, calculate the cost within the disparity range (0-60 pixels) to generate a cost cube of 3200×2400×60. Cost aggregation: One-dimensional cost aggregation is performed along four directions: horizontal, vertical, 45°, and 135° to smooth cost fluctuations (e.g., after aggregation of the red background area of ​​the stop sign, the standard deviation of the post-value decreases from 8 to 3, avoiding parallax misjudgment caused by color interference). Sparse disparity map generation: Select the minimum aggregation cost disparity value of each second key feature point. For example, the disparity d of the top triangle vertex of the stop sign is 12 pixels. Perform regional voting on the disparity results of all feature points (feature point disparity deviation ≤ 3 pixels within the same object is valid) to generate a sparse disparity map containing 272 valid disparities. Mismatch removal: For the 25 feature points in the Factory-4-2 sign category, the median disparity was calculated to be 18 pixels. A threshold of ±3 pixels was set, and one mismatch point with a disparity of 22 pixels was removed (because this point was close to the shadow area of ​​the sign edge). Finally, 24 valid feature points were retained. For the 8 feature points in the Reflector category, the median disparity was 25 pixels. There were no points exceeding the threshold, so all were retained. S32. Based on the principle of triangulation, and using the disparity d, calculate the physical depth value corresponding to each second key feature point. :

[0035] in, This represents the depth distance of the second key feature point relative to the wide-angle camera. This indicates the baseline distance of the wide-angle camera in a multi-view vision system. Indicates the focal length of a wide-angle camera; For example, based on the principle of triangulation, the camera baseline distance B = 0.5m and the focal length f = 8mm (converted to pixel units: (pixels) to calculate depth for two types of core object feature points: The top triangle of the stop sign (d=12 pixels):

[0036] That is, the distance between the vertex and the camera is 1.6m, which is only 1.26% different from the actual manual measurement (1.58m), and meets the installation specification of 1.5m between the intersection sign and the road edge in the park. Top of reflector pillars at the edge of the road (d=25 pixels): ; S33. Integrate the two-dimensional image coordinates, semantic information, and physical depth values ​​of all second key feature points. By transforming the coordinates of the points in the image coordinate system and corresponding depth values Transformed to 3D points in camera coordinate system And based on semantic information, the three-dimensional points Clustering is performed to generate point cloud clusters based on semantic objects, and a local perception map containing 3D point clouds with semantic labels is constructed. For example, by fusing the two-dimensional coordinates of 271 valid second key feature points (such as the top vertex of the stop sign (1285, 450)), semantic labels (stop sign), and depth values ​​(1.6m), and through coordinate transformation (converting the image coordinate system to the camera coordinate system, with the origin of the camera coordinate system at the optical center of the main camera, the X-axis along the forward direction, the Y-axis perpendicular to the ground and upward, and the Z-axis horizontal to the right), the three-dimensional coordinates of the vertex are obtained (X=1.6×cos(0°)=1.6m, Y=0.8m (sign installation height), Z=0.15m). Based on semantic label clustering, four point cloud clusters are generated for the stop sign, factory-4-2, speed limit sign, and reflective pillars (one more reflective pillar cluster than the original example), constructing a local perception map (covering an area of ​​5m×8m) with a point cloud density of 38 points / m. 2 (Due to the addition of reflective column feature points, the density is increased by 8.6%), and the semantic label accuracy is 100%, perfectly matching the actual scenario of the park without traffic lights. S4. Match the local perception map with the pre-constructed high-precision map to determine the current location of the unmanned light truck; the specific steps of step S4 are as follows: S41. Call up a high-precision map of the park that has been pre-built using LiDAR, the high-precision map of the park containing three-dimensional point clouds and feature description information of roads, factories and landmarks with centimeter-level precision; For example, when the light truck travels to the intersection between factory 3 and factory 4, it calls a pre-built high-precision map of the area to obtain centimeter-level feature information of the core landmarks in the area: Parking yield sign: 3D coordinates (X=256.32m, Y=0.8m, Z=128.45m) (Y is the installation height, consistent with the actual height); Corner sign of factory building 4-2: 3D coordinates (X=258.15m, Y=1.5m, Z=126.78m) (Y is the height of the sign from the ground); Roadside reflective post (No. C-12): 3D coordinates (X=257.90m, Y=0.6m, Z=129.20m) (Y is the half-height position of the reflective post, for easy feature matching); S42. Extract the second key feature point in the local perception map and match it with the feature point descriptor in the high-precision map. Use Hamming distance as a similarity measure to find the correspondence between the two sets of data. For example, feature point descriptors for stop signs, factory-4-2, and reflective pillars in the local perception map are extracted and matched with feature point descriptors of the corresponding categories in the high-precision map. The similarity is measured using Hamming distance (Hamming distance ≤ 10 indicates a successful match). For instance, the descriptor of the top vertex of the stop sign has a Hamming distance of 8 with the descriptor of the stop sign at intersection C3-C4 (intersection of factory 3-4) in the high-precision map, indicating a successful match. Seven out of the eight feature points of reflective pillar C-12 are successfully matched (Hamming distance 5-9). A total of 35 pairs of feature points are successfully matched (3 more pairs than the original example, due to the supplementation of reflective pillar features), with a matching success rate of 94.6% (an improvement of 1.8%). S43. Based on the successfully matched feature point pairs, the transformation matrix is ​​estimated using the random sampling consensus algorithm. By aligning and fusing the local perception map with the high-precision map, the current position of the autonomous light truck in the global coordinate system can be calculated. and the current position Output as observed values; For example, based on 35 pairs of matching feature points, the RANSAC algorithm (1000 iterations, 2cm inlier threshold) is used to estimate the transformation matrix. The local perception map is aligned to the high-precision map, and the coordinates of the light truck's current position in the global coordinate system are calculated as (X=257.23m, Y=0.5m, Z=127.61m). The deviation from the GNSS (unobstructed) positioning result (X=257.21m, Y=0.5m, Z=127.63m) is only 0.03m, which meets the centimeter-level positioning requirement. Moreover, the matched objects are all fixed landmarks that actually exist in the park, and there is no logical contradiction. S5. The current position is corrected in real time using the Kalman filter algorithm to obtain the optimized positioning result; the specific steps of step S5 are as follows: S51. Based on the optimized state vector of the unmanned light truck at time k-1. The prediction equation using Kalman filtering is used to predict the prior state estimate at time k. and prior estimation of covariance matrix The prediction equation is: ; ; in, The state prior estimate vector at time k includes position and velocity; A is the prior estimate of the covariance matrix; A is the state transition matrix; B is the posterior state estimate vector at time k-1 (i.e., the optimized state at the previous time step); B is the control input matrix. is the control vector at time k-1 (derived from wheel speed sensor information provided by the vehicle's CAN bus); Q is the process noise covariance matrix. For example, the optimized state vector of a light truck at time k-1 (0.067s ago) is known. (No motion in the Y direction, velocity is 0), state transition matrix (T=0.067s is the sampling interval), control input matrix Control vector at time k-1 (The light truck is traveling at a constant speed, and has slowed down to the speed limit of 5 km / h as it approaches an intersection, with an acceleration of 0). The process noise covariance matrix is... Substitute into the prediction equation:

[0037] Let the posterior covariance matrix at time k-1 be denoted as . Calculated ; S52. Solve for the current position As observed value Based on the prediction results, the Kalman gain is calculated. And update the state estimate and covariance, with the update equation being:

[0038]

[0039]

[0040] Where H is the observation matrix, R is the observation noise covariance matrix, and I is the identity matrix; For example, the current position (X=257.23m, Z=127.61m) calculated in step S4 is used as the observation value. (This location is calculated based on matching three types of landmarks: "stop sign + factory building-4-2 + reflective post," providing an additional reference compared to the original example, resulting in stronger observation stability.) Observation Matrix Observation noise covariance matrix (based on historical observation error statistics; observation noise is reduced by 10% compared to the original instance due to the addition of a new type of reference marker); calculate Kalman gain: ; Substituting into the update equation, we obtain the posterior state vector at time k. posterior covariance matrix (Covariance is reduced by 5% compared to the original instance, and state estimation is more stable); S53. Update the state vector The location information in the data is output as the final positioning result after optimization at time k. For example, The location information (X=257.25m, Z=127.63m) in the data, as the optimized positioning result at time k, has a deviation of only 0.014m from the actual manually measured position of the light truck (X=257.24m, Z=127.62m), which is 53.3% lower than the positioning deviation before correction (0.03m). Moreover, because the matching markers are more in line with the actual situation in the park, the on-site verifiability of the positioning result is stronger (for example, a person can directly check whether the light truck is 1.5m in front of the stop and yield sign).

[0041] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0042] like Figure 2 As shown, the following are embodiments of the visual perception-based unmanned light truck positioning system for industrial parks provided in this disclosure. This system and the visual perception-based unmanned light truck positioning methods for industrial parks described above belong to the same inventive concept. For details not described in detail in the embodiments of the visual perception-based unmanned light truck positioning system for industrial parks, please refer to the embodiments of the visual perception-based unmanned light truck positioning methods for industrial parks described above.

[0043] The system includes: The data acquisition module is used to install a multi-view vision system at the front of the driverless light truck to acquire visual images of the park scene in real time. The visual processing module is used to extract feature points from the visual image using computer vision algorithms, and to identify and classify objects in the park scene in the visual image by combining a pre-trained deep learning model. The local map construction module is used to calculate the depth information of objects in the park scene based on the extracted feature points and the object recognition results of the park scene, and generate a local perception map of the location of the unmanned light truck. The global positioning module is used to match the local perception map with a pre-built high-precision map to determine the current location of the unmanned light truck. The filtering optimization module is used to perform real-time correction on the current position using the Kalman filtering algorithm to obtain an optimized positioning result.

[0044] This embodiment achieves high-precision and robust positioning through the interactive collaboration of the data acquisition module, visual processing module, local map construction module, global positioning module, and filtering optimization module. It also integrates semantic information to improve the level of intelligence and adapt to complex dynamic park environments.

[0045] The vision-based positioning method for unmanned light trucks in industrial parks provided in this application can be applied to electronic devices. Those skilled in the art will understand that the electronic device structure involved in the embodiments of this invention does not constitute a limitation on the electronic device. An electronic device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. In the embodiments of this invention, the electronic device includes, but is not limited to, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.

[0046] Electronic devices may include processors, external memory interfaces, internal memory, universal serial bus (USB) interfaces, charging management modules, power management modules, batteries, wireless communication modules, audio modules, speakers, microphones, sensor modules, buttons, cameras, displays, and SIM card interfaces, etc.

[0047] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0048] A processor may include one or more processing units, such as a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0049] The processor can serve as the nerve center and command center of an electronic device. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0050] The processor may also include memory for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. This memory can store instructions or data that the processor has just used or that are used repeatedly. If the processor needs to use the instruction or data again, it can retrieve it directly from this memory. This avoids repeated accesses, reduces processor latency, and thus improves system efficiency.

[0051] The aforementioned electronic device implements the visual perception-based positioning method for unmanned light trucks in industrial parks, as described in this application. It involves installing a multi-view vision system at the front of the unmanned light truck to acquire real-time visual images of the industrial park scene; using computer vision algorithms to extract feature points from the visual images, and combining this with a pre-trained deep learning model to identify and classify objects in the visual images within the industrial park scene; based on the extracted feature points and the object recognition results, using a stereo vision algorithm to calculate the depth information of the objects in the industrial park scene, generating a local perception map of the unmanned light truck's location; matching the local perception map with a pre-constructed high-precision map to determine the current position of the unmanned light truck; and using a Kalman filter algorithm to perform real-time correction on the current position to obtain an optimized positioning result. This technical solution achieves the beneficial effects of combining multi-view vision and deep learning to realize centimeter-level precise positioning, reducing reliance on GNSS, and enhancing the system's real-time performance and stability.

[0052] The storage medium provided in this application stores a program product capable of implementing a vision-based method for locating unmanned light trucks in a park.

[0053] The vision-based method for locating unmanned light trucks in industrial parks includes: installing a multi-view vision system at the front of the unmanned light truck to acquire real-time visual images of the park scene; using computer vision algorithms to extract feature points from the visual images, and combining a pre-trained deep learning model to identify and classify objects in the park scene within the visual images; based on the extracted feature points and the object recognition results, using a stereo vision algorithm to calculate the depth information of the objects in the park scene, generating a local perception map of the unmanned light truck's location; matching the local perception map with a pre-constructed high-precision map to determine the current position of the unmanned light truck; and using a Kalman filter algorithm to correct the current position in real time to obtain an optimized positioning result.

[0054] In some possible implementations, the vision-based autonomous light truck positioning method for industrial parks disclosed herein can be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0055] The storage medium disclosed herein may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0056] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for locating unmanned light trucks in a park based on visual perception, characterized in that, Includes the following steps: S1. Install a multi-view vision system at the front of the driverless light truck to collect visual images of the park scene in real time; S2. Use computer vision algorithms to extract feature points from the visual image, and combine a pre-trained deep learning model to identify and classify objects in the park scene in the visual image; S3. Based on the extracted feature points and the object recognition results of the park scene, use stereo vision algorithms to calculate the depth information of the objects in the park scene and generate a local perception map of the location of the unmanned light truck. S4. Match the local perception map with the pre-constructed high-precision map to determine the current location of the unmanned light truck; S5. The current position is corrected in real time using the Kalman filter algorithm to obtain the optimized positioning result.

2. The method for locating unmanned light trucks in a park based on visual perception according to claim 1, characterized in that, The specific steps of step S1 are as follows: S11. Deploy at least two wide-angle cameras at the front of the driverless light truck to form a multi-view vision system, and the baseline distance between each wide-angle camera is a fixed value; S12. Control all wide-angle cameras by synchronizing the clock to capture visual images of the park scene from a preset number of perspectives at the same time. S13. Perform denoising and distortion correction preprocessing on the acquired raw visual images; The denoising algorithm uses a Gaussian filtering algorithm, and the convolution kernel function is: In the formula, Let σ be the pixel coordinates, and σ be the standard deviation of the Gaussian distribution. Distortion correction uses the Brownian distortion algorithm. The formulas for radial and tangential distortion correction are as follows: In the formula, The coordinates of the original visual image, For the corrected coordinates, , , The radial distortion coefficient is... , The tangential distortion coefficient is... .

3. The method for locating unmanned light trucks in a park based on visual perception according to claim 2, characterized in that, The specific steps of step S2 are as follows: S21. The ORB algorithm is used to extract the first key feature point from the preprocessed visual image, and the descriptor of the first key feature point is calculated. Specifically, the ORB algorithm first uses the FAST corner detector to locate the first key feature point, then uses the intensity centroid method to assign an orientation to each first key feature point, and finally uses the rBRIEF descriptor generation algorithm to calculate the binary string form descriptor for each first key feature point. S22. Input the visual image into the pre-trained YOLOv5 model to identify and classify objects in the park scene, and output the category label of each object in the park scene and the bounding box coordinates of each object in the visual image. The YOLOv5 model consists of three parts: the main network, the neck network, and the head. The training loss function is composed of localization loss, confidence loss, and classification loss. The YOLOv5 model was obtained through supervised training on a labeled image dataset containing a set number of park scene elements. S23. Determine the spatial location of the extracted first key feature point, and associate the first key feature point with the bounding box coordinates of each park scene object in the visual image based on the spatial location; If the first key feature point is located within the bounding box of an object in a certain park scene, then the corresponding first key feature point is assigned the category label of the corresponding park scene object, and a second key feature point with semantic information is generated. All second key feature points constitute an enhanced feature point set.

4. The method for locating unmanned light trucks in a park based on visual perception according to claim 3, characterized in that, The specific steps of step S3 are as follows: S31. Based on the SGBM algorithm, using the enhanced feature point set as a guide, the search range for matching the second key feature point is limited to the bounding box of the park scene object in the visual image and the spatial structure region between adjacent park scene objects, so as to calculate the disparity d of each second key feature point in the enhanced feature point set in the left and right views. S32. Based on the principle of triangulation, and using the disparity d, calculate the physical depth value corresponding to each second key feature point. : in, This represents the depth distance of the second key feature point relative to the wide-angle camera. This indicates the baseline distance of the wide-angle camera in a multi-view vision system. Indicates the focal length of a wide-angle camera; S33. Integrate the two-dimensional image coordinates, semantic information, and physical depth values ​​of all second key feature points. By transforming the coordinates of the points in the image coordinate system and corresponding depth values Transformed to 3D points in camera coordinate system And based on semantic information, the three-dimensional points Clustering is performed to generate point cloud clusters based on semantic objects, and a local perception map containing 3D point clouds with semantic labels is constructed.

5. The method for locating unmanned light trucks in a park based on visual perception according to claim 4, characterized in that, The specific steps of step S31 are as follows: S311. Calculate the cost of all possible parallaxes within the parallax range for each pixel, and construct a cost cube; S312. The cost value is smoothed in several predetermined directions by one-dimensional cost aggregation of multiple paths; S313. For each second key feature point in the enhanced feature point set, select the disparity value pair with the minimum aggregation cost corresponding to the location of the second key feature point as the reliable disparity d of the corresponding second key feature point, and perform regional voting on the disparity results of all second key feature points to generate a sparse disparity map. S314. For second key feature points belonging to the same semantic category, if the deviation between the disparity value and the median disparity value within the corresponding semantic category group exceeds a preset threshold, the corresponding second key feature point is determined to be a mismatch point and is removed.

6. The method for locating unmanned light trucks in a park based on visual perception according to claim 4, characterized in that, The specific steps of step S4 are as follows: S41. Call up a high-precision map of the park that has been pre-built using LiDAR, the high-precision map of the park containing three-dimensional point clouds and feature description information of roads, factories and landmarks with centimeter-level precision; S42. Extract the second key feature point in the local perception map and match it with the feature point descriptor in the high-precision map. Use Hamming distance as a similarity measure to find the correspondence between the two sets of data. S43. Based on the successfully matched feature point pairs, the transformation matrix is ​​estimated using the random sampling consensus algorithm. By aligning and fusing the local perception map with the high-precision map, the current position of the autonomous light truck in the global coordinate system can be calculated. and the current position Output as observed values.

7. The method for locating unmanned light trucks in a park based on visual perception according to claim 6, characterized in that, The specific steps of step S5 are as follows: S51. Based on the optimized state vector of the unmanned light truck at time k-1. The prediction equation using Kalman filtering is used to predict the prior state estimate at time k. and prior estimation of covariance matrix The prediction equation is: ； ； in, The state prior estimate vector at time k includes position and velocity; A is the prior estimate of the covariance matrix; A is the state transition matrix; B is the posterior state estimation vector at time k-1; B is the control input matrix. Let Q be the control vector at time k-1; Q is the process noise covariance matrix. S52. Solve for the current position As observed value Based on the prediction results, the Kalman gain is calculated. And update the state estimate and covariance, with the update equation being: Where H is the observation matrix, R is the observation noise covariance matrix, and I is the identity matrix; S53. Update the state vector The location information in the data is output as the final positioning result after optimization at time k.

8. A positioning system for unmanned light trucks in a park based on visual perception, characterized in that, include: The data acquisition module is used to install a multi-view vision system at the front of the driverless light truck to acquire visual images of the park scene in real time. The visual processing module is used to extract feature points from the visual image using computer vision algorithms, and to identify and classify objects in the park scene in the visual image by combining a pre-trained deep learning model. The local map construction module is used to calculate the depth information of objects in the park scene based on the extracted feature points and the object recognition results of the park scene, and generate a local perception map of the location of the unmanned light truck. The global positioning module is used to match the local perception map with a pre-built high-precision map to determine the current location of the unmanned light truck. The filtering optimization module is used to perform real-time correction on the current position using the Kalman filtering algorithm to obtain an optimized positioning result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the vision-based unmanned light truck positioning method for parks as described in any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the vision-based unmanned light truck positioning method for parks as described in any one of claims 1 to 7.

Images