Obstacle recognition method and device based on static obstacle memory and electronic equipment

Abstract

The invention relates to the technical field of intelligent driving, and provides an obstacle recognition method and device based on static obstacle memory and electronic equipment. Generating real-time obstacle information and position information of the target vehicle in the global map; according to the real-time obstacle information, obstacles of a preset category and obstacles with the size within a preset small target size range are screened out; performing coordinate conversion according to the position information, and outputting the position of the screened obstacle in the global map; based on the positions of the screened obstacles in the global map, matching the screened obstacles with obstacles in a pre-established static obstacle storage list to generate static obstacle information; and performing de-overlapping combination on the real-time obstacle information and the static obstacle information, and outputting comprehensive obstacle information. According to the method, the perception capability of the vehicle to the static obstacle is improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology, and in particular to an obstacle recognition method, device and electronic equipment based on static obstacle memory. Background Art

[0002] With the rapid development of autonomous driving technology, intelligent driving systems are becoming increasingly widely used in ground transportation. Intelligent driving vehicles integrate multiple sensors (such as lidar, cameras, and millimeter-wave radar) and advanced algorithms to achieve environmental perception, decision-making, and control. These technological advances enable vehicles to navigate complex traffic environments safely and efficiently.

[0003] Existing intelligent driving technologies typically rely on multiple sensors to acquire obstacle information. However, in certain scenarios, such as coal mines and tunnels, high dust concentrations, large light fluctuations, confined spaces, and the frequent presence of small objects contribute to the sparse point cloud generated by lidar in high dust levels, and the inability of cameras to effectively recognize semantic objects in low light. Consequently, the vehicle's perception system cannot effectively identify obstacles in the surrounding environment, increasing the risk of accidents. Summary of the Invention

[0004] The present invention provides an obstacle recognition method, device and electronic equipment based on static obstacle memory, which are used to solve the defect of the existing technology that the vehicle's perception ability is reduced in environments such as mine tunnels, thereby reducing the risk of accidents.

[0005] The present invention provides an obstacle recognition method based on static obstacle memory, comprising: collecting environmental data and vehicle data through sensors of a target vehicle; generating real-time obstacle information and position information of the target vehicle in a global map based on the environmental data and the vehicle data; screening obstacles of a preset category and obstacles within a preset small target size range based on the real-time obstacle information; performing coordinate conversion based on the position information and outputting the positions of the screened obstacles in the global map; matching the screened obstacles with obstacles in a pre-established static obstacle storage list based on the positions of the screened obstacles in the global map to generate static obstacle information; and de-overlapping and merging the real-time obstacle information and the static obstacle information to output comprehensive obstacle information.

[0006] According to the obstacle recognition method based on static obstacle memory provided by the present invention, the sensors include a camera and a laser radar; and the generation of real-time obstacle information and the position information of the target vehicle in the global map based on the environmental data and the vehicle's own data includes: time synchronization and coordinate system processing of the collected data of the camera and the laser radar, and outputting synchronized and unified sensor data; matching the real-time point cloud with the global map based on the synchronized and unified sensor data, and outputting a preliminary position estimation result; using an inertial measurement unit and odometer data to perform short-term motion prediction on the preliminary position estimation result, compensating for delays or errors in point cloud matching to obtain a prediction result; using a Kalman filter or graph optimization method to fuse the prediction result with the point cloud matching result, the inertial measurement unit and the odometer data, and outputting an optimized pose estimate; during long-term operation, detecting whether the vehicle has returned to a known position based on the optimized pose estimate, performing global optimization to correct accumulated errors, and outputting a globally optimized pose; and outputting the globally optimized pose as the position information of the target vehicle in the global map.

[0007] According to the obstacle recognition method based on static obstacle memory provided by the present invention, the obstacle recognition method generates real-time obstacle information and the position information of the target vehicle in the global map based on the environmental data and the vehicle's own data, and also includes: aligning the camera and the lidar sensors, calibrating the coordinate relationship between the two, and outputting the calibrated coordinate parameters; normalizing and enhancing the multi-view images captured by the camera, and outputting enhanced image data; extracting image semantic features based on the enhanced image data; estimating the depth probability of each pixel based on the image semantic features, and outputting a depth probability distribution; projecting the image onto a grid to generate image branch features based on the depth probability distribution and the calibrated coordinate parameters; based on the calibrated coordinate parameters, range filtering the lidar point cloud and segmenting it into a grid to generate point cloud branch features; generating fused features based on the image branch features and the point cloud branch features; and outputting real-time obstacle information based on the fused features through a target detection algorithm and an image segmentation algorithm.

[0008] According to the obstacle recognition method based on static obstacle memory provided by the present invention, the method matches the screened obstacles with obstacles in a pre-established static obstacle storage list based on the positions of the screened obstacles in the global map to generate static obstacle information, including: using the Hungarian algorithm to match the screened obstacles with obstacles in the pre-established static obstacle storage list, and the matching score calculation formula is as follows:

[0009] ;

[0010] ;

[0011] Among them, (x1, y1) and (x2, y2) represent the positions of the center points of the two obstacles respectively; Dis refers to the Euclidean distance between the two center points; match_score represents the matching score of the two obstacles, D th is the set distance value; if the matching score is greater than the preset score threshold, the obstacle is successfully matched, the matching counter of the successfully matched obstacle in the static obstacle list is incremented, and the obstacle position and timestamp in the static obstacle list are updated; if the matching score is less than the preset score threshold, the obstacle is failed, the unmatched counter of the unmatched obstacle in the static obstacle list is incremented, and the real-time obstacle information of the unmatched obstacle is added to the static obstacle list and the initial position of the obstacle is recorded; the obstacles in the static obstacle list whose unmatched counter is greater than the preset value or whose initial position is greater than the current position are offset by a distance threshold ( ) object; output the static obstacle information whose matching counter in the static obstacle list is greater than a preset value.

[0012] According to the obstacle recognition method based on static obstacle memory provided by the present invention, the distance value is determined by the following formula:

[0013] ;

[0014] Among them, W is the current lane width (obtained by real-time scanning of the laser radar), W0 is the preset standard lane width, and W max is the maximum width of underground mine tunnels, and α is the adjustment coefficient.

[0015] According to the obstacle recognition method based on static obstacle memory provided by the present invention, the distance threshold ( ) is determined by the following formula:

[0016] ;

[0017] in, represents the distance threshold, v is the real-time speed of the target vehicle, and γ is the speed influence coefficient.

[0018] According to the obstacle recognition method based on static obstacle memory provided by the present invention, the laser radar includes an overhead laser radar and a blind spot laser radar. The overhead laser radar performs 360° rotation scanning at a frequency of 10Hz to generate high-density point cloud data. The blind spot laser radar is deployed in front of, behind, left and right of the vehicle, respectively, and covers the near blind spot at 10Hz. The camera includes a front view, a side view wide-angle and a rear view lens. The camera synchronously captures multispectral images at 30FPS. The side view camera covers the intersection field of view. The image data collected by the camera is processed in real time by an image signal processor and annotated with a timestamp. The data collected by the overhead laser radar, the blind spot laser radar and the camera are aligned through hardware-level time synchronization.

[0019] The present invention also provides an obstacle recognition device based on static obstacle memory, characterized by comprising: an acquisition module configured to acquire environmental data and vehicle data through sensors of a target vehicle; a first generation module configured to generate real-time obstacle information and position information of the target vehicle in a global map based on the environmental data and the vehicle data; a screening module configured to screen out obstacles of a preset category and obstacles within a preset small target size range based on the real-time obstacle information; a first output module configured to perform coordinate conversion based on the position information and output the positions of the screened obstacles in the global map; a second generation module configured to match the screened obstacles with obstacles in a pre-established static obstacle storage list based on the positions of the screened obstacles in the global map to generate static obstacle information; and a second output module configured to de-overlap and merge the real-time obstacle information and the static obstacle information to output comprehensive obstacle information.

[0020] The obstacle recognition method, device, and electronic device based on static obstacle memory provided by the present invention propose an obstacle recognition solution based on a static obstacle storage list, which can improve the vehicle's ability to perceive obstacles in harsh environments such as underground mine tunnels, thereby improving vehicle driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 It is a flow chart of the obstacle recognition method based on static obstacle memory provided by the present invention.

[0023] Figure 2 It is a schematic diagram of the process of static obstacle recognition provided by the present invention.

[0024] Figure 3 It is a structural diagram of the obstacle recognition method based on static obstacle memory provided by the present invention.

[0025] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0026] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0027] Unless otherwise defined, the technical or scientific terms used in the present invention shall have the usual meaning understood by persons of ordinary skill in the field to which the present invention belongs. The words "first", "second" and similar terms used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "one", "an" or "the" do not indicate a quantity limitation, but rather indicate the presence of at least one. Words such as "include" or "comprise" mean that the elements or objects preceding the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Words such as "connect" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

[0028] The following is a brief explanation of the terms involved in the present invention.

[0029] The following combination Figure 1-Figure 4 The present invention describes an obstacle recognition method, device and electronic device based on static obstacle memory.

[0030] Figure 1 is a flow chart of the obstacle recognition method based on static obstacle memory provided by the present invention, such as Figure 1 As shown, the method includes the following:

[0031] Step 101: Collect environmental data and vehicle data through sensors of the target vehicle;

[0032] In this embodiment, the target vehicle can be an autonomous vehicle such as an unmanned mining truck or inspection vehicle performing underground mine tunnel perception tasks. Sensors can include an inertial navigation system, an odometer, a lidar, and a camera. Environmental data can include lidar point clouds (including obstacle distances and reflection intensity), the 3D structure of the tunnel, and camera images. Vehicle data can include real-time position, velocity, and acceleration.

[0033] Step 102: Generate real-time obstacle information and target vehicle position information in a global map based on the environmental data and the vehicle's own data.

[0034] In this embodiment, sensor fusion technology, such as lidar and cameras, can be used to ensure accurate vehicle positioning and obstacle identification even in poor visibility conditions. Data fusion algorithms, such as Kalman filters and extended Kalman filters (EKF), can be used to improve positioning accuracy and robustness. Kalman filters can fuse multi-sensor data, eliminating noise and errors and improving positioning accuracy, while extended Kalman filters (EKFs) can process nonlinear sensor data, further improving positioning robustness.

[0035] Step 103: Filter out obstacles of a preset category and obstacles whose sizes are within a preset small target size range according to the real-time obstacle information.

[0036] In this embodiment, the preset categories may include cones and signboards, and obstacles within the preset small target size range may include obstacles with a length and width of less than 1.2m. The preset categories and the preset small target size range can be determined based on the working environment of the target vehicle.

[0037] Step 104: Perform coordinate conversion based on the position information and output the position of the screened obstacles in the global map.

[0038] In this embodiment, coordinate conversion based on position information refers to the conversion relationship between the vehicle body coordinate system (base_link) and the global map coordinate system (map). The position information of the selected obstacles is converted from the vehicle body coordinate system to the global map coordinate system. The obstacle attributes such as the outline, category, and speed remain unchanged during the conversion process. The final output is the position of these obstacles in the global map.

[0039] Step 105 : Based on the positions of the screened obstacles in the global map, the screened obstacles are matched with obstacles in a pre-established static obstacle storage list to generate static obstacle information.

[0040] In this embodiment, when matching the positions of the screened obstacles in the global map, these obstacles can be compared with the obstacles in the pre-established static obstacle storage list using algorithms such as the Hungarian matching algorithm and the feature similarity-based matching algorithm to generate static obstacle information.

[0041] Step 106: De-duplication and merge the real-time obstacle information and the static obstacle information to output comprehensive obstacle information.

[0042] In this embodiment, a flag indicating whether a static obstacle is successfully matched can be used: if the output static obstacle and the real-time perceived obstacle match successfully, it means that the obstacle has not been missed and does not need to be output; if the match is not successful, the static obstacle is added to the final obstacle output list, and the comprehensive obstacle information is obtained by merging.

[0043] The obstacle recognition method based on static obstacle memory provided by the present invention proposes an obstacle recognition scheme based on a static obstacle storage list, which can improve the vehicle's ability to perceive obstacles in harsh environments such as underground mine tunnels, reduce the accident rate, and lower the impact of harsh environments on production efficiency.

[0044] In some optional implementations, the sensors include cameras and lidars; and based on environmental data and the vehicle's own data, real-time obstacle information and the position information of the target vehicle in the global map are generated, including: time synchronization and coordinate system processing of the collected data of the camera and lidar, and output of synchronized and unified sensor data; based on the synchronized and unified sensor data, the real-time point cloud is matched with the global map, and a preliminary position estimation result is output; short-term motion prediction of the preliminary position estimation result is performed using inertial measurement unit and odometer data, and the delay or error of point cloud matching is compensated to obtain the prediction result; Kalman filtering or graph optimization method is used to fuse the prediction result with the point cloud matching result, inertial measurement unit and odometer data, and output an optimized pose estimate; during long-term operation, based on the optimized pose estimate, whether it has returned to a known position, global optimization is performed to correct the accumulated error, and the globally optimized pose is output; the globally optimized pose is output as the position information of the target vehicle in the global map.

[0045] In some optional implementations, real-time obstacle information and the position information of the target vehicle in the global map are generated based on environmental data and the vehicle's own data, and the method also includes: aligning the camera and lidar sensors, calibrating the coordinate relationship between the two, and outputting the calibrated coordinate parameters; normalizing and enhancing the multi-view images captured by the camera, and outputting enhanced image data; extracting image semantic features based on the enhanced image data; estimating the depth probability of each pixel based on the image semantic features, and outputting the depth probability distribution; projecting the image onto a grid to generate image branch features based on the depth probability distribution and the calibrated coordinate parameters; based on the calibrated coordinate parameters, range filtering the lidar point cloud and segmenting it into a grid to generate point cloud branch features; generating fused features based on the image branch features and the point cloud branch features; and outputting real-time obstacle information based on the fused features through the target detection algorithm and the image segmentation algorithm.

[0046] In this implementation, the grid can be a BEV (Bird's Eye View) grid. During the BEV feature fusion process, a weighting coefficient can be assigned to the BEV grid features of the area corresponding to the screened small targets. The weighting coefficient is inversely proportional to the size of the small target, that is, the smaller the target, the higher the weight of the corresponding grid feature. The weighted fused BEV features are used for target prediction in the subsequent detection head; the size information of the small target is provided by the real-time obstacle information output by the perception fusion module as input for the calculation of the weighting coefficient.

[0047] In some optional implementations, based on the positions of the screened obstacles in the global map, the screened obstacles are matched with obstacles in a pre-established static obstacle storage list to generate static obstacle information, including: using the Hungarian algorithm to match the screened obstacles with obstacles in the pre-established static obstacle storage list, and the matching score calculation formula is as follows:

[0048] ;

[0049] ;

[0050] Among them, (x1, y1) and (x2, y2) represent the positions of the center points of the two obstacles respectively; Dis refers to the Euclidean distance between the two center points; match_score represents the matching score of the two obstacles, D this the set distance value; if the matching score is greater than the preset score threshold, the obstacle is matched successfully, the matching counter of the successfully matched obstacle in the static obstacle list is incremented, and the obstacle position and timestamp in the static obstacle list are updated; if the matching score is less than the preset score threshold, the obstacle is matched unsuccessfully, the unmatched counter of the unmatched obstacle in the static obstacle list is incremented, and the real-time obstacle information of the unmatched obstacle is added to the static obstacle list and the initial position of the obstacle is recorded; the obstacles in the static obstacle list whose unmatched counter is greater than the preset value or whose initial position is greater than the current position is greater than the distance threshold are deleted. ) object; outputs the static obstacle information in the static obstacle list whose matching counter is greater than the preset value.

[0051] See also Figure 2 , Figure 2 The implementation process of static obstacle recognition is shown, mainly demonstrating the complete process from positioning and perception result input to static obstacle output, including small target obstacle filtering (screening out preset categories such as cones and signs, and obstacles with a length and width of less than 1.2m), global coordinate conversion (converting the small target position from the vehicle coordinate system to the global map coordinate system), matching factor calculation (using Euclidean distance and matching score formula), static obstacle list creation and matching (using Hungarian matching to distinguish successfully matched objects, unmatched old objects, and unmatched new objects), information update (updating the number of matches, unmatched counts, and adding new objects), list maintenance (deleting objects that have not been matched for more than 50 frames or whose positions differ by more than 2m), and static obstacle output (outputting objects with a match_count value greater than 10).

[0052] In some optional implementations, the distance value is determined by the following formula:

[0053] ;

[0054] Among them, W is the current lane width (obtained by real-time scanning of the laser radar), W0 is the preset standard lane width, and W max is the maximum width of underground mine tunnels, α is the adjustment coefficient, and the range can be 0.2~0.5.

[0055] In some optional implementations, the distance threshold ( ) is determined by the following formula:

[0056] ;

[0057] in, represents the distance threshold, v is the target vehicle's real-time speed, and γ is the speed influence coefficient. Alternatively, the distance value can be fixed, for example, 2.5 meters. Alternatively, the matching factor threshold can be dynamically adjusted based on the dust concentration and light intensity in the underground mine tunnel environment. For example, when dust concentration is above a preset value or light intensity is below a preset value, the matching factor distance threshold is adjusted from 2.5m to 2m to enhance the recognition stability of small targets in low-visibility environments. The dust concentration and light intensity are collected in real time by sensors on the target vehicle and serve as inputs for adjusting the threshold.

[0058] Furthermore, during static obstacle list maintenance, the system measures the duration of a static obstacle's existence. When the duration exceeds a preset value, the threshold for matching that obstacle is dynamically increased from 10 to 15 times to filter out long-standing, but potentially inactive, static objects. This duration, which begins when the obstacle is first added to the static obstacle storage list, serves as an input for threshold adjustment. Before adding a new, unmatched object to the static obstacle storage list, the system verifies the object's presence over three consecutive frames. Only if the object is detected in all three consecutive frames with a position deviation of less than 0.5m is it added to the list. The detection results from these three consecutive frames serve as input for determining whether to add the new object, with the position deviation calculated based on the object's position within the global map in each frame.

[0059] In some optional implementations, the lidar includes a top-mounted lidar and a blind spot-filling lidar. The top-mounted lidar performs 360-degree rotational scanning at a 10Hz frequency, generating high-density point cloud data. The blind spot-filling lidars are deployed on the front, rear, left, and right sides of the vehicle, covering near-blind spots at a 10Hz frequency. The cameras include front-facing, side-facing, wide-angle, and rear-facing lenses, synchronously capturing multispectral images at 30 FPS. The side-facing cameras cover intersections. The image data captured by the cameras is processed and timestamped in real time by an image signal processor. The data collected by the top-mounted lidar, blind spot-filling lidar, and cameras are aligned through hardware-level time synchronization. Specific parameters such as frequency can be determined based on actual conditions. For example, when the point cloud density detected by the top lidar falls below a preset value, the scanning frequency of the blind spot-filling lidar is triggered to increase from 10Hz to 20Hz, and the blind spot-filling lidar's point cloud data is prioritized for initial detection of small targets. The top lidar's point cloud density serves as an input for frequency adjustment of the blind spot-filling lidar, and the adjusted blind spot-filling lidar data is prioritized for processing by the perception fusion module.

[0060] The obstacle recognition device based on static obstacle memory provided by the present invention is described below. The obstacle recognition device based on static obstacle memory described below and the obstacle recognition method based on static obstacle memory described above can be referred to in correspondence with each other.

[0061] Figure 3 A schematic diagram of the structure of an obstacle recognition device based on static obstacle memory provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, the system specifically includes: a collection module 301 configured to collect environmental data and vehicle data through sensors of the target vehicle; a first generation module 302 configured to generate real-time obstacle information and the target vehicle's position information in the global map based on the environmental data and the vehicle's data; a screening module 303 configured to screen out obstacles of a preset category and obstacles within a preset small target size range based on the real-time obstacle information; a first output module 304 configured to perform coordinate conversion based on the position information and output the positions of the screened obstacles in the global map; a second generation module 305 configured to match the screened obstacles with obstacles in a pre-established static obstacle storage list based on the positions of the screened obstacles in the global map to generate static obstacle information; and a second output module 306 configured to de-overlap and merge the real-time obstacle information and the static obstacle information to output comprehensive obstacle information.

[0062] The obstacle recognition device based on static obstacle memory provided by the present invention can integrate the data of the vehicle's own sensors with the data obtained by other positioning and perception technologies in harsh environments such as underground mine tunnels, thereby improving the vehicle's perception of static obstacles.

[0063] Figure 4 An example of a physical structure diagram of an electronic device is shown below. Figure 4As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communications bus 440. The processor 410, the communications interface 420, and the memory 430 communicate with each other via the communications bus 440. The processor 410 may invoke logic instructions in the memory 430 to execute an obstacle recognition method based on static obstacle memory. The method includes: collecting environmental data and vehicle data through sensors of the target vehicle; generating real-time obstacle information and the target vehicle's position in a global map based on the environmental data and the vehicle data; screening obstacles of a preset category and obstacles within a preset small target size range based on the real-time obstacle information; performing coordinate conversion based on the position information and outputting the positions of the screened obstacles in the global map; matching the screened obstacles with obstacles in a pre-established static obstacle storage list based on their positions in the global map to generate static obstacle information; and de-overlapping and merging the real-time obstacle information and the static obstacle information to output comprehensive obstacle information.

[0064] Furthermore, the logic instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0065] On the other hand, the present invention also provides a computer program product, comprising a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the obstacle recognition method based on static obstacle memory provided by the above-mentioned methods, comprising: collecting environmental data and vehicle data through sensors of a target vehicle; generating real-time obstacle information and position information of the target vehicle in a global map based on the environmental data and the vehicle data; screening obstacles of preset categories and obstacles within a preset small target size range based on the real-time obstacle information; performing coordinate conversion based on the position information and outputting the positions of the screened obstacles in the global map; matching the screened obstacles with obstacles in a pre-established static obstacle storage list based on the positions of the screened obstacles in the global map to generate static obstacle information; and de-overlapping and merging the real-time obstacle information and the static obstacle information to output comprehensive obstacle information.

[0066] In yet another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program is implemented to perform the obstacle recognition method based on static obstacle memory provided by the above-mentioned methods, comprising: collecting environmental data and vehicle data through sensors of a target vehicle; generating real-time obstacle information and position information of the target vehicle in a global map based on the environmental data and the vehicle data; screening obstacles of preset categories and obstacles within a preset small target size range based on the real-time obstacle information; performing coordinate conversion based on the position information and outputting the positions of the screened obstacles in the global map; matching the screened obstacles with obstacles in a pre-established static obstacle storage list based on the positions of the screened obstacles in the global map to generate static obstacle information; and de-overlapping and merging the real-time obstacle information and the static obstacle information to output comprehensive obstacle information.

[0067] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0068] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0069] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. An obstacle recognition method based on static obstacle memory, characterized in that: include: Collect environmental data and vehicle data through the target vehicle's sensors; generating real-time obstacle information and position information of the target vehicle in a global map based on the environmental data and the vehicle's own data; Filtering out obstacles of a preset category and obstacles whose sizes are within a preset small target size range based on the real-time obstacle information; Perform coordinate conversion based on the position information and output the position of the filtered obstacles in the global map; Based on the positions of the screened obstacles in the global map, the screened obstacles are matched with obstacles in a pre-established static obstacle storage list to generate static obstacle information; The real-time obstacle information and the static obstacle information are de-overlapped and merged to output comprehensive obstacle information.

2. The method according to claim 1, characterized in that The sensors include cameras and laser radars; And generating real-time obstacle information and position information of the target vehicle in a global map based on the environmental data and the vehicle's own data includes: Performing time synchronization and coordinate system processing on the collected data of the camera and the lidar, and outputting synchronized and unified sensor data; Based on the synchronized and unified sensor data, matching the real-time point cloud with the global map and outputting a preliminary position estimate; Performing short-term motion prediction on the preliminary position estimate using inertial measurement unit and odometer data to compensate for delays or errors in point cloud matching to obtain a prediction result; Using Kalman filtering or graph optimization methods, the prediction results are integrated with point cloud matching results, inertial measurement unit and odometry data to output an optimized pose estimate; During long-term operation, based on the optimized pose estimation, the detection is performed to determine whether the position has returned to the known position, a global optimization is performed to correct the accumulated error, and a globally optimized pose is output; The globally optimized posture is output as the position information of the target vehicle in the global map.

3. The method according to claim 2, characterized in that The generating of real-time obstacle information and position information of the target vehicle in a global map based on the environmental data and the vehicle's own data further includes: Performing sensor alignment on the camera and the laser radar, calibrating the coordinate relationship between the two, and outputting the calibrated coordinate parameters; Normalizing and enhancing the multi-view images captured by the camera, and outputting enhanced image data; Extracting image semantic features based on the enhanced image data; estimating a depth probability of each pixel based on the image semantic features, and outputting a depth probability distribution; Projecting the image onto a grid to generate image branch features based on the depth probability distribution and the calibrated coordinate parameters; Based on the calibrated coordinate parameters, range filtering is performed on the laser radar point cloud and segmenting is performed into grids to generate point cloud branch features; generating fused features based on the image branch features and the point cloud branch features; Based on the fused features, real-time obstacle information is output through target detection algorithm and image segmentation algorithm.

4. The method according to claim 1, wherein The step of matching the filtered obstacles with obstacles in a pre-established static obstacle storage list based on the positions of the filtered obstacles in the global map to generate static obstacle information includes: The Hungarian algorithm is used to match the screened obstacles with the obstacles in the pre-established static obstacle storage list. The matching score calculation formula is as follows: ； ； Among them, (x1, y1) and (x2, y2) represent the positions of the center points of the two obstacles respectively; Dis refers to the Euclidean distance between the two center points; match_score represents the matching score of the two obstacles, D th is the set distance value; If the matching score is greater than the preset score threshold, the obstacle is successfully matched, the matching counter of the successfully matched obstacle in the static obstacle list is incremented, and the obstacle position and timestamp in the static obstacle list are updated; If the matching score is less than the preset score threshold, the obstacle matching fails, an unmatched counter of the obstacle in the static obstacle list that failed to match is incremented, and the real-time obstacle information of the obstacle that failed to match is added to the static obstacle list and the initial position of the obstacle is recorded; Delete the static obstacle list if the unmatched counter is greater than the preset value or the displacement between the initial position and the current position is greater than the distance threshold ( ) objects; Output the static obstacle information in the static obstacle list whose matching counter is greater than a preset value.

5. The method according to claim 4, characterized in that The distance value is determined by the following formula: ； Among them, W is the current lane width (obtained by real-time scanning of the laser radar), W0 is the preset standard lane width, and W max is the maximum width of underground mine tunnels, and α is the adjustment coefficient.

6. The method according to claim 4, characterized in that The distance threshold ( ) is determined by the following formula: ； in, represents the distance threshold, v is the real-time speed of the target vehicle, and γ is the speed influence coefficient.

7. The method according to claim 2, characterized in that The laser radar includes a top-mounted laser radar and a blind spot-filling laser radar. The top-mounted laser radar performs 360° rotation scanning at a frequency of 10 Hz to generate high-density point cloud data. The blind spot-filling laser radars are deployed in front of, behind, on the left and right sides of the vehicle, covering near blind spots at 10 Hz. The camera includes a front-view, side-view wide-angle and rear-view lens. The camera synchronously collects multispectral images at 30 FPS. The side-view camera covers the intersection field of view. The image data collected by the camera is processed in real time by an image signal processor and timestamps are annotated. The data collected by the top-mounted laser radar, the blind spot-filling laser radar and the camera are aligned through hardware-level time synchronization.

8. An obstacle recognition device based on static obstacle memory, characterized in that: include: a collection module configured to collect environmental data and vehicle data through sensors of the target vehicle; A first generating module is configured to generate real-time obstacle information and position information of the target vehicle in a global map based on the environmental data and the vehicle's own data; a screening module configured to screen out obstacles of a preset category and obstacles whose sizes are within a preset small target size range based on the real-time obstacle information; a first output module, configured to perform coordinate conversion according to the position information and output the position of the screened obstacle in the global map; a second generating module configured to match the screened obstacles with obstacles in a pre-established static obstacle storage list based on positions of the screened obstacles in the global map to generate static obstacle information; The second output module is configured to de-overlap and merge the real-time obstacle information and the static obstacle information, and output comprehensive obstacle information.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the obstacle recognition method based on static obstacle memory according to any one of claims 1 to 7 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the obstacle recognition method based on static obstacle memory according to any one of claims 1 to 7 is implemented.

Images