Obstacle information processing method and device, storage medium and electronic equipment

Abstract

The invention provides an obstacle information processing method and device, a storage medium and electronic equipment, and relates to the field of smart mines, automatic driving and unmanned vehicles. The obstacle information processing method comprises the following steps: acquiring a first obstacle queue, a second obstacle queue and a third obstacle queue of a first vehicle; performing fusion processing on the first obstacle queue and the second obstacle queue to obtain a fourth obstacle queue; and performing fusion processing on the fourth obstacle queue and the third obstacle queue to obtain a target obstacle queue. According to the multi-level information fusion method, limitation possibly brought by a single information source is overcome, more comprehensive understanding of the surrounding environment is achieved, and then the risk of misjudgment and missed judgment is reduced.

Description

Technical Field

[0001] The present application relates to the fields of smart mines, unmanned driving, and perception technology, and specifically to a method and device for processing obstacle information, a storage medium, and an electronic device. Background Art

[0002] In recent years, with the development of Internet of Vehicles technology, its application in vehicle dispatching, vehicle safety and other aspects has become increasingly widespread.

[0003] However, most manufacturers currently only focus on the application of V2X (Vehicle to Everything) information, and do not address how to improve the accuracy of V2X information. Summary of the invention

[0004] In view of this, embodiments of the present application provide a method and device for processing obstacle information, a storage medium, and an electronic device.

[0005] In a first aspect, an embodiment of the present application provides a method for processing obstacle information, which is applied to a first vehicle, the first vehicle is communicatively connected to at least one second vehicle, and the first vehicle and the at least one second vehicle are respectively communicatively connected to a control end; the method includes: obtaining a first obstacle queue, a second obstacle queue and a third obstacle queue of the first vehicle, wherein the first obstacle queue is queue information of obstacles determined based on V2X information sent by the second vehicle to the first vehicle, the second obstacle queue is queue information of obstacles determined based on V2X information sent by the control end to the first vehicle, and the third obstacle queue is queue information of obstacles determined based on perception information collected by a perception device of the first vehicle; fusing the first obstacle queue and the second obstacle queue to obtain a fourth obstacle queue; fusing the fourth obstacle queue and the third obstacle queue to obtain a target obstacle queue.

[0006] In combination with the first aspect, in some implementations of the first aspect, the first obstacle queue includes a V2V obstacle queue based on a single frame and a V2V obstacle tracking queue based on multi-frame fusion; the second obstacle queue includes a V2N obstacle queue based on a single frame and a V2N obstacle tracking queue based on multi-frame fusion; the first obstacle queue and the second obstacle queue are fused to obtain a fourth obstacle queue, including: fusing the V2V obstacle queue based on a single frame and the V2V obstacle tracking queue to obtain a new V2V obstacle tracking queue; fusing the V2N obstacle queue based on a single frame and the V2N obstacle tracking queue to obtain a new V2N obstacle tracking queue; matching elements in the new V2V obstacle tracking queue with elements in the new V2N obstacle tracking queue to determine whether the two elements in each matched element pair are from the same vehicle-side device; fusing the successfully matched element pairs into elements in the fourth obstacle queue, wherein successful matching indicates that the two elements in the element pair are from the same vehicle-side device.

[0007] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: when it is determined that the distance between two elements in an element pair that failed to match is less than a target distance threshold, recording information corresponding to the element pair that failed to match into a fault information queue for troubleshooting and modification.

[0008] In combination with the first aspect, in certain implementations of the first aspect, a single-frame-based V2V obstacle queue and a V2V obstacle tracking queue are fused to obtain a new V2V obstacle tracking queue, including: matching elements in the V2V obstacle queue and elements in the V2V obstacle tracking queue to determine whether two elements in each matched element pair are from the same vehicle-side device; for successfully matched element pairs, updating the elements in the V2V obstacle tracking queue in the element pair based on the elements in the V2V obstacle queue in the element pair; for elements in the V2V obstacle queue that fail to match, if it is determined that the obstacle represented by the element is within the target range of the first vehicle, determining that the obstacle information represented by the element is correct, and adding the element to the V2V obstacle tracking queue; for elements in the V2V obstacle queue that fail to match, if it is determined that the obstacle represented by the element is not within the target range of the first vehicle, determining that the obstacle information represented by the element is incorrect, and adding the element to the V2V obstacle tracking queue.

[0009] In combination with the first aspect, in certain implementations of the first aspect, after updating the elements in the V2V obstacle tracking queue in the element pair based on the elements in the V2V obstacle queue in the element pair, it also includes: when it is obtained that the obstacle information represented by the updated element in the V2V obstacle tracking queue is wrong, determining whether the position change of the updated element meets the first preset condition; if the position change of the updated element meets the first preset condition, modifying the obstacle information represented by the updated element to be correct.

[0010] In combination with the first aspect, in certain implementations of the first aspect, a single-frame based V2N obstacle queue and a V2N obstacle tracking queue are fused to obtain a new V2N obstacle tracking queue, including: matching elements in the V2N obstacle queue and elements in the V2N obstacle tracking queue to determine whether the two elements in each matched element pair are from the same vehicle-side device; for successfully matched element pairs, updating the elements in the V2N obstacle tracking queue in the element pair based on the elements in the V2N obstacle queue in the element pair; for elements in the V2N obstacle queue that fail to match, if it is determined that the obstacle represented by the element is within the target range of the first vehicle, determining that the obstacle information represented by the element is correct, and adding the element to the V2N obstacle tracking queue; for elements in the V2N obstacle queue that fail to match, if it is determined that the obstacle represented by the element is not within the target range of the first vehicle, determining that the obstacle information represented by the element is incorrect, and adding the element to the V2N obstacle tracking queue.

[0011] In combination with the first aspect, in certain implementations of the first aspect, after updating the elements in the V2N obstacle tracking queue in the element pair based on the elements in the V2N obstacle queue in the element pair, it also includes: when it is obtained that the obstacle information represented by the updated element in the V2N obstacle tracking queue is wrong, determining whether the position change of the updated element meets the second preset condition; if the position change of the updated element meets the second preset condition, modifying the obstacle information represented by the updated element to be correct.

[0012] In combination with the first aspect, in certain implementations of the first aspect, the fourth obstacle queue and the third obstacle queue are fused to obtain a target obstacle queue, including: matching elements in the fourth obstacle queue with elements in the third obstacle queue based on obstacle distance and obstacle overlap information to determine whether two elements in each matched element pair represent the same obstacle; fusing successfully matched element pairs into elements in the target obstacle queue, and updating the number of obstacle matches corresponding to the elements in the third obstacle queue in the element pair.

[0013] In combination with the first aspect, in some implementations of the first aspect, the fourth obstacle queue and the third obstacle queue are merged to obtain a target obstacle queue, further comprising: for elements in the third obstacle queue that fail to match, if it is determined that there is an obstacle whose matching times are greater than the target times and the obstacle is in an unmatched state, storing identification information corresponding to the obstacle in the target container; if there is an obstacle that fails to match and whose identification information is not in the target container, matching the elements in the third obstacle queue that fail to match with the elements in the fourth obstacle queue only according to the obstacle distance, and for elements that successfully match, updating the matching times of the obstacles corresponding to the elements, and adding the unmatched elements to the target obstacle queue; preferably, the method further comprises: filtering out elements in the target obstacle queue that are in the target container or elements that represent incorrect obstacle information.

[0014] In a second aspect, an embodiment of the present application provides an obstacle information processing device, which is applied to a first vehicle, the first vehicle is communicatively connected to at least one second vehicle, and the first vehicle and the at least one second vehicle are respectively communicatively connected to a control end; wherein the device includes: an acquisition module, used to acquire a first obstacle queue, a second obstacle queue and a third obstacle queue of the first vehicle, wherein the first obstacle queue is queue information of obstacles determined based on V2X information sent by the second vehicle to the first vehicle, the second obstacle queue is queue information of obstacles determined based on V2X information sent by the control end to the first vehicle, and the third obstacle queue is queue information of obstacles determined based on perception information collected by a perception device of the first vehicle; a first determination module, used to fuse the first obstacle queue and the second obstacle queue to obtain a fourth obstacle queue; and a second determination module, used to fuse the fourth obstacle queue and the third obstacle queue to obtain a target obstacle queue.

[0015] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program is used to execute the obstacle information processing method described in the first aspect.

[0016] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: a processor; a memory for storing instructions executable by the processor; and the processor is used to execute the obstacle information processing method described in the first aspect.

[0017] In the present application, the first obstacle queue and the second obstacle queue are first fused to obtain the fourth obstacle queue. This process not only integrates the real-time information from other vehicles and the control end, but also eliminates information redundancy through comparison and matching, thereby improving the accuracy of the fourth obstacle queue. Then, the fourth obstacle queue is further fused with the third obstacle queue, so as to utilize the first vehicle's own perception capability to verify and supplement external information (referring to the fourth obstacle queue), and further improve the reliability of obstacle information processing. The multi-level information fusion method of this embodiment effectively overcomes the limitations that may be brought about by a single information source, such as delays, errors, or incompleteness of V2X information, and achieves a more comprehensive understanding of the surrounding environment, thereby reducing the risk of misjudgment and missed judgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0019] Figure 1 Shown is a schematic diagram of the obstacle splitting scenario.

[0020] Figure 2 The figure is a flow chart of a method for processing obstacle information provided in one embodiment of the present application.

[0021] Figure 3 The figure is a schematic diagram of a process of obtaining a fourth obstacle queue provided in an embodiment of the present application.

[0022] Figure 4 The figure is a schematic diagram of a process of obtaining a V2V obstacle tracking queue provided in an embodiment of the present application.

[0023] Figure 5 The figure is a schematic diagram of a process of obtaining a V2N obstacle tracking queue provided in an embodiment of the present application.

[0024] Figure 6 The figure is a schematic diagram of a process of obtaining a target obstacle queue provided in an embodiment of the present application.

[0025] Figure 7 The figure is a schematic diagram of a process of obtaining a target obstacle queue provided by another embodiment of the present application.

[0026] Figure 8 Shown is a schematic diagram of the structure of an obstacle information processing device provided in one embodiment of the present application.

[0027] Fig. 9 Shown is a schematic structural diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] In special scenarios such as mines, it is particularly important to equip vehicles with V2X communication technology. Specifically, V2X communication technology allows vehicles to communicate with other vehicles (V2V) and networks (V2N), thereby realizing real-time information exchange. This information exchange can significantly improve the vehicle's environmental perception ability, enhance the accuracy of obstacle detection, and provide auxiliary decision support in complex or obstructed environments.

[0030] However, although existing autonomous driving technologies emphasize the use and integration of V2X information and give this information a high weight, they ignore the judgment of the accuracy of V2X information and the corresponding processing strategies. In actual operation, the accuracy of V2X information is crucial to the performance of the autonomous driving system. Incorrect or delayed V2X information can lead to misjudgment of the environment, such as misidentifying one obstacle as two (obstacle splitting) or failing to respond to an obstacle ahead in time, which can trigger emergency braking of the autonomous vehicle. This misjudgment not only affects operational efficiency, but also threatens the safety of vehicles and personnel, ultimately leading to economic losses. Figure 1 The following is a schematic diagram of the scene where obstacles are split. Figure 1 As shown, there is actually only obstacle A in lane a, but in the obstacle recognition result, there is also a split obstacle A1 near obstacle A.

[0031] Therefore, in order to improve the obstacle detection accuracy and ensure the driving safety of vehicles, the present application provides a method for processing obstacle information. Specifically, Figure 2 The figure is a flow chart of a method for processing obstacle information provided by an embodiment of the present application. Exemplarily, the method is applied to a first vehicle, the first vehicle is communicatively connected to at least one second vehicle, and the first vehicle and the at least one second vehicle are communicatively connected to a control terminal respectively.

[0032] like Figure 2 As shown, the method includes the following steps.

[0033] Step S210, obtaining a first obstacle queue, a second obstacle queue, and a third obstacle queue of the first vehicle.

[0034] Specifically, the first obstacle queue is the queue information of obstacles determined based on the V2X information sent by the second vehicle to the first vehicle, the second obstacle queue is the queue information of obstacles determined based on the V2X information sent by the control end to the first vehicle, and the third obstacle queue is the queue information of obstacles determined based on the perception information collected by the perception device of the first vehicle.

[0035] Exemplarily, the control end refers to the cloud or data center, which are the central nodes for remotely controlling and managing devices in the network (the first vehicle in this application). The sensing device refers to a device that can obtain information about an object (the first vehicle in this application), for example, a laser radar, a millimeter-wave radar, or a camera. The sensing information collected by the sensing device refers to data collected by the sensor function of the device. These data can be environmental parameters or state information (such as position, speed, acceleration, etc.) of an object (the first vehicle in this application). Exemplarily, the third obstacle queue refers to the queue information of the obstacle obtained by fusing the detection results of the laser radar, the detection results of the millimeter-wave radar, and the detection results of the camera. In addition, in this embodiment, the obstacle queue information refers to a series of obstacle data sets arranged in a specific order, including characteristic information such as the position, speed, size, shape, etc. of the obstacle, as well as identification information of the obstacle, such as the VIN number, etc.

[0036] Step S220: The first obstacle queue and the second obstacle queue are merged to obtain a fourth obstacle queue.

[0037] Exemplarily, during the fusion process, the information in the first obstacle queue and the second obstacle queue will be compared to find matching obstacles. For example, if there is an obstacle A in the first obstacle queue, and its position and speed match the obstacle B in the second obstacle queue, then the two pieces of information will be merged to form a new obstacle information included in the fourth obstacle queue, which contains all relevant data of obstacles A and B and is marked as verified and fused. If there is a mismatch, prediction and inference are performed according to a certain algorithm. For example, obstacle C in the first obstacle queue has no corresponding information in the second obstacle queue, but it is also added to the fourth obstacle queue.

[0038] Ultimately, the fourth obstacle queue will contain all fused obstacle information, which is more comprehensive and accurate, and can provide the first vehicle with more reliable environmental perception to support its safe driving and decision-making.

[0039] Step S230: fusing the fourth obstacle queue and the third obstacle queue to obtain a target obstacle queue.

[0040] As before, the information in the fourth obstacle queue is compared and matched with the information in the third obstacle queue to determine the exact position, speed and characteristics of the obstacle. For example, if an obstacle information in the fourth obstacle queue is very close in position to a perceived obstacle in the third obstacle queue, then the two obstacles will be considered as the same obstacle and their data will be merged to form a more accurate description of the obstacle.

[0041] Optionally, for those obstacles that exist in the fourth obstacle queue but are not detected in the third obstacle queue, these obstacles will be marked as high risk so as to alert the driver or automatically take evasive measures. Conversely, for those information that are detected in the third obstacle queue but not in the fourth obstacle queue, further analysis will be performed to determine whether it is due to the detection range limitation of the perception device or the information delay of the third obstacle queue.

[0042] Ultimately, the target obstacle queue will contain all fused obstacle information, which combines data from vehicle-to-vehicle communication, vehicle-to-control end communication, and the vehicle's own perception devices.

[0043] In this embodiment, the first obstacle queue and the second obstacle queue are first fused to obtain the fourth obstacle queue. This process not only integrates the real-time information from other vehicles and the control end, but also eliminates information redundancy through comparison and matching, thereby improving the accuracy of the fourth obstacle queue. Then, the fourth obstacle queue is further fused with the third obstacle queue, so as to utilize the perception capability of the first vehicle itself to verify and supplement external information (referring to the fourth obstacle queue), and further improve the reliability of obstacle information processing. This multi-level information fusion method of this embodiment effectively overcomes the limitations that may be brought about by a single information source, such as delays, errors or incompleteness of V2X information, and achieves a more comprehensive understanding of the surrounding environment, thereby reducing the risk of misjudgment and missed judgment.

[0044] Figure 3 FIG. 1 is a flow chart of obtaining a fourth obstacle queue according to an embodiment of the present application. Figure 2 Based on the embodiment shown, Figure 3 The embodiment shown is described below in detail. Figure 3 The embodiment shown and Figure 2 The differences and similarities of the illustrated embodiments are not described in detail.

[0045] First, in this embodiment, the first obstacle queue includes a V2V obstacle queue based on a single frame and a V2V obstacle tracking queue based on multi-frame fusion; the second obstacle queue includes a V2N obstacle queue based on a single frame and a V2N obstacle tracking queue based on multi-frame fusion.

[0046] Specifically, a single-frame based V2V obstacle queue refers to a set of obstacle information determined using V2V information received from a second vehicle within a single moment, and the queue reflects the obstacle status of the first vehicle's surrounding environment at a specific moment.

[0047] The V2V obstacle tracking queue based on multi-frame fusion involves analyzing and processing the V2V information received at multiple time points to form continuous tracking information about obstacles. The queue contains the information of obstacles in a single frame, and by matching and associating the obstacle information between consecutive frames, a dynamic, time-varying obstacle tracking model is constructed. The model can provide the movement trajectory of obstacles over time and the ability to predict future positions, allowing the first vehicle to more accurately understand the dynamic changes of the surrounding environment and conduct long-term path planning and safety strategy formulation.

[0048] Corresponding to the above explanation, the V2N obstacle queue based on a single frame refers to a set of obstacle information determined by using the V2N received from the control end in a single moment. The queue reflects the obstacle-related information that the control end can provide at a specific moment. The V2N obstacle tracking queue based on multi-frame fusion involves comprehensive analysis and processing of the V2N information received by the first vehicle at multiple consecutive time points to achieve continuous tracking and status update of obstacles.

[0049] Furthermore, if Figure 3 As shown, the first obstacle queue and the second obstacle queue are merged to obtain a fourth obstacle queue, including the following steps.

[0050] Step S310 , fusing the single-frame-based V2V obstacle queue and the V2V obstacle tracking queue to obtain a new V2V obstacle tracking queue.

[0051] Exemplarily, the V2V obstacle queues are first analyzed, and then these V2V obstacle queues are compared and matched with obstacles in the V2V obstacle tracking queue to determine whether they are the same obstacle. Optionally, the matching process is based on multiple factors such as the position, speed, size and shape of the obstacle. For obstacles that are successfully matched, their tracking information is updated; for obstacles that are not successfully matched, they are added to the V2V obstacle tracking queue as new tracking targets. Finally, through this fusion process, a new V2V obstacle tracking queue can be obtained, which not only contains all obstacles detected in the current frame, but also integrates the historical tracking information of these obstacles.

[0052] Step S320: fusing the single-frame based V2N obstacle queue and the V2N obstacle tracking queue to obtain a new V2N obstacle tracking queue.

[0053] As in the embodiment of step S310, the V2N obstacle queue is first analyzed, and then the V2N obstacle queue is compared and matched with the obstacles in the V2N obstacle tracking queue to determine whether they are the same obstacle. For obstacles that are successfully matched, their tracking information is updated; and for obstacles that are not successfully matched, they are added to the V2N obstacle tracking queue as new tracking targets. Finally, a new V2N obstacle tracking queue is obtained.

[0054] Step S330: Match the elements in the new V2V obstacle tracking queue with the elements in the new V2N obstacle tracking queue.

[0055] The purpose of step S330 is to determine whether the two elements in each matched element pair are from the same vehicle-side device. The vehicle-side device refers to an electronic device installed on the first vehicle for data collection, processing, communication and control, and optionally, the vehicle-side device is a vehicle-mounted sensor or a global positioning system.

[0056] Exemplarily, first ensure that the elements in the new V2V obstacle tracking queue and the new V2N obstacle tracking queue contain sufficient information to identify the vehicle-side device from which they come, for example, each element contains a unique vehicle-side device ID. Then, by comparing the vehicle-side device IDs of the elements in the two queues, it can be determined whether they match. In other words, if the vehicle-side device IDs of the two elements are the same, then they can be considered to come from the same vehicle-side device.

[0057] Optionally, a nearest neighbor search or hash table lookup is used for matching to find and compare the vehicle-side device ID in the new V2V obstacle tracking queue and the new V2N obstacle tracking queue. In addition, in order to improve the accuracy of the matching, the timestamp and spatial location information are also considered to ensure that the elements that are successfully matched are close in time and space, thereby reducing the possibility of mismatching.

[0058] Step S340: Merge the successfully matched element pairs into elements in the fourth obstacle queue.

[0059] Specifically, a successful match indicates that the two elements in the element pair come from the same vehicle-side device.

[0060] Exemplarily, data fusion is performed on the successfully matched element pairs, including integrating the attributes and state information of each element pair, such as position, speed, direction, etc., to form a comprehensive obstacle description. In addition, fusion also involves time synchronization and spatial calibration to ensure that the information from different data sources is consistent in time and space. Finally, the fused data corresponding to each element pair is added to the fourth obstacle queue, providing unified and comprehensive environmental perception data for the decision and control of the first vehicle.

[0061] In this embodiment, a new V2V obstacle tracking queue is generated by fusing a single-frame V2V obstacle queue and a V2V obstacle tracking queue; a new V2N obstacle tracking queue is generated by fusing a single-frame V2N obstacle queue and a V2N obstacle tracking queue. This fusion process can utilize historical tracking data and current detection data, thereby improving the accuracy and continuity of obstacle detection. Then, the new V2V obstacle tracking queue is element-matched with the new V2N obstacle tracking queue to determine whether the matched element pairs are from the same vehicle-side device, thereby ensuring the consistency and reliability of the data and reducing misjudgments caused by data inconsistency between different devices. Finally, the successfully matched element pairs are fused into elements in the fourth obstacle queue, realizing the integration of obstacle information from the same vehicle-side device and improving the first vehicle's ability to identify and track obstacles.

[0062] exist Figure 3 Based on the illustrated embodiment, if it is determined that the distance between two elements in the element pair that failed to match is less than the target distance threshold, the information corresponding to the element pair that failed to match is recorded in the fault information queue for troubleshooting and modification.

[0063] Exemplarily, the method of determining that the distance between two elements in the element pair that failed to match is less than the target distance threshold includes: extracting the spatial position information of each element, including the coordinate data of the element, such as the longitude and latitude or the X, Y, Z coordinates in the Cartesian coordinate system. Then, applying the distance calculation formula to determine the distance between them. For example, in a two-dimensional space, the distance between two elements can be calculated using the Euclidean distance formula Among them, (x1, y1) and (x2, y2) represent the coordinate data of the two elements respectively. If in three-dimensional space, the distance between the two elements is

[0064] The calculated distance value d is compared with the preset target distance threshold. If d is less than the threshold, although they failed to match, the two elements are still considered to be close in space. This comparison can help identify those situations where the matching failure may be caused by sensor errors, data transmission errors, or processing algorithm defects. Furthermore, the corresponding information of these failed matching but close-distance element pairs is recorded in the fault information queue, providing important data support for subsequent maintenance and performance optimization, allowing technicians to troubleshoot and modify these specific situations to improve matching accuracy and overall reliability.

[0065] Figure 4 FIG. 1 is a flow chart of obtaining a V2V obstacle tracking queue according to an embodiment of the present application. Figure 3 Based on the embodiment shown, Figure 4 The embodiment shown is described below in detail. Figure 4 The embodiment shown and Figure 3 The differences and similarities of the illustrated embodiments are not described in detail.

[0066] like Figure 4 As shown, a single-frame-based V2V obstacle queue and a V2V obstacle tracking queue are fused to obtain a new V2V obstacle tracking queue, including the following steps.

[0067] Step S410: Match the elements in the V2V obstacle queue and the elements in the V2V obstacle tracking queue.

[0068] The purpose of step S410 is to determine whether the two elements in each matched element pair are from the same vehicle-side device.

[0069] As mentioned above, based on the characteristic information of the elements, such as timestamp, location coordinates, speed, direction, etc., and the unique identifier of the vehicle-side device associated with each element, it is possible to identify which pairs of elements are matched, that is, they are highly consistent in space and time and have the same device identifier.

[0070] If the match is successful, step S420 is executed to update the elements in the V2V obstacle tracking queue in the element pair based on the elements in the V2V obstacle queue in the element pair for the successfully matched element pair.

[0071] In this step, data of successfully matched element pairs needs to be updated. Specifically, the elements in the V2V obstacle tracking queue will be updated according to the corresponding elements in the V2V obstacle queue.

[0072] Optionally, firstly identify the successfully matched element pairs, and then update the corresponding elements in the V2V obstacle tracking queue according to the latest data of the V2V obstacle queue elements in these element pairs. Exemplarily, the updated data includes key information such as the position, speed, size, direction, etc. of the obstacle. In addition, the update operation needs to consider the timestamp of the data to ensure that the update is based on the latest data.

[0073] In some embodiments, after updating the elements in the V2V obstacle tracking queue in the element pair based on the elements in the V2V obstacle queue in the element pair, it also includes: when it is obtained that the obstacle information represented by the updated element in the V2V obstacle tracking queue is wrong, determining whether the position change of the updated element meets the first preset condition; if the position change of the updated element meets the first preset condition, modifying the obstacle information represented by the updated element to be correct.

[0074] Exemplarily, the situation where the obstacle information represented by the updated element in the V2V obstacle tracking queue is erroneous means that in the process of updating the elements in the V2V obstacle tracking queue, due to sensor errors, data processing errors, environmental interference or algorithm defects, the updated obstacle information is inconsistent with the actual situation, which manifests as a false alarm of key information such as the obstacle's position, speed, and size, thereby affecting the autonomous driving system's correct understanding and response to the surrounding environment.

[0075] Specifically, the first preset condition refers to whether the position change of the obstacle information represented by the updated element between consecutive frames in the V2V obstacle tracking queue conforms to the expected motion law and logic. In particular, this condition requires checking whether the position change of the obstacle represented by the updated element in N consecutive frames remains reasonable, where reasonable means that the movement of the obstacle does not show abnormal jumps or changes that do not conform to the laws of physics, such as sudden high-speed movement or unnatural path turns. Optionally, N is equal to 5.

[0076] If the position change of the updated element meets the first preset condition in N consecutive frames, that is, there is no unreasonable position change in any frame, then the position of the updated element can be considered correct, and it can be determined that the obstacle information represented by the previously updated element is correct. On the contrary, if the position change of the updated element does not meet the first preset condition in N consecutive frames, it will be marked as an error and corresponding measures will be taken, such as re-evaluating the sensor data, adjusting the data processing algorithm, or reminding the driver to pay attention, to ensure the safety and reliability of the autonomous driving system.

[0077] This solution can ensure the continuity and stability of information in the V2V obstacle tracking queue, reduce misjudgments due to accidental sensor errors or data processing errors, and thus provide more reliable environmental perception data for the first vehicle.

[0078] If the matching fails, step S430 is executed to determine whether the obstacle represented by the element in the V2V obstacle queue that failed to match is located within the target range of the first vehicle.

[0079] The target range is an area defined by a specific distance and angle with the first vehicle as the center, which represents an area of ​​the environment that the first vehicle needs to pay special attention to in order to ensure safe driving. For example, the target range is a rectangular area with the first vehicle as the center, extending 20 meters forward, 20 meters backward, 10 meters to the left, and 80 meters to the right.

[0080] In actual operation, whether the obstacle is within the target range of the first vehicle can be determined by the following steps: determining the position of the obstacle represented by the element in the V2V obstacle queue; determining the position of the first vehicle; calculating the distance between the two; and comparing the calculated distance with the target range.

[0081] Exemplarily, if the judgment result of step S430 is yes, step S440 is executed; if the judgment result of step S430 is no, step S450 is executed.

[0082] Step S440: determine whether the obstacle information represented by the element is correct, and add the element to the V2V obstacle tracking queue.

[0083] Step S450: determine that the obstacle information represented by the element is wrong, and add the element to the V2V obstacle tracking queue.

[0084] Specifically, if the obstacle information represented by an element is determined to be correct, the information representing the obstacle is confirmed to be valid, and then the element is added to the V2V obstacle tracking queue for further tracking and analysis. On the contrary, although the obstacle information represented by the element is determined to be wrong, since all detected obstacles need to be recorded and monitored, it still needs to be added to the V2V obstacle tracking queue for subsequent review, error analysis and system improvement.

[0085] In this embodiment, by matching the elements of the V2V obstacle queue and the V2V obstacle tracking queue, it is possible to identify which obstacle information comes from the same vehicle-side device, which helps to ensure the consistency and reliability of the data, thereby improving the accuracy of obstacle tracking. Next, the successfully matched element pairs are updated to ensure the real-time and effectiveness of the obstacle information. In the case of failed matching, it is checked whether these obstacles are within the target range of the first vehicle to ensure that even obstacles that have not been successfully matched are not ignored, especially those that pose a potential threat to the safe driving of the vehicle. Finally, confirming that the correct obstacle information is added to the V2V tracking queue helps maintain an accurate and updated obstacle list to support the decision-making of the first vehicle. However, even obstacle information that is judged to be erroneous will be recorded, which will help with subsequent error analysis and can also keep all potential obstacles under monitoring.

[0086] Figure 5 FIG. 1 is a flow chart of obtaining a V2N obstacle tracking queue according to an embodiment of the present application. Figure 3 Based on the embodiment shown, Figure 5 The embodiment shown is described below in detail. Figure 5 The embodiment shown and Figure 3 The differences and similarities of the illustrated embodiments are not described in detail.

[0087] like Figure 5 As shown, in this embodiment, a fusion process is performed on a single-frame-based V2N obstacle queue and a V2N obstacle tracking queue to obtain a new V2N obstacle tracking queue, including the following steps.

[0088] Step S510, matching the elements in the V2N obstacle queue and the elements in the V2N obstacle tracking queue.

[0089] The purpose of step S510 is to determine whether the two elements in each matched element pair are from the same vehicle-side device. The specific matching method can be found in the above embodiment, which will not be described here.

[0090] If the match is successful, step S520 is executed, and for the successfully matched element pair, the element in the V2N obstacle tracking queue in the element pair is updated based on the element in the V2N obstacle queue in the element pair.

[0091] Corresponding to the embodiment in step S420, in step S520, firstly, the successfully matched element pairs are identified, and then the corresponding elements in the V2N obstacle tracking queue are updated according to the latest data of the V2N obstacle queue elements in these element pairs. Exemplarily, the updated data includes key information such as the position, speed, size, and direction of the obstacle. In addition, the update operation needs to consider the timestamp of the data to ensure that the update is based on the latest data.

[0092] In some embodiments, after updating the elements in the V2N obstacle tracking queue in the element pair based on the elements in the V2N obstacle queue in the element pair, it also includes: when it is obtained that the obstacle information represented by the updated element in the V2N obstacle tracking queue is wrong, determining whether the position change of the updated element meets the second preset condition; if the position change of the updated element meets the second preset condition, modifying the obstacle information represented by the updated element to be correct.

[0093] Exemplarily, the obstacle information represented by the updated element in the V2N obstacle tracking queue is erroneous, which means that in the process of updating the elements in the V2N obstacle tracking queue, due to sensor errors, data processing errors, environmental interference or algorithm defects, the updated obstacle information is inconsistent with the actual situation, which manifests as a false alarm of key information such as the obstacle's position, speed, and size, thereby affecting the autonomous driving system's correct understanding and response to the surrounding environment.

[0094] Similarly, the second preset condition refers to whether the position change of the obstacle information represented by the updated element between consecutive frames in the V2N obstacle tracking queue conforms to the expected motion law and logic. In particular, this condition requires checking whether the position change of the obstacle represented by the updated element in consecutive M frames remains reasonable, where reasonable means that the movement of the obstacle does not show abnormal jumps or changes that do not conform to the laws of physics, such as sudden high-speed movement or unnatural path turns. Optionally, M is equal to 5.

[0095] If the position change of the updated element meets the second preset condition in M ​​consecutive frames, that is, there is no unreasonable position change in any frame, then the position of the updated element can be considered correct, and it can be determined that the obstacle information represented by the previously updated element is correct. On the contrary, if the position change of the updated element does not meet the second preset condition in M ​​consecutive frames, it will be marked as an error and corresponding measures will be taken, such as re-evaluating the sensor data, adjusting the data processing algorithm, or reminding the driver to pay attention, to ensure the safety and reliability of the autonomous driving system.

[0096] This solution can ensure the continuity and stability of information in the V2N obstacle tracking queue, reducing misjudgments caused by accidental sensor errors or data processing errors. By checking multiple consecutive frames, the accuracy of obstacle information can be confirmed with greater confidence, thereby providing more reliable environmental perception data for the first vehicle.

[0097] If the matching fails, step S530 is executed to determine whether the obstacle represented by the element in the V2N obstacle queue that failed to match is located within the target range of the first vehicle.

[0098] Exemplarily, the implementation method of step S530 may refer to the embodiment corresponding to step S430, which will not be described in detail here.

[0099] If the judgment result of step S530 is yes, then step S540 is executed; if the judgment result of step S530 is no, then step S550 is executed.

[0100] Step S540: determine whether the obstacle information represented by the element is correct, and add the element to the V2N obstacle tracking queue.

[0101] Step S550: determine that the obstacle information represented by the element is wrong, and add the element to the V2N obstacle tracking queue.

[0102] Specifically, if the obstacle information represented by an element is determined to be correct, the obstacle information is confirmed to be valid, and then the element is added to the V2N obstacle tracking queue for further tracking and analysis. On the contrary, although the obstacle information represented by the element is determined to be wrong, since all detected obstacles need to be recorded and monitored, it still needs to be added to the V2N obstacle tracking queue for subsequent review, error analysis and system improvement.

[0103] In this embodiment, by matching the elements of the V2N obstacle queue and the V2N obstacle tracking queue, it is possible to identify which obstacle information comes from the same vehicle-side device, which helps to ensure the consistency and reliability of the data, thereby improving the accuracy of obstacle tracking. Next, the successfully matched element pairs are updated to ensure the real-time and effectiveness of the obstacle information. In the case of failed matching, it is checked whether these obstacles are within the target range of the first vehicle to ensure that even obstacles that have not been successfully matched are not ignored, especially those that pose a potential threat to the safe driving of the vehicle. Finally, confirming that the correct obstacle information is added to the V2N tracking queue helps maintain an accurate and updated obstacle list to support the decision-making of the first vehicle. However, even obstacle information that is judged to be erroneous will be recorded, which will help with subsequent error analysis and can also keep all potential obstacles under monitoring.

[0104] Figure 6 FIG. 1 is a flow chart of obtaining a target obstacle queue according to an embodiment of the present application. Figure 2 Based on the embodiment shown, Figure 6 The embodiment shown is described below in detail. Figure 6 The embodiment shown and Figure 2 The differences and similarities of the illustrated embodiments are not described in detail.

[0105] like Figure 6 As shown, in this embodiment, the fourth obstacle queue and the third obstacle queue are fused to obtain a target obstacle queue, including the following steps.

[0106] Step S610: Matching elements in the fourth obstacle queue with elements in the third obstacle queue based on obstacle distance and obstacle overlap information.

[0107] The purpose of step S610 is to determine whether the two elements in each matched element pair represent the same obstacle.

[0108] Specifically, the obstacle distance refers to the distance between the obstacle represented by the element in the fourth obstacle queue and the obstacle represented in the third obstacle queue. The distance is calculated by referring to the distance calculation formula of two obstacles described in the above embodiment.

[0109] Obstacle overlap information refers to the overlapping information of the obstacles represented by the elements in the fourth obstacle queue and the obstacles represented in the third obstacle queue. Exemplarily, the obstacle overlap information is determined by calculating the intersection of the spatial position and size data of the two obstacles. Specifically, each obstacle is represented as a bounding box (for example, a rectangular area), and then the overlapping area of ​​the two bounding boxes is found. The ratio of the area of ​​this overlapping area to the area of ​​any bounding box is regarded as the obstacle overlap information. Optionally, if the ratio exceeds the target overlap threshold and the obstacle distance is less than the target distance threshold, it can be considered that the two obstacles represent the same obstacle, that is, the two obstacles are matched successfully.

[0110] Step S620: merge the successfully matched element pairs into elements in the target obstacle queue, and update the obstacle matching times corresponding to the elements in the third obstacle queue in the element pairs.

[0111] Exemplarily, the successfully matched element pairs are identified, i.e., the element pairs representing the same obstacle in the fourth obstacle queue and the third obstacle queue. Then, the information of the two elements is fused, including their coordinate position, size, speed, direction and other attributes, to form a comprehensive obstacle description, which is added to the corresponding element in the target obstacle queue.

[0112] In addition, while fusing the information, the obstacle matching times corresponding to the elements in the third obstacle queue of the successfully matched element pair will also be updated. The update of this matching times reflects the number of times the obstacle is successfully matched and confirmed, which can be used as an indicator of obstacle tracking stability and reliability.

[0113] In this embodiment, by calculating the obstacle distance and obstacle overlap information, the effective matching of the elements in the fourth obstacle queue and the third obstacle queue is achieved, and then the element pairs representing the same obstacle are identified, reducing the cases of mismatching and missed matching. In addition, through the matching process, the obstacle information from different queues can be associated, providing a consistent and continuous obstacle description for subsequent data processing and analysis. Then, these successfully matched element pairs are merged into elements in the target obstacle queue, and the number of obstacle matching is updated, which provides a quantitative indicator of the obstacle tracking quality, and at the same time, more comprehensive and accurate obstacle information is obtained.

[0114] Figure 7 FIG. 1 is a flow chart of obtaining a target obstacle queue according to another embodiment of the present application. Figure 6 Based on the embodiment shown, Figure 7 The embodiment shown is described below in detail. Figure 7 The embodiment shown and Figure 6The differences and similarities of the illustrated embodiments are not described in detail.

[0115] like Figure 7 As shown, in this embodiment, the fourth obstacle queue and the third obstacle queue are fused to obtain a target obstacle queue, and the following steps are also included.

[0116] Step S710: For the elements in the third obstacle queue that have failed to match, if it is determined that there is an obstacle whose matching times are greater than the target times and the obstacle is in an unmatched state, the identification information corresponding to the obstacle is stored in the target container.

[0117] Exemplarily, the number of matches of each unmatched obstacle is checked. If the number of matches exceeds the set target number and the obstacle is in an unmatched state, that is, after multiple attempts, the obstacle still fails to successfully match any element in the fourth obstacle queue, in this case, it is considered that the probability of the obstacle appearing is high, or it is not correctly identified and matched due to some reason (such as sensor error, environmental change, etc.). Optionally, the target number is 3. Therefore, the identification information of the obstacle, that is, the data that can uniquely identify the obstacle (such as ID, location, timestamp, etc.), is stored in the target container. Optionally, the target container is a data structure for storing and tracking the information of these obstacles that failed to match.

[0118] Step S720: If there is an obstacle that has not been successfully matched and whose identification information is not in the target container, the elements in the third obstacle queue that have not been successfully matched are matched with the elements in the fourth obstacle queue only according to the obstacle distance. For the successfully matched elements, the matching number of the obstacle corresponding to the element is updated, and the unsuccessfully matched elements are added to the target obstacle queue.

[0119] Specifically, if the elements in the third obstacle queue that have not been successfully matched and their identification information are not in the previously determined target container, that is, these obstacles may be new obstacles that have not been recorded before, or obstacles that exist but cannot be identified for some reason, then only the distance information of the obstacles is used to match the elements in the third obstacle queue that have not been successfully matched with the elements in the fourth obstacle queue. This matching method is relatively simple and mainly depends on the spatial proximity between obstacles, that is, if the distance between two obstacles is within a certain preset threshold range, they are considered to be matched.

[0120] In this embodiment, if an obstacle fails to match after multiple attempts, this may be because the obstacle is new or its characteristics do not match the existing tracking model. The identification information of these obstacles is stored in the target container so that these obstacles can be given special attention and subsequent analysis. For those obstacles that are not matched successfully and whose identification information is not in the target container, matching is performed only based on the obstacle distance, and the number of matches is updated for the elements that are matched successfully. This allows for obstacles that are not given special attention to quickly identify possible matches through simple distance matching, and quickly update the information of obstacles that actually exist in the environment but are not immediately identified. For those obstacles that cannot be matched even based on the obstacle distance, they are added to the target obstacle queue for further analysis and processing, which helps to capture obstacles that may not be immediately identified due to environmental changes or sensor limitations.

[0121] exist Figure 7 Based on the illustrated embodiment, the method further includes: filtering out the elements in the target obstacle queue that are in the target container or the elements that indicate that the obstacle information is wrong.

[0122] Exemplarily, each element in the target obstacle queue is checked and compared with the obstacle identification information stored in the target container to determine whether there is a match. If the identification information of an element matches an entry in the target container, or the obstacle information represented by the element is determined to be wrong, then the element will be marked as an object that needs to be filtered out.

[0123] Then, a filtering operation is performed, that is, all marked elements are removed from the target obstacle queue. In this way, only those elements whose identification information is not in the target container and is confirmed to be correct obstacle information are retained in the target obstacle queue, maintaining the accuracy and reliability of the target obstacle queue, ensuring that the target obstacle queue only contains obstacle information that has been verified and is valuable to the decision-making of the autonomous driving system, reducing the interference of erroneous data and redundant information, and improving the accuracy of obstacle tracking.

[0124] Combination of the above Figures 1 to 7 , describes in detail the embodiment of the method for processing obstacle information of the present application, and the following is combined with Figure 8 , describes in detail the embodiment of the obstacle information processing device of the present application. It should be understood that the description of the obstacle information processing method embodiment corresponds to the description of the obstacle information processing device embodiment, so the part not described in detail can refer to the previous method embodiment.

[0125] Figure 8FIG. 1 is a schematic diagram of a device for processing obstacle information provided by an embodiment of the present application. For example, the device is applied to a first vehicle, the first vehicle is connected to at least one second vehicle for communication, and the first vehicle and the at least one second vehicle are connected to a control terminal for communication. Figure 8 As shown, the obstacle information processing device 80 provided in the embodiment of the present application includes:

[0126] The acquisition module 810 is used to acquire a first obstacle queue, a second obstacle queue and a third obstacle queue of the first vehicle, wherein the first obstacle queue is queue information of obstacles determined based on V2X information sent by the second vehicle to the first vehicle, the second obstacle queue is queue information of obstacles determined based on V2X information sent by the control terminal to the first vehicle, and the third obstacle queue is queue information of obstacles determined based on perception information collected by a perception device of the first vehicle;

[0127] A first determining module 820 is used to merge the first obstacle queue and the second obstacle queue to obtain a fourth obstacle queue;

[0128] The second determining module 830 is configured to perform fusion processing on the fourth obstacle queue and the third obstacle queue to obtain a target obstacle queue.

[0129] In an embodiment of the present application, the first obstacle queue includes a V2V obstacle queue based on a single frame and a V2V obstacle tracking queue based on multi-frame fusion; the second obstacle queue includes a V2N obstacle queue based on a single frame and a V2N obstacle tracking queue based on multi-frame fusion; the first determination module 820 is also used to fuse the V2V obstacle queue based on a single frame and the V2V obstacle tracking queue to obtain a new V2V obstacle tracking queue; fuse the V2N obstacle queue based on a single frame and the V2N obstacle tracking queue to obtain a new V2N obstacle tracking queue; match the elements in the new V2V obstacle tracking queue with the elements in the new V2N obstacle tracking queue to determine whether the two elements in each matched element pair are from the same vehicle-side device; fuse the successfully matched element pairs into elements in the fourth obstacle queue, wherein a successful match indicates that the two elements in the element pair are from the same vehicle-side device.

[0130] In one embodiment of the present application, the first determination module 820 is also used to record information corresponding to the element pair that failed to match into a fault information queue for troubleshooting and modification when it is determined that the distance between two elements in the element pair that failed to match is less than a target distance threshold.

[0131] In one embodiment of the present application, the first determination module 820 is further used to match the elements in the V2V obstacle queue and the elements in the V2V obstacle tracking queue to determine whether the two elements in each matched element pair are from the same vehicle-side device; for the successfully matched element pairs, the elements in the V2V obstacle tracking queue in the element pair are updated based on the elements in the V2V obstacle queue in the element pair; for the elements in the V2V obstacle queue that failed to match, if it is determined that the obstacle represented by the element is within the target range of the first vehicle, it is determined that the obstacle information represented by the element is correct, and the element is added to the V2V obstacle tracking queue; for the elements in the V2V obstacle queue that failed to match, if it is determined that the obstacle represented by the element is not within the target range of the first vehicle, it is determined that the obstacle information represented by the element is wrong, and the element is added to the V2V obstacle tracking queue.

[0132] In one embodiment of the present application, the first determination module 820 is also used to determine whether the position change of the updated element meets the first preset condition when it is obtained that the obstacle information represented by the updated element in the V2V obstacle tracking queue is wrong; if the position change of the updated element meets the first preset condition, the obstacle information represented by the updated element is modified to be correct.

[0133] In one embodiment of the present application, the first determination module 820 is also used to match the elements in the V2N obstacle queue and the elements in the V2N obstacle tracking queue to determine whether the two elements in each matched element pair are from the same vehicle-side device; for the element pairs that are successfully matched, the elements in the V2N obstacle tracking queue in the element pair are updated based on the elements in the V2N obstacle queue in the element pair; for the elements in the V2N obstacle queue that fail to match, if it is determined that the obstacle represented by the element is within the target range of the first vehicle, it is determined that the obstacle information represented by the element is correct, and the element is added to the V2N obstacle tracking queue; for the elements in the V2N obstacle queue that fail to match, if it is determined that the obstacle represented by the element is not within the target range of the first vehicle, it is determined that the obstacle information represented by the element is incorrect, and the element is added to the V2N obstacle tracking queue.

[0134] In one embodiment of the present application, the first determination module 820 is also used to determine whether the position change of the updated element meets the second preset condition when it is obtained that the obstacle information represented by the updated element in the V2N obstacle tracking queue is wrong; if the position change of the updated element meets the second preset condition, the obstacle information represented by the updated element is modified to be correct.

[0135] In one embodiment of the present application, the second determination module 830 is further used to match the elements in the fourth obstacle queue with the elements in the third obstacle queue based on the obstacle distance and obstacle overlap information to determine whether the two elements in each matched element pair represent the same obstacle; merge the successfully matched element pairs into elements in the target obstacle queue, and update the obstacle matching count corresponding to the elements in the third obstacle queue in the element pair.

[0136] In an embodiment of the present application, the second determination module 830 is further used for, for elements in the third obstacle queue that have failed to match, if it is determined that there is an obstacle whose matching times are greater than the target times and the obstacle is in an unmatched state, then storing the identification information corresponding to the obstacle in the target container; if there is an obstacle that has not been matched successfully and whose identification information is not in the target container, matching the elements in the third obstacle queue that have not been matched successfully with the elements in the fourth obstacle queue only according to the obstacle distance, and for the elements that have been matched successfully, updating the matching times of the obstacles corresponding to the elements, and adding the unmatched elements to the target obstacle queue; and filtering out the elements in the target obstacle queue that are in the target container or the elements that represent incorrect obstacle information.

[0137] Below, reference Fig. 9 To describe an electronic device according to an embodiment of the present application. Fig. 9 Shown is a schematic structural diagram of an electronic device provided by an exemplary embodiment of the present application.

[0138] like Fig. 9 As shown, the electronic device 90 includes one or more processors 901 and a memory 902 .

[0139] The processor 901 may be a central processing unit (CPU) or other forms of processing units having the ability to process obstacle information and / or the ability to execute instructions, and may control other components in the electronic device 90 to perform desired functions.

[0140] The memory 902 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 901 may run the program instructions to implement the obstacle information processing method of each embodiment of the present application described above and / or other desired functions. Various contents such as the first obstacle queue, the second obstacle queue, the third obstacle queue, the fourth obstacle queue, the target obstacle queue, etc. may also be stored in the computer-readable storage medium.

[0141] In one example, the electronic device 90 may further include: an input device 903 and an output device 904 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0142] The input device 903 may include, for example, a keyboard, a mouse, etc.

[0143] The output device 904 can output various information to the outside, including the first obstacle queue, the second obstacle queue, the third obstacle queue, the fourth obstacle queue, the target obstacle queue, etc. The output device 904 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, etc.

[0144] Of course, to simplify, Fig. 9 Only some of the components related to the present application in the electronic device 90 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 90 may also include any other appropriate components.

[0145] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the obstacle information processing method according to various embodiments of the present application described above in this specification.

[0146] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0147] In addition, an embodiment of the present application may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the method for processing obstacle information according to various embodiments of the present application described above in this specification.

[0148] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0149] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.

[0150] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0151] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0152] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0153] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for processing obstacle information, characterized in that: Applied to a first vehicle, the first vehicle is communicatively connected to at least one second vehicle, and the first vehicle and the at least one second vehicle are communicatively connected to a control end respectively; wherein the method comprises: Acquire a first obstacle queue, a second obstacle queue, and a third obstacle queue of the first vehicle, wherein the first obstacle queue is queue information of obstacles determined based on V2X information sent by the second vehicle to the first vehicle, the second obstacle queue is queue information of obstacles determined based on V2X information sent by the control end to the first vehicle, and the third obstacle queue is queue information of obstacles determined based on perception information collected by a perception device of the first vehicle; Performing fusion processing on the first obstacle queue and the second obstacle queue to obtain a fourth obstacle queue; The fourth obstacle queue and the third obstacle queue are fused to obtain a target obstacle queue.

2. The method for processing obstacle information according to claim 1, characterized in that: The first obstacle queue includes a V2V obstacle queue based on a single frame and a V2V obstacle tracking queue based on multi-frame fusion; the second obstacle queue includes a V2N obstacle queue based on a single frame and a V2N obstacle tracking queue based on multi-frame fusion; The fusing the first obstacle queue and the second obstacle queue to obtain a fourth obstacle queue includes: Performing a fusion process on the single-frame based V2V obstacle queue and the V2V obstacle tracking queue to obtain a new V2V obstacle tracking queue; Performing fusion processing on the single-frame based V2N obstacle queue and the V2N obstacle tracking queue to obtain a new V2N obstacle tracking queue; Matching the elements in the new V2V obstacle tracking queue with the elements in the new V2N obstacle tracking queue to determine whether the two elements in each matched element pair are from the same vehicle-side device; The successfully matched element pairs are merged into the elements in the fourth obstacle queue, wherein the successful matching indicates that the two elements in the element pair are from the same vehicle-side device.

3. The method for processing obstacle information according to claim 2, characterized in that: Also includes: When it is determined that the distance between two elements in the element pair that failed to match is less than the target distance threshold, information corresponding to the element pair that failed to match is recorded in a fault information queue for troubleshooting and modification.

4. The method for processing obstacle information according to claim 2, characterized in that: The fusing process of the single-frame based V2V obstacle queue and the V2V obstacle tracking queue to obtain a new V2V obstacle tracking queue includes: Matching the elements in the V2V obstacle queue and the elements in the V2V obstacle tracking queue to determine whether two elements in each matched element pair are from the same vehicle-side device; For a successfully matched element pair, updating the element in the V2V obstacle tracking queue in the element pair based on the element in the V2V obstacle queue in the element pair; For an element in the V2V obstacle queue that fails to match, if it is determined that the obstacle represented by the element is located within the target range of the first vehicle, determining that the obstacle information represented by the element is correct, and adding the element to the V2V obstacle tracking queue; For an element in the V2V obstacle queue that fails to match, if it is determined that the obstacle represented by the element is not within the target range of the first vehicle, it is determined that the obstacle information represented by the element is wrong, and the element is added to the V2V obstacle tracking queue.

5. The method for processing obstacle information according to claim 4, characterized in that: After updating the element in the V2V obstacle tracking queue in the element pair based on the element in the V2V obstacle queue in the element pair, the method further includes: In a case where it is obtained that the obstacle information represented by the updated element in the V2V obstacle tracking queue is wrong, determining whether the position change of the updated element meets a first preset condition; If the position change of the updated element meets the first preset condition, the obstacle information represented by the updated element is modified to be correct.

6. The method for processing obstacle information according to claim 2, characterized in that: The fusing the single-frame-based V2N obstacle queue and the V2N obstacle tracking queue to obtain a new V2N obstacle tracking queue includes: Matching the elements in the V2N obstacle queue and the elements in the V2N obstacle tracking queue to determine whether two elements in each matched element pair are from the same vehicle-side device; For a successfully matched element pair, updating the element in the V2N obstacle tracking queue in the element pair based on the element in the V2N obstacle queue in the element pair; For an element in the V2N obstacle queue that fails to match, if it is determined that the obstacle represented by the element is located within the target range of the first vehicle, determining that the obstacle information represented by the element is correct, and adding the element to the V2N obstacle tracking queue; For an element in the V2N obstacle queue that fails to match, if it is determined that the obstacle represented by the element is not within the target range of the first vehicle, it is determined that the obstacle information represented by the element is wrong, and the element is added to the V2N obstacle tracking queue.

7. The method for processing obstacle information according to claim 6, characterized in that: After updating the element in the V2N obstacle tracking queue in the element pair based on the element in the V2N obstacle queue in the element pair, the method further includes: In a case where it is obtained that the obstacle information represented by the updated element in the V2N obstacle tracking queue is wrong, determining whether the position change of the updated element meets a second preset condition; If the position change of the updated element meets the second preset condition, the obstacle information represented by the updated element is modified to be correct.

8. The method for processing obstacle information according to any one of claims 1 to 7, characterized in that: The fusing the fourth obstacle queue and the third obstacle queue to obtain a target obstacle queue includes: Based on the obstacle distance and the obstacle overlap information, matching the elements in the fourth obstacle queue with the elements in the third obstacle queue to determine whether two elements in each matched element pair represent the same obstacle; The successfully matched element pairs are merged into the elements in the target obstacle queue, and the obstacle matching times corresponding to the elements in the third obstacle queue in the element pairs are updated.

9. The method for processing obstacle information according to claim 8, characterized in that: The fusing the fourth obstacle queue and the third obstacle queue to obtain a target obstacle queue further includes: For the elements in the third obstacle queue that have failed to match, if it is determined that there is an obstacle whose matching times are greater than the target times and the obstacle is in an unmatched state, the identification information corresponding to the obstacle is stored in the target container; If there is an obstacle that has not been successfully matched and whose identification information is not in the target container, the elements in the third obstacle queue that have not been successfully matched are matched with the elements in the fourth obstacle queue only according to the obstacle distance, and for the successfully matched elements, the matching count of the obstacle corresponding to the element is updated, and the unsuccessfully matched elements are added to the target obstacle queue; Preferably, the method further comprises: Elements in the target obstacle queue that are in the target container or elements that represent incorrect obstacle information are filtered out.

10. An obstacle information processing device, characterized in that: Applied to a first vehicle, the first vehicle is communicatively connected to at least one second vehicle, and the first vehicle and the at least one second vehicle are communicatively connected to a control end respectively; Wherein, the device comprises: an acquisition module, configured to acquire a first obstacle queue, a second obstacle queue, and a third obstacle queue of the first vehicle, wherein the first obstacle queue is queue information of obstacles determined based on V2X information sent by the second vehicle to the first vehicle, the second obstacle queue is queue information of obstacles determined based on V2X information sent by the control end to the first vehicle, and the third obstacle queue is queue information of obstacles determined based on perception detection information collected by a perception device of the first vehicle; a first determining module, configured to perform fusion processing on the first obstacle queue and the second obstacle queue to obtain a fourth obstacle queue; The second determining module is used to perform fusion processing on the fourth obstacle queue and the third obstacle queue to obtain a target obstacle queue.

11. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the obstacle information processing method according to any one of claims 1 to 9.

12. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is used to execute the obstacle information processing method described in any one of claims 1 to 9 above.

Images