A method and device for obstacle avoidance in a high-speed construction scene, equipment and medium

Abstract

The application discloses a kind of obstacle avoidance methods, devices, equipment and medium for high-speed construction scene.It comprises: in response to vehicle entering high-speed construction area, multi-source data acquisition is carried out by event camera, image collector and radar, and the fusion characteristic parameter of target is obtained;The fusion characteristic parameter is matched with the distributed obstacle feature library to determine whether the target is a distributed obstacle;If yes, the relative parameter of vehicle and target is determined;According to the relative parameter and the current lane environment data, generate avoidance decision instruction, to control vehicle to execute avoidance action.The present application realizes the accurate perception and rapid identification of the distributed obstacle such as irregular construction waste, damaged cone barrel and irregular cement pier in construction area by multi-sensor fusion and distributed obstacle feature library, improves the traffic safety and system reliability of autonomous vehicle in complex construction section.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to an obstacle avoidance method, device, equipment and medium for high-speed construction scenarios. Background Technology

[0002] With the development of autonomous driving technology, L2+ and above level vehicles have achieved relatively mature autonomous driving capabilities on structured highways. However, highway construction zones, as typical semi-structured and complex road scenarios, represent a weak link in the perception and decision-making of autonomous driving systems.

[0003] Current mainstream autonomous driving perception systems are generally built on deep learning models, which rely on the consistency between the training dataset and the distribution of the actual scene. However, targets such as irregularly shaped cement barriers, deformed and damaged cones, and scattered construction waste in highway construction areas are typical out-of-distribution (OOD) obstacles. Their shape, optical reflection characteristics, and spatial arrangement are significantly different from those of conventional road obstacles and training samples. This directly leads to problems such as semantic segmentation misjudgment, target omission, and delayed recognition response in the traditional perception combination based on frame cameras and LiDAR. The perception accuracy drops significantly and cannot provide reliable data support for subsequent avoidance decisions. Summary of the Invention

[0004] This application provides an obstacle avoidance method, device, equipment, and medium for high-speed construction scenarios. The method uses multi-sensor fusion and a distributed external obstacle feature library to enable autonomous vehicles to accurately perceive and quickly identify distributed external obstacles such as irregularly shaped cement blocks, damaged cones, and irregular construction waste in the construction area while driving at high speeds, thereby improving the traffic safety and system reliability of autonomous vehicles in complex construction sections.

[0005] According to one aspect of this application, an obstacle avoidance method for high-speed construction scenarios is provided, the method comprising: In response to a vehicle entering a highway construction area, data is collected from the road ahead using an event camera, image acquisition device, and radar to obtain fused feature parameters of the target within the highway construction area. The fused feature parameters are matched in an off-distribution obstacle feature library to determine whether the target is an off-distribution obstacle; If the target is an external obstacle, then determine the relative parameters between the vehicle and the target; Based on the relative parameters and the lane environment data where the vehicle is currently located, an avoidance decision command is generated to control the vehicle to perform corresponding avoidance actions according to the avoidance decision command.

[0006] According to another aspect of this application, an obstacle avoidance device for high-speed construction scenarios is provided, characterized in that the device comprises: The data fusion module is used to respond to vehicles entering the highway construction area by collecting data on the road ahead through event cameras, image acquisition devices and radar, and obtaining the fusion feature parameters of targets in the highway construction area; An obstacle recognition module is used to match the fused feature parameters in an off-distribution obstacle feature library to determine whether the target is an off-distribution obstacle. A relative parameter determination module is used to determine the relative parameters between the vehicle and the target if the target is an externally distributed obstacle. The decision planning module is used to generate avoidance decision instructions based on the relative parameters and the lane environment data where the vehicle is currently located, so as to control the vehicle to perform corresponding avoidance actions according to the avoidance decision instructions.

[0007] According to another aspect of this application, an electronic device is provided, the device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the obstacle avoidance method for high-speed construction scenarios as described in any embodiment of this application.

[0008] According to another aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions, the computer instructions being configured to cause a processor to execute and implement the obstacle avoidance method for high-speed construction scenarios as described in any embodiment of this application.

[0009] According to another aspect of this application, a computer program product is provided, the computer program product including a computer program that, when executed by a processor, implements the obstacle avoidance method for high-speed construction scenarios described in any embodiment of this application.

[0010] The technical solution provided in this application, in response to a vehicle entering a high-speed construction area, acquires multi-source data based on event cameras, image acquisition devices, and radar, and obtains the fused feature parameters of the target. These fused feature parameters are then matched with a distributed obstacle feature library to determine whether the target is an external obstacle. If so, the relative parameters between the vehicle and the target are determined. Based on the relative parameters and current lane environment data, an avoidance decision command is generated to control the vehicle to perform an avoidance maneuver. This invention, through multi-sensor fusion and a distributed obstacle feature library, enables autonomous vehicles to accurately perceive and quickly identify external obstacles such as irregularly shaped concrete blocks, damaged cones, and irregular construction waste in construction areas while traveling at high speeds, thus improving the safety and reliability of autonomous vehicles in complex construction sections.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart of an obstacle avoidance method for high-speed construction scenarios provided in Embodiment 1 of this application.

[0014] Figure 2 This is a flowchart of an obstacle avoidance method for high-speed construction scenarios provided in Embodiment 2 of this application.

[0015] Figure 3 This is a structural schematic diagram of an obstacle avoidance device for high-speed construction scenarios provided in Embodiment 3 of the present invention.

[0016] Figure 4 This is a structural schematic diagram of a device for implementing an obstacle avoidance method for high-speed construction scenarios according to an embodiment of this application. Detailed Implementation

[0017] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0018] It should be noted that the terms "first," "second," "target," "relative," and "current," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0019] It should also be noted that the acquisition, storage, use, and processing of data in the technical solution of this application all comply with the relevant provisions of national laws and regulations.

[0020] Example 1 Figure 1 This is a flowchart of an obstacle avoidance method for high-speed construction scenarios provided in Embodiment 1 of this application. This embodiment is applicable to situations where autonomous vehicles identify obstacles in high-speed construction scenarios. The method can be executed by an obstacle avoidance device for high-speed construction scenarios. The obstacle avoidance device for high-speed construction scenarios can be implemented in hardware and / or software and can be configured in a device with data processing capabilities.

[0021] like Figure 1 As shown, the method includes the following steps.

[0022] S110. In response to a vehicle entering a high-speed construction area, data is collected from the road ahead using an event camera, image acquisition device, and radar to obtain fused feature parameters of targets within the high-speed construction area.

[0023] In highway construction areas, event cameras, image acquisition devices, and radar mounted on vehicles can simultaneously collect road environment data ahead. Combined with data from the vehicle's GPS positioning module, multi-dimensional feature markers of the highway construction area can be extracted, such as construction warning signs, temporary lane lines, cone clusters, and construction area barriers, to determine whether a vehicle has entered the highway construction area.

[0024] If the vehicle is determined to have entered a highway construction area, the distributed obstacle recognition and avoidance cooperative control mode will be activated immediately, and vehicle speed pre-control will be implemented to limit the vehicle's movement to a preset proportion of the current lane's legal speed limit, in order to avoid the risk of exceeding the performance inflection point of the AEB automatic emergency braking system and ensure the effectiveness of subsequent braking and avoidance actions; if the vehicle is determined not to have entered a highway construction area, the vehicle can continue to maintain the normal automatic driving mode and will not trigger the special cooperative control logic.

[0025] An event camera is a motion capture camera whose pixels operate independently. It captures sudden changes in brightness and dynamic contours within the environment and outputs signals asynchronously, i.e., an event stream. Event cameras have high-speed response characteristics, with a single-frame signal response time of ≤30ms, enabling millisecond-level rapid locking of abnormal targets and distinguishing between static irregular obstacles and dynamic sudden obstacles.

[0026] In this application, the image acquisition device can be a high-definition camera used to acquire image frame data of the road environment ahead, in order to extract information such as target contour, size, and texture features. Of course, the image acquisition device can also be a binocular stereo camera, etc. The radar can be a millimeter-wave radar, which detects the relative distance, relative speed, and azimuth angle data of the target by transmitting and receiving radio waves. Of course, the radar can also be ultrasonic radar, lidar, etc. This application does not specifically limit this aspect.

[0027] Specifically, data can be collected from the road ahead using event cameras, image acquisition devices, and radar, and after spatiotemporal alignment and correlation matching, fused feature parameters of targets within the highway construction area can be generated. It should be noted that multiple targets within the highway construction area can be identified simultaneously.

[0028] In some embodiments, optionally, before acquiring data from the road ahead using an event camera, an image acquisition device, and a radar to obtain the fusion feature parameters of the target within the highway construction area, the method further includes: acquiring the operating status of the event camera, the image acquisition device, and the radar; and when the operating status of any one of the event camera, the image acquisition device, and the radar is invalid, acquiring the fusion feature parameters of the target within the highway construction area based on the data acquired from the road ahead by the other two sensors, and triggering an early warning action.

[0029] Operating status refers to the health status of the sensor, including whether it is powered on, whether the signal output is normal, and whether the data quality (such as whether it is blocked or the signal strength) is within a reasonable range.

[0030] Specifically, by continuously monitoring the heartbeat signals and data validity of the three sensors (such as whether the event stream is stagnant for a long time, whether the image is completely black / white, and whether the number of radar point clouds is abnormal), if the monitoring indicators of a certain sensor continuously exceed the threshold, it is determined to be faulty. Then, it switches to a dual-sensor redundancy fusion mode, relying on the other two normal sensors to maintain basic perception capabilities and triggering in-vehicle warning and cloud-based warning actions.

[0031] For example, if the image acquisition device fails, a fusion perception scheme using an event camera and radar is activated. The event camera provides the target's dynamic contour and motion trend, while the radar provides the target's precise distance, radial velocity, and azimuth. A general obstacle feature library is used for identification and avoidance, and a warning is triggered to alert the driver. If the event camera fails, a fusion perception scheme using an image acquisition device and radar is activated. The image acquisition device provides the target's appearance, texture, and semantic category, while the radar provides the target's precise distance and velocity. Since the image acquisition device's frame rate is much lower than the event camera's, a conservative deceleration strategy can be implemented to compensate for the reduced dynamic response capability, and a warning is triggered to alert the driver. If the radar fails, a fusion perception scheme using an event camera and image sensor is activated. The event camera provides the target's asynchronous dynamic contour and extremely high temporal resolution motion information, while the image acquisition device provides the target's high-resolution texture, color, and semantic details. However, the perception accuracy for target collision time decreases, so the maximum vehicle speed can be reduced and the warning enhanced to compensate for the risk of inaccurate ranging, and a warning is triggered to alert the driver.

[0032] In some embodiments, the warning action may optionally include at least one of limiting the maximum speed of the vehicle to a predetermined value or increasing the frequency of in-vehicle alarms.

[0033] The predetermined value can be determined by multiplying the maximum reliable sensing distance when all event cameras, image acquisition units, and radars are operating normally by a degradation factor. The degradation factor can be determined based on the type of sensor that failed. For example, if the event camera fails, the system's dynamic response slows down, and the degradation factor can be set to 0.8-0.9; if the image sensor fails, the system loses its ability to recognize details, and the degradation factor can be set to 0.7-0.8; if the radar fails, the system loses its accurate ranging capability and relies on visual distance estimation, which has a large error, and the degradation factor can be set to 0.4-0.6.

[0034] Increasing the frequency of in-vehicle alarms refers to enhancing the level of alertness to the driver through the frequency and / or intensity of visual warning icons / text on the dashboard and head-up display, as well as audible alarms (such as beeps and voice prompts).

[0035] For example, the maximum vehicle speed can be reduced to 50 km / h, increasing the frequency of in-vehicle alarms and continuously reminding the driver to pay attention to road conditions and take over the vehicle at any time, thus comprehensively ensuring driving safety in extreme failure scenarios and preventing safety accidents caused by perception failure.

[0036] S120. The fused feature parameters are matched in the feature library of external obstacles to determine whether the target is an external obstacle.

[0037] The Out-of-Distribution (OOD) Obstacle Feature Library is a pre-built digital database that stores standardized feature parameter templates for common OOD obstacles (such as irregularly shaped concrete blocks, damaged cones, construction waste piles, and temporary irregular barriers) in high-speed construction areas. Each template represents the average feature value or feature distribution range of a typical out-of-distribution obstacle.

[0038] Specifically, the fused feature parameters can be sequentially compared with each template in the out-of-distribution obstacle feature library for similarity calculation. A similarity threshold (e.g., 95%) is set. If the similarity between the target's fused feature parameters and any template in the out-of-distribution obstacle feature library exceeds this threshold, the target is determined to be a known type of out-of-distribution obstacle. The final output matching result can include: 1) whether the target is an out-of-distribution obstacle; 2) if so, its most likely category; 3) the matching confidence.

[0039] S130. If the target is an external obstacle, then determine the relative parameters between the vehicle and the target.

[0040] Relative parameters refer to the relative physical quantities relating the kinematic relationship and collision risk between a vehicle and an off-center obstacle. Typically, relative parameters include relative distance, relative speed, and collision time. Relative distance is the straight-line distance between the vehicle and the target. Relative speed is the approach speed of the vehicle and the target along the line connecting them. Collision time is the ratio of relative distance to relative speed.

[0041] Specifically, the relative distance and relative velocity are directly provided by the radar with high-precision measurement values, and are supplemented and verified by the image acquisition unit. The collision time is calculated in real time based on the relative distance and relative velocity reported by the radar.

[0042] S140. Based on the relative parameters and the lane environment data where the vehicle is currently located, generate an avoidance decision command to control the vehicle to perform corresponding avoidance actions according to the avoidance decision command.

[0043] Lane environment data refers to the status of the lane in which a vehicle is located and information about the surrounding passable space. It typically includes: lane width, occupancy status of adjacent lanes (e.g., whether there are vehicles and their distance / speed), road edge type (e.g., guardrails, construction barriers), and temporary road markings.

[0044] Specifically, the collision risk between the vehicle and the target can be determined based on relative parameters, and the variable lane space of the lane to the side of the vehicle can be determined based on the lane environment data of the current lane of the vehicle; thus, avoidance decision instructions can be generated based on the collision risk and the variable lane space.

[0045] Furthermore, the avoidance decision command is simultaneously sent to the vehicle's braking control unit, steering control unit, and power control unit to achieve coordinated action of multiple actuators, ensuring smooth and continuous operation such as deceleration and trajectory correction. During execution, real-time data on vehicle driving status (such as vehicle speed, steering angle, braking torque, and yaw rate) and the real-time position data of the target are collected. If the target position dynamically deviates, the avoidance strategy and execution intensity are immediately adjusted in real time to ensure the effectiveness of hazard avoidance. After the vehicle has completely left the construction area, the system automatically exits the cooperative control mode and resumes the normal autonomous driving state. At the same time, the entire process data of this construction area passage, external obstacle recognition, and avoidance execution is uploaded to the cloud server, and the local external obstacle feature library is iteratively updated to continuously optimize the subsequent recognition accuracy and decision rationality.

[0046] In some embodiments, the relative parameter may optionally include the relative distance between the vehicle and the target; The step of generating an avoidance decision instruction based on the relative parameters and the lane environment data currently in which the vehicle is located includes, but is not limited to, the following steps S141-S143.

[0047] S141. If the relative distance is greater than or equal to the first warning distance, and it is determined based on the lane environment data that the side lane has safe lane-changing conditions, then a first avoidance decision command is generated to control the vehicle to decelerate and make lateral corrections.

[0048] Safe lane change conditions can refer to the following conditions being met by the adjacent lane: 1) The lane line is a dashed line, allowing lane change; 2) There are no vehicles in the target lane, or the distance between adjacent vehicles is far enough and the relative speed is stable to ensure that no collision will occur during the lane change process; 3) There are no other obstacles in front of the target lane.

[0049] Lateral correction refers to making a small lateral adjustment within the lane away from the obstacle (e.g., a slight adjustment to the right of 0.2 meters) without crossing the lane lines, in order to increase the lateral distance from the obstacle, as a preparation or supplement before changing lanes.

[0050] In related technologies, when vehicles are traveling at high speeds of 90-120 km / h, the system is limited by the imitation learning decision-making logic. When encountering sudden obstacles, it tends to prioritize steering and avoidance operations, which violates the core safety principle of giving way to speed rather than lanes in high-speed driving. This significantly increases the risk of vehicle skidding, collisions with temporary isolation facilities, and scrapes and collisions with vehicles in adjacent lanes, which can easily lead to secondary accidents. Moreover, this hidden danger is further amplified in scenarios where lanes narrow in construction areas and road conditions change abruptly.

[0051] Therefore, this application follows the core safety principle of giving way to speed rather than lane in high-speed driving, and formulates differentiated avoidance strategies in a tiered manner to completely avoid the risk of loss of control caused by sharp turns at high speeds.

[0052] Specifically, when the relative distance between the vehicle and the target is greater than or equal to the first warning distance, and when it is determined based on the lane environment data that the lateral lane has safe lane-changing conditions, a combined avoidance strategy of slow gradient deceleration and small lane correction is implemented. For example, the vehicle speed can be smoothly reduced to below 60 km / h, the lateral lane correction range can not exceed 0.5m, the vehicle's driving trajectory can be kept stable, and there can be no aggressive steering actions.

[0053] The first avoidance decision instruction typically includes two control objectives: a small deceleration value (e.g., -1 m / s²). 2 ), and lateral displacement offset.

[0054] S142. If the relative distance is less than the first warning distance and greater than or equal to the second warning distance, a second avoidance decision command is generated to control the vehicle to decelerate and maintain its lane.

[0055] When the relative distance between the vehicle and the target is less than the first warning distance but greater than or equal to the second warning distance, regardless of whether there are vehicles in the side lane, the vehicle should abandon steering to avoid the target, prioritize graded emergency deceleration, activate the AEB automatic emergency braking system, and quickly reduce the vehicle speed to a certain speed (such as 40km / h). The vehicle should maintain its current lane throughout the process and avoid any steering operations to prevent the vehicle from skidding or colliding with adjacent facilities or vehicles.

[0056] The second avoidance decision instruction typically includes two control objectives: a large deceleration value (such as triggering AEB comfort braking or partial braking, -3 to -5 m / s²). 2 ), and the lateral displacement offset is 0.

[0057] S143. If the relative distance is less than the second warning distance, a third avoidance decision command that triggers the maximum braking force of the vehicle is generated.

[0058] When the relative distance between the vehicle and the target is less than the second warning distance, the maximum braking force is immediately triggered for emergency braking, and the multi-level audible and visual alarms in the vehicle are activated simultaneously to forcibly remind the driver to take over control of the vehicle. If the driver does not take over within 3 seconds, the system will automatically maintain the maximum braking state and continue to decelerate until the vehicle comes to a complete stop, minimizing the risk of collision.

[0059] The third avoidance decision command typically includes two control objectives: maximum braking deceleration (e.g., -8 m / s²). 2 (or larger), and accompanied by audible and visual alarms to alert the driver to take emergency control.

[0060] The first and second warning distances can be fixed values, for example, the first warning distance is 50 meters and the second warning distance is 30 meters.

[0061] In some embodiments, the relative parameters may optionally include the relative speed between the vehicle and the target; the first warning distance and the second warning distance are determined at least based on the vehicle's current speed, the relative speed, and the road surface adhesion coefficient.

[0062] Understandably, a safe driving distance is the distance a vehicle travels from detecting an obstacle to coming to a complete stop, ensuring that the distance traveled is less than the initial distance between the vehicle and the obstacle. The safe distance includes the constant-speed travel distance during the system's reaction time and the constant-deceleration braking distance after the braking system engages. The braking distance directly depends on the current vehicle speed, relative speed, and the maximum coefficient of friction (i.e., road surface adhesion coefficient) that the road surface can provide.

[0063] Therefore, in this application, the first warning distance and the second warning distance can be determined by the vehicle's current speed, relative speed, and road surface adhesion coefficient. For example, they can be calculated using empirical formulas.

[0064] The advantage of the above technical solution is that, based on the three-level response system of dynamic safety distance, it achieves a smooth transition from vehicle avoidance to emergency braking, thereby improving the vehicle's environmental adaptability and reliability.

[0065] This invention provides an obstacle avoidance method for high-speed construction scenarios. In response to a vehicle entering a high-speed construction area, the method acquires multi-source data using an event camera, image acquisition device, and radar to obtain fused feature parameters of the target. These fused feature parameters are then matched with a feature library of external obstacles to determine if the target is an external obstacle. If so, the relative parameters between the vehicle and the target are determined. Based on the relative parameters and current lane environment data, an avoidance decision command is generated to control the vehicle to perform an avoidance maneuver. This invention, through multi-sensor fusion and an external obstacle feature library, enables autonomous vehicles to accurately perceive and quickly identify external obstacles such as irregularly shaped concrete blocks, damaged cones, and irregular construction waste in construction areas while traveling at high speeds, improving the safety and reliability of autonomous vehicles in complex construction sections.

[0066] Example 2 Figure 2 This is a flowchart of an obstacle avoidance method for high-speed construction scenarios provided in Embodiment 2 of this application. This embodiment is based on the above embodiment and is optimized, specifically by optimizing the feature fusion process.

[0067] like Figure 2 As shown, the method includes the following steps.

[0068] S210: In response to a vehicle entering a highway construction area, the event stream of the road ahead is asynchronously collected by the event camera.

[0069] An event stream is a sequence of events continuously generated by an event camera and arranged in chronological order. Each event contains pixel coordinates, a timestamp, and the polarity of the brightness change.

[0070] Specifically, each pixel in the event camera independently monitors logarithmic brightness changes. Once the change exceeds a threshold, the pixel immediately outputs an event to the data bus. These asynchronous events are continuously received and buffered to form a real-time event stream.

[0071] S220. In response to the event stream reaching a preset condition in a specific area, determine that there is a target in the road ahead and generate a spatiotemporal attention signal containing the spatiotemporal information of the target.

[0072] Preset conditions can be preset density or specific spatiotemporal patterns.

[0073] Preset density refers to the number of events per unit time and per unit pixel area. For example, a fast-moving small target (such as a bird) or a suddenly appearing large target (such as a falling object) will cause the local event density to increase sharply.

[0074] A specific spatiotemporal pattern refers to the characteristics of an event in terms of spatial distribution and temporal evolution. For example, a walking person may generate a cluster of events with specific periodicity and directionality.

[0075] Spatiotemporal attention signals are signals that contain spatiotemporal information about the target, which may include the spatiotemporal attention area, target category confidence, direction of motion, etc.

[0076] Specifically, real-time clustering analysis can be performed on event streams to identify spatiotemporally continuous event clusters; the event density, spatial distribution, motion trajectory, and other features of each cluster can be calculated; the features can be compared with a preset abnormal pattern library; if a match is successful, the cluster is determined to be a potential target; and spatiotemporal attention signals can be generated based on the spatiotemporal range of the cluster.

[0077] S230. Based on the spatiotemporal attention signal, data is collected through an image sensor and radar to obtain image data and radar point cloud data of the target.

[0078] Specifically, image sensors and radar collect data to obtain image data and radar point cloud data of the road ahead. Based on the spatiotemporal attention area indicated by the spatiotemporal attention signal, the collected image data and radar point cloud data are extracted to obtain the image data and radar point cloud data of the target.

[0079] S240. The event stream, the image data, and the radar point cloud data are spatiotemporally synchronized and correlated to generate fusion feature parameters of the target.

[0080] Since the event camera, image acquisition device and radar have different acquisition frame rates and transmission delays, this application needs to perform spatiotemporal alignment of the event stream, image data and radar point cloud data of the relevant target before fusion processing.

[0081] Time synchronization can be achieved by using a unified hardware clock source to provide globally synchronized timestamps for event cameras, image acquisition units, and radar. The data collected by the event cameras, image acquisition units, and radar are assigned a high-precision time stamp by this clock source when they are generated. Alternatively, the timestamps of the event cameras can be used as a reference to align the time of the image acquisition units and radar using interpolation.

[0082] Spatial synchronization can determine the relative position and attitude relationship between the event camera, image acquisition device and radar based on the spatial calibration parameters of the radar.

[0083] After spatiotemporally aligning event streams, image data, and radar point cloud data, they are fused to generate fused feature parameters of the target. For example, within a cropped region of interest, appearance and texture features (such as color, shape, and semantic features) of the target can be extracted from the image data; geometric and motion features (such as 3D size, point cloud distribution, and radial velocity) can be extracted from the radar point cloud data; and the above heterogeneous features can be fused into a unified high-dimensional feature vector, i.e., fused feature parameters, through a feature fusion network.

[0084] In some embodiments, optionally, the step of spatiotemporally synchronizing and associating the event stream, the image data, and the radar point cloud data to generate fusion feature parameters of the target includes at least, but is not limited to, the following steps S241-S242.

[0085] S241. Based on the Hungarian correlation algorithm, the image data and the radar point cloud data are synchronized and aligned in time using the timestamp of the event stream as a reference.

[0086] Specifically, the process can be based on cached event streams, image frames corresponding to image data, and radar frames corresponding to radar point cloud data. For each image frame, a content consistency cost matrix is ​​constructed between it and multiple candidate time points in the event stream. Similarly, for each radar frame, a content consistency cost matrix is ​​constructed between it and multiple candidate time points in the event stream. The cost matrices are solved using the Hungarian association algorithm to match the most suitable corresponding time point in the event stream for each image frame and each radar frame. Based on the matching results, the image frames and radar frames are assigned synchronization timestamps aligned with the event stream.

[0087] S242. Based on the spatial region of interest indicated by the spatiotemporal attention signal, the image data, the radar point cloud data, and the event stream are correlated and matched to generate fusion feature parameters of the target in the same spatiotemporal coordinate system.

[0088] Spatial attention area refers to the three-dimensional spatial range indication of a target in a spatiotemporal attention signal.

[0089] Specifically, by using pre-calibrated sensor extrinsic parameters, all image pixels and radar point clouds after time alignment can be converted to the vehicle coordinate system. Based on the coordinate range of the spatial region of interest, data belonging only to the target area can be cropped from the converted image feature map and radar point cloud and then matched to generate fused feature parameters of the target in the same spatiotemporal coordinate system.

[0090] The advantage of the above technical solution is that it can achieve spatiotemporal synchronization and alignment of three-source sensing data, eliminating data deviations caused by transmission delays and differences in acquisition frame rates between different sensors.

[0091] S250. The fused feature parameters are matched in the feature library of external obstacles to determine whether the target is an external obstacle.

[0092] S260. If the target is an external obstacle, then determine the relative parameters between the vehicle and the target.

[0093] S270. Based on the relative parameters and the lane environment data where the vehicle is currently located, generate an avoidance decision command to control the vehicle to perform corresponding avoidance actions according to the avoidance decision command.

[0094] This embodiment provides an obstacle avoidance method for high-speed construction scenarios. The method achieves millisecond-level anomaly triggering through an event camera, guides on-demand and directional acquisition of images and radar, and constructs an asynchronous fusion perception architecture of event stream, image, and point cloud, which significantly improves the real-time perception capability and overall energy efficiency of sudden obstacles in high-speed construction areas.

[0095] Example 3 Figure 3 This is a structural schematic diagram of an obstacle avoidance device for high-speed construction scenarios provided in Embodiment 3 of the present invention. Figure 3 As shown, the device includes: The data fusion module 310 is used to collect data on the road ahead through an event camera, an image acquisition device and radar in response to a vehicle entering a high-speed construction area, and to obtain the fusion feature parameters of the target in the high-speed construction area. The obstacle recognition module 320 is used to match the fused feature parameters in the off-distribution obstacle feature library to determine whether the target is an off-distribution obstacle. The relative parameter determination module 330 is used to determine the relative parameters between the vehicle and the target if the target is an external obstacle. The decision planning module 340 is used to generate avoidance decision instructions based on the relative parameters and the lane environment data where the vehicle is currently located, so as to control the vehicle to perform corresponding avoidance actions according to the avoidance decision instructions.

[0096] This invention provides an obstacle avoidance device for high-speed construction scenarios. In response to a vehicle entering a high-speed construction area, the device collects multi-source data using an event camera, image acquisition unit, and radar, acquiring fused feature parameters of the target. These fused feature parameters are then matched with a feature library of external obstacles to determine if the target is an external obstacle. If so, the relative parameters between the vehicle and the target are determined. Based on the relative parameters and current lane environment data, an avoidance decision command is generated to control the vehicle to perform an avoidance maneuver. This invention, through multi-sensor fusion and an external obstacle feature library, enables autonomous vehicles to accurately perceive and quickly identify external obstacles such as irregularly shaped concrete blocks, damaged cones, and irregular construction waste in construction areas while traveling at high speeds, improving the safety and reliability of autonomous vehicles in complex construction sections.

[0097] Furthermore, the data fusion module 310 includes: The event stream acquisition unit is used to asynchronously acquire the event stream of the road ahead via the event camera; A spatiotemporal attention signal generation unit is used to determine the existence of a target in the road ahead and generate a spatiotemporal attention signal containing spatiotemporal information of the target in response to the event flow reaching a preset condition in a specific area. The data synchronization acquisition unit is used to acquire image data and radar point cloud data of the target based on the spatiotemporal attention signal through image sensors and radar. The feature parameter fusion unit is used to perform spatiotemporal synchronization and correlation of the event stream, the image data, and the radar point cloud data to generate fused feature parameters of the target.

[0098] Furthermore, the feature parameter fusion unit includes: The time alignment subunit is used to perform time synchronization alignment of the image data and the radar point cloud data based on the Hungarian correlation algorithm and the timestamp of the event stream. The association matching subunit is used to associate and match the image data, the radar point cloud data and the event stream based on the spatial interest area indicated by the spatiotemporal attention signal, and generate fusion feature parameters of the target in the same spatiotemporal coordinate system.

[0099] Furthermore, the relative parameters include the relative distance between the vehicle and the target; The decision planning module 340 includes: The first instruction generation unit is used to generate a first avoidance decision instruction to control the vehicle to decelerate and make lateral corrections if the relative distance is greater than or equal to the first warning distance and the lane environment data determines that the side lane has safe lane-changing conditions. The second instruction generation unit is used to generate a second avoidance decision instruction to control the vehicle to decelerate and maintain its lane if the relative distance is less than the first warning distance and greater than or equal to the second warning distance. The third instruction generation unit is used to generate a third avoidance decision instruction that triggers the maximum braking force of the vehicle if the relative distance is less than the second warning distance.

[0100] Furthermore, the relative parameters also include the relative speed between the vehicle and the target; The first warning distance and the second warning distance are determined at least based on the vehicle's current speed, the relative speed, and the road surface adhesion coefficient.

[0101] Furthermore, the device also includes: The working status acquisition module is used to acquire the working status of the event camera, the image acquisition device and the radar before the data acquisition of the road ahead through the event camera, the image acquisition device and the radar is carried out to obtain the fusion feature parameters of the target in the high-speed construction area. The dual-sensor fusion module is used to obtain the fused feature parameters of the target in the highway construction area based on the data collected by the other two sensors on the road ahead when any one of the event camera, the image acquisition device and the radar fails to function, and to trigger an early warning action.

[0102] Furthermore, the warning action includes at least one of limiting the vehicle's maximum speed to a predetermined value or increasing the frequency of in-vehicle alarms.

[0103] The obstacle avoidance device for high-speed construction scenarios provided in the embodiments of the present invention can execute the obstacle avoidance method for high-speed construction scenarios provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method.

[0104] Example 4 Figure 4 A schematic diagram of the structure of a device 10 that can be used to implement embodiments of this application is shown. The device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0105] like Figure 4As shown, device 10 includes at least one processor 11 and a memory, such as read-only memory (ROM) 12, random access memory (RAM) 13, etc., communicatively connected to at least one processor 11. The memory stores computer programs executable by at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded into the RAM 13 from storage unit 18. The RAM 13 may also store various programs and data required for the operation of device 10. The processor 11, ROM 12, and RAM 13 are interconnected via bus 14. Input / output (I / O) interface 15 is also connected to bus 14.

[0106] Multiple components in device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of monitors, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0107] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as obstacle avoidance methods for high-speed construction scenarios.

[0108] In some embodiments, the obstacle avoidance method for high-speed construction scenarios can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the obstacle avoidance method for high-speed construction scenarios described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the obstacle avoidance method for high-speed construction scenarios by any other suitable means (e.g., by means of firmware).

[0109] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0110] Computer programs used to implement the methods of this application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0111] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0112] To provide interaction with a user, the systems and techniques described herein can be implemented on a device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0113] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0114] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0115] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication unit 19, or installed from storage unit 18, or installed from ROM 12. When the computer program is executed by processor 11, it performs the functions defined in the methods of the embodiments of the present invention.

[0116] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved, and this is not limited herein.

[0117] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. An obstacle avoidance method for high-speed construction scenarios, characterized in that, include: In response to a vehicle entering a high-speed construction area, data is collected from the road ahead using an event camera, image acquisition device, and radar to obtain fused feature parameters of the target within the high-speed construction area; The fused feature parameters are matched in an off-distribution obstacle feature library to determine whether the target is an off-distribution obstacle; If the target is an external obstacle, then determine the relative parameters between the vehicle and the target; Based on the relative parameters and the lane environment data where the vehicle is currently located, an avoidance decision command is generated to control the vehicle to perform corresponding avoidance actions according to the avoidance decision command.

2. The obstacle avoidance method for high-speed construction scenarios according to claim 1, characterized in that, The process of acquiring data about the road environment ahead using event cameras, image acquisition devices, and radar to obtain fused feature parameters of targets within the highway construction area includes: Event streams on the road ahead are asynchronously captured using an event camera; In response to the event stream reaching a preset condition in a specific area, it is determined that there is a target in the road ahead and a spatiotemporal attention signal containing spatiotemporal information of the target is generated; Based on the spatiotemporal attention signal, data acquisition is performed through image sensors and radar to obtain image data and radar point cloud data of the target; The event stream, the image data, and the radar point cloud data are spatiotemporally synchronized and correlated to generate fusion feature parameters of the target.

3. The obstacle avoidance method for high-speed construction scenarios according to claim 2, characterized in that, The step of synchronizing and correlating the event stream, the image data, and the radar point cloud data in time and space to generate fused feature parameters of the target includes: Based on the Hungarian correlation algorithm, the image data and the radar point cloud data are synchronized and aligned in time using the timestamp of the event stream as a reference. Based on the spatial region of interest indicated by the spatiotemporal attention signal, the image data, the radar point cloud data, and the event stream are correlated and matched to generate fusion feature parameters of the target in the same spatiotemporal coordinate system.

4. The obstacle avoidance method for high-speed construction scenarios according to claim 1, characterized in that, The relative parameters include the relative distance between the vehicle and the target; The step of generating an avoidance decision instruction based on the relative parameters and the lane environment data currently in which the vehicle is located includes: If the relative distance is greater than or equal to the first warning distance, and it is determined based on the lane environment data that the side lane has safe lane-changing conditions, then a first avoidance decision command is generated to control the vehicle to decelerate and make lateral corrections. If the relative distance is less than the first warning distance and greater than or equal to the second warning distance, a second avoidance decision command is generated to control the vehicle to decelerate and maintain its lane. If the relative distance is less than the second warning distance, a third avoidance decision command is generated to trigger the vehicle's maximum braking force.

5. The obstacle avoidance method for high-speed construction scenarios according to claim 4, characterized in that, The relative parameters also include the relative speed between the vehicle and the target; The first warning distance and the second warning distance are determined at least based on the vehicle's current speed, the relative speed, and the road surface adhesion coefficient.

6. The obstacle avoidance method for high-speed construction scenarios according to claim 1, characterized in that, Before acquiring data from the road ahead using event cameras, image acquisition devices, and radar to obtain the fused feature parameters of targets within the highway construction area, the method further includes: Acquire the operating status of the event camera, the image acquisition device, and the radar; When any of the sensors in the event camera, the image acquisition unit, and the radar fails, the fused feature parameters of the target in the highway construction area are obtained based on the data collected by the other two sensors on the road ahead, and an early warning action is triggered.

7. The obstacle avoidance method for high-speed construction scenarios according to claim 6, characterized in that, The warning action includes at least one of limiting the vehicle's maximum speed to a predetermined value or increasing the frequency of in-vehicle alarms.

8. An obstacle avoidance device for high-speed construction scenarios, characterized in that, The device includes: The data fusion module is used to respond to vehicles entering the highway construction area by collecting data on the road ahead through event cameras, image acquisition devices and radar, and obtaining the fusion feature parameters of targets in the highway construction area; An obstacle recognition module is used to match the fused feature parameters in an off-distribution obstacle feature library to determine whether the target is an off-distribution obstacle. A relative parameter determination module is used to determine the relative parameters between the vehicle and the target if the target is an externally distributed obstacle. The decision planning module is used to generate avoidance decision instructions based on the relative parameters and the lane environment data where the vehicle is currently located, so as to control the vehicle to perform corresponding avoidance actions according to the avoidance decision instructions.

9. An electronic device, characterized in that, The device includes: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the obstacle avoidance method for high-speed construction scenarios according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the obstacle avoidance method for high-speed construction scenarios as described in any one of claims 1-7.

Images