Self-adaptive cruise control method and device, electronic equipment and storage medium

Abstract

The invention provides a self-adaptive cruise control method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining visual sensor information and millimeter wave radar information of a vehicle in response to a starting operation of a vehicle self-adaptive cruise function; wherein the visual sensor information at least comprises lane line information; performing screening processing on the millimeter wave radar information according to the lane line information to determine target millimeter wave radar information; performing information fusion processing on the visual sensor information and the target millimeter wave radar information to generate fused target information; therefore, according to the fusion target information, adaptive cruise control is carried out on the vehicle, adaptive cruise control is carried out according to the fusion target information of the visual sensor information and the target millimeter wave radar information, and the limitation caused by the fact that related technologies mainly depend on a visual sensor is overcome; and the adaptability and robustness of the adaptive cruise system under various environmental conditions are obviously improved.

Description

Technical Field

[0001] The present invention relates to the field of adaptive cruise technology, and in particular to an adaptive cruise control method, device, electronic equipment and storage medium. Background Art

[0002] With the continuous development of intelligent automobile technology, the intelligent adaptive cruise control (IACC) system has become an important technology in the modern automobile field. It can automatically adjust parameters such as vehicle speed or direction according to the detection results of the adaptive cruise target, thereby ensuring vehicle driving safety.

[0003] In related technologies, intelligent adaptive cruise control systems mainly rely on visual information to identify adaptive cruise targets, thereby providing accurate target position, speed, and type information, so that the back-end vehicle control algorithm can clearly know the motion properties of the adaptive cruise target, and then achieve accurate following, lane changing, deceleration, braking, etc. This method is simple to implement, only requires a certain amount of engineering experience, and can be quickly developed and deployed.

[0004] However, in practical applications, visual sensors are generally monocular and lack image depth information, so longitudinal distance measurement often has large errors. At the same time, the camera may fail to output, have poor vision in rainy and foggy weather, or have short detection time at night. Traditional fusion algorithms still use visual information to a large extent in the above situations, which greatly reduces the detection accuracy of adaptive cruise targets. Summary of the invention

[0005] The embodiments of the present invention provide an adaptive cruise control method, device, electronic device and storage medium, so as to at least solve the problem that the intelligent adaptive cruise system in the related art mainly relies on visual information to identify the adaptive cruise target, lacks image depth information, and thus the longitudinal distance measurement often has a large error, and the camera may have a malfunction and cannot output, the field of view is poor in rainy and foggy weather, and the detection is short at night; the traditional fusion algorithm will still use visual information to a large extent in the above situation, resulting in a significant reduction in the detection accuracy of the adaptive cruise target.

[0006] An embodiment of the present invention discloses an adaptive cruise control method, the method comprising:

[0007] In response to a start-up operation of the vehicle adaptive cruise function, obtaining visual sensor information and millimeter wave radar information of the vehicle; wherein the visual sensor information at least includes lane line information;

[0008] Filtering the millimeter-wave radar information according to the lane line information to determine target millimeter-wave radar information;

[0009] Performing information fusion processing on the visual sensor information and the target millimeter wave radar information to generate fused target information;

[0010] Adaptive cruise control is performed on the vehicle according to the fused target information.

[0011] Optionally, the screening the millimeter-wave radar information according to the lane line information to determine the target millimeter-wave radar information includes:

[0012] Determine, according to the lane line information, an adaptive cruise target that is in the same lane as the vehicle and has the shortest distance from the vehicle;

[0013] The millimeter-wave radar information corresponding to the adaptive cruise target is determined as the target millimeter-wave radar information.

[0014] Optionally, the performing information fusion processing on the visual sensor information and the target millimeter wave radar information to generate fused target information includes:

[0015] Determine the frame number data and the lateral position variance data of the target millimeter-wave radar information, and perform radar data stability judgment processing on the target millimeter-wave radar information according to the frame number data and the lateral position variance data;

[0016] If the result of the radar data stability judgment processing is qualified, the visual sensor information and the target millimeter-wave radar information are subjected to the information fusion processing to generate the fused target information.

[0017] Optionally, performing radar data stability judgment processing on the target millimeter-wave radar information according to the frame number data and the lateral position variance data includes:

[0018] Determine whether the frame number data is greater than a first preset threshold, and whether the lateral position variance data is less than a second preset threshold;

[0019] If the frame number data is greater than the first preset threshold, and the lateral position variance data is less than the second preset threshold, it is determined that the processing result of the radar data stability determination processing is qualified.

[0020] Optionally, determining the millimeter-wave radar information corresponding to the adaptive cruise target as the target millimeter-wave radar information includes:

[0021] Acquire historical millimeter-wave radar information corresponding to the adaptive cruise target, and perform position stability comparison processing on the millimeter-wave radar information corresponding to the adaptive cruise target and the historical millimeter-wave radar information;

[0022] If the processing result of the position stability comparison processing is qualified, the millimeter-wave radar information corresponding to the adaptive cruise target is determined as the target millimeter-wave radar information.

[0023] Optionally, the performing information fusion processing on the visual sensor information and the target millimeter wave radar information to generate fused target information includes:

[0024] The body sensor information of the vehicle is acquired, and the visual sensor information, the target millimeter-wave radar information and the body sensor information are subjected to the information fusion processing to generate the fusion target information.

[0025] Optionally, before performing adaptive cruise control on the vehicle according to the fused target information, the method further includes:

[0026] Perform rationality judgment processing on the fusion target information.

[0027] The embodiment of the present invention further discloses an adaptive cruise control device, the device comprising:

[0028] An information acquisition module, configured to acquire visual sensor information and millimeter wave radar information of the vehicle in response to a start-up operation of the vehicle adaptive cruise function; wherein the visual sensor information at least includes lane line information;

[0029] A screening and processing module, used to screen the millimeter-wave radar information according to the lane line information to determine the target millimeter-wave radar information;

[0030] A fusion target information generation module is used to perform information fusion processing on the visual sensor information and the target millimeter wave radar information to generate fusion target information;

[0031] An adaptive cruise control module is used to perform adaptive cruise control on the vehicle according to the fused target information.

[0032] Optionally, the screening processing module includes:

[0033] An adaptive cruise target determination submodule, used to determine, based on the lane line information, an adaptive cruise target that is in the same lane as the vehicle and has the shortest distance from the vehicle;

[0034] The target millimeter-wave radar information determination submodule is used to determine the millimeter-wave radar information corresponding to the adaptive cruise target as the target millimeter-wave radar information.

[0035] Optionally, the fusion target information generation module includes:

[0036] A radar data stability judgment submodule, used to determine the frame number data and the lateral position variance data of the target millimeter-wave radar information, and perform radar data stability judgment processing on the target millimeter-wave radar information according to the frame number data and the lateral position variance data;

[0037] The fused target information generating submodule is used to perform the information fusion processing on the visual sensor information and the target millimeter-wave radar information to generate the fused target information if the processing result of the radar data stability judgment processing is qualified.

[0038] Optionally, the radar data stability judgment submodule includes:

[0039] A threshold judgment unit, used to judge whether the frame number data is greater than a first preset threshold, and whether the horizontal position variance data is less than a second preset threshold;

[0040] The radar data stability result determination unit is used to determine that the processing result of the radar data stability determination processing is qualified if the frame number data is greater than the first preset threshold and the lateral position variance data is less than the second preset threshold.

[0041] Optionally, the target millimeter-wave radar information determination submodule includes:

[0042] a position stability comparison unit, used to obtain the historical millimeter-wave radar information corresponding to the adaptive cruise target, and perform position stability comparison processing on the millimeter-wave radar information corresponding to the adaptive cruise target and the historical millimeter-wave radar information;

[0043] The target millimeter-wave radar information determining unit is used to determine the millimeter-wave radar information corresponding to the adaptive cruise target as the target millimeter-wave radar information if the processing result of the position stability comparison processing is qualified.

[0044] Optionally, the fusion target information generation module includes:

[0045] The body sensor information fusion module is used to obtain the body sensor information of the vehicle, and perform the information fusion processing on the visual sensor information, the target millimeter wave radar information and the body sensor information to generate the fused target information.

[0046] Optionally, the device further comprises:

[0047] The rationality judgment processing module is used to perform rationality judgment processing on the fusion target information.

[0048] The embodiment of the present invention further discloses an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0049] The memory is used to store computer programs;

[0050] The processor is used to implement the method described in the embodiment of the present invention when executing the program stored in the memory.

[0051] The embodiment of the present invention further discloses one or more computer-readable media on which instructions are stored. When executed by one or more processors, the processors are enabled to execute the method described in the embodiment of the present invention.

[0052] The embodiments of the present invention include the following advantages:

[0053] In an embodiment of the present invention, in response to a start-up operation of the vehicle adaptive cruise function, visual sensor information and millimeter-wave radar information of the vehicle are obtained; wherein the visual sensor information at least includes lane line information; then the millimeter-wave radar information is screened and processed according to the lane line information to determine the target millimeter-wave radar information; and the visual sensor information and the target millimeter-wave radar information are fused to generate fused target information; thereby, the vehicle is adaptively cruise controlled according to the fused target information, and adaptive cruise control is realized according to the fused target information of the visual sensor information and the target millimeter-wave radar information, and the millimeter-wave radar information with the most fusion value can be selected through the lane line information for fusion to perform adaptive cruise target recognition, so that the adaptive cruise target recognition is no longer completely dependent on the visual information, and the limitation brought by the main reliance on the visual sensor in the related technology is overcome; it is ensured that even when the camera function is limited, the function of the IACC system can be maintained through the millimeter-wave radar information, which significantly improves the adaptability and robustness of the adaptive cruise system under various environmental conditions, and further improves the control effect of the adaptive cruise. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a flowchart of the steps of an adaptive cruise control method provided in an embodiment of the present invention;

[0055] Figure 2 It is a schematic diagram of the overall framework of the algorithm provided in the embodiment of the present invention;

[0056] Figure 3 is a structural block diagram of an adaptive cruise control device provided in an embodiment of the present invention;

[0057] Figure 4 is a block diagram of an electronic device provided in an embodiment of the present invention;

[0058] Figure 5 is a schematic diagram of a computer-readable medium provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0060] With the continuous development of smart car technology, the intelligent adaptive cruise system has become an important technology in the modern automobile field. It can automatically adjust parameters such as vehicle speed or direction according to the detection results of the adaptive cruise target, thereby ensuring vehicle driving safety.

[0061] In related technologies, intelligent adaptive cruise control systems mainly rely on visual information to identify adaptive cruise targets, thereby providing accurate target position, speed, and type information, so that the back-end vehicle control algorithm can clearly know the motion properties of the adaptive cruise target, and then achieve accurate following, lane changing, deceleration, braking, etc. This method is simple to implement, only requires a certain amount of engineering experience, and can be quickly developed and deployed.

[0062] However, in practical applications, visual sensors are generally monocular and lack image depth information, so longitudinal distance measurement often has large errors. At the same time, the camera may fail to output, have poor vision in rainy and foggy weather, or have short detection time at night. Traditional fusion algorithms still use visual information to a large extent in the above situations, which greatly reduces the detection accuracy of adaptive cruise targets.

[0063] In order to solve the above problems, the present invention is based on the core technical concept of performing adaptive cruise control based on the fusion target information of the visual sensor information and the target millimeter-wave radar information, and improves the adaptive cruise control method in the related art. The present invention will be described in detail in conjunction with the accompanying drawings as follows:

[0064] Reference Figure 1 , shows a flowchart of a method for adaptive cruise control provided in an embodiment of the present invention, which may specifically include the following steps:

[0065] Step 101, in response to a start-up operation of an adaptive cruise control function of a vehicle, obtaining visual sensor information and millimeter wave radar information of the vehicle; wherein the visual sensor information at least includes lane line information;

[0066] In an embodiment of the present invention, the visual sensor can be an 800W pixel variable focal length camera, which can identify the adaptive cruise target and output target-level information with target attributes as basic units. The camera target recognition algorithm can use the horizon J5 (Journey 5, Journey 5 chip) BEV (Bird's-Eye-View) framework data results, obtain the vehicle body data through a pre-defined data interface for target calculation, and send the calculation results to the subsequent pre-processing module through a predetermined protocol for use. Specifically, the main function of the 800W pixel camera can be to detect all vehicles, pedestrians and other adaptive cruise targets within a range of 120° in front. Realize the conventional recognition of adaptive cruise targets such as target vehicles, pedestrians, animals, and cyclists on structured roads. The target recognition algorithm can be output via Ethernet, and the receiving end can receive it according to the defined data structure. The target output information (visual sensor information) can include: position, speed, length and width, tracking number, heading information, target type and other information. At the same time, the lane line information at the corresponding time can also be synchronously output, which can be specifically given in the form of cubic curves.

[0067] Furthermore, the millimeter-wave radar can detect targets within a 120° range in front of the vehicle using the Doppler effect, that is, the greater the speed difference between the target and the vehicle in the radial direction, the clearer the detection and the more accurate the output. The millimeter-wave radar can use the bus communication interface to obtain the speed and heading information of the vehicle and calibrate the output object attributes in real time. Since RCS (Radar Cross Section) is used for target identification and tracking, the millimeter-wave radar information is more obviously affected by the environment. For example, the lateral position information and speed information generally have large errors, while the longitudinal position information is more accurate, and the stationary target can only output a point without size. At the same time, the millimeter-wave radar will provide the horizontal and longitudinal position variance information of each output target in real time. The size of this data can intuitively represent the accuracy of the radar's recognition of the target at this time. The larger the variance, the greater the information position fluctuation of the target in the previous few frames or even more than ten frames as of the current time, which indirectly reflects that the radar is interfered with or the target itself is blocked, resulting in signal attenuation. Similarly, the smaller the variance, the more accurate the target position recognition and the higher the credibility.

[0068] It should be emphasized that millimeter wave radar can detect the point cloud information of the target in front and to the side through the Doppler effect, and obtain the target information through the internal point cloud clustering algorithm. Radar sensors can output point clouds and clustered target results at the same time, and transmit them to the corresponding back-end processing module for use according to the pre-defined protocol.

[0069] Furthermore, after obtaining the sensor information data, it can be preprocessed through a pre-set information preprocessing module. Specifically, the sensor usually sends out raw information such as target position, speed, type, etc., but due to different types and models, the sending format and expression of the above information are different. Therefore, the same data format needs to be pre-defined within the program, and the received raw signal is assigned to the corresponding internal data variable according to the content, so as to unify the data format and achieve software and hardware isolation. At the same time, all sensor data are timestamped with the vehicle system time for time synchronization. Since each sensor is installed at a different position on the vehicle body, it uses a different coordinate system for detecting the target. Therefore, the information preprocessing module can also move the measurement point of each sensor to the midpoint of the front bumper of the vehicle by rotation and translation for spatial synchronization.

[0070] Step 102, screening the millimeter-wave radar information according to the lane line information to determine target millimeter-wave radar information;

[0071] In a specific implementation, lane line information can be used to determine the relationship between each adaptive cruise target and the lane line. For example, the adaptive cruise target that is in the lane and closest to the vehicle can be used as a high-risk processing target, and the millimeter-wave radar information corresponding to the high-risk processing target can be determined as the target millimeter-wave radar information for subsequent fusion steps.

[0072] In some embodiments of the present invention, the screening of the millimeter-wave radar information according to the lane line information to determine the target millimeter-wave radar information includes:

[0073] Determine, according to the lane line information, an adaptive cruise target that is in the same lane as the vehicle and has the shortest distance from the vehicle;

[0074] The millimeter-wave radar information corresponding to the adaptive cruise target is determined as the target millimeter-wave radar information.

[0075] In practical applications, the lane line information can be used to set the adaptive cruise target that is in the lane and closest to the vehicle as a high-risk processing target, and the millimeter-wave radar information corresponding to the high-risk processing target can be determined as the target millimeter-wave radar information for subsequent fusion steps.

[0076] In some embodiments of the present invention, determining the millimeter-wave radar information corresponding to the adaptive cruise target as the target millimeter-wave radar information includes:

[0077] Acquire historical millimeter-wave radar information corresponding to the adaptive cruise target, and perform position stability comparison processing on the millimeter-wave radar information corresponding to the adaptive cruise target and the historical millimeter-wave radar information;

[0078] If the processing result of the position stability comparison processing is qualified, the millimeter-wave radar information corresponding to the adaptive cruise target is determined as the target millimeter-wave radar information.

[0079] In a specific implementation, the previous frame of historical information of the millimeter-wave radar information corresponding to the adaptive cruise target can be further utilized to perform a position stability comparison process with the current millimeter-wave radar information. For example, the first lateral distance between the adaptive cruise target corresponding to the historical information and the lane line and the second lateral distance between the adaptive cruise target corresponding to the current millimeter-wave radar information and the lane line can be determined. When the difference between the first lateral distance and the second lateral distance is less than a preset distance threshold, it is considered that the millimeter-wave radar is in a stable state and is not disturbed, that is, the processing result of the position stability comparison process is qualified, so that the current millimeter-wave radar information can be determined as the target millimeter-wave radar information for subsequent fusion operations.

[0080] Furthermore, the timestamp and position information corresponding to the initial stable moment of the millimeter-wave radar can be recorded at the same time as the subsequent judgment reference. For example, when the entire algorithm cycle runs to the next time, the new adaptive cruise target that is in the same lane as the vehicle and has the shortest distance from the vehicle can be compared with the timestamp and position information corresponding to the new adaptive cruise target after the aforementioned historical information comparison process. For example, the millimeter-wave radar information corresponding to the new adaptive cruise target can be subsequently fused only after the time interval change (i.e., the time difference calculated according to the timestamp) or the longitudinal distance change (i.e., the longitudinal distance difference calculated according to the position information) is greater than the corresponding preset threshold. In this way, it can be intuitively characterized that after the new adaptive cruise target enters the area directly in front of the vehicle, the position is close enough to the vehicle or continues to travel in front of the vehicle. If any of the conditions are met, it can be considered that the new adaptive cruise target identified by the millimeter-wave radar information is real and continuously and stably exists in the regionId (nearest vehicle in the front area), and the conditions for subsequent fusion operations are met.

[0081] Furthermore, the frame number data and lateral position variance data output by the millimeter-wave radar can be used to perform radar data stability judgment processing on the target millimeter-wave radar information to judge the data stability of the millimeter-wave radar. For example, when the radar output frame number age is greater than the first preset threshold and the lateral position variance is less than the second preset threshold, it is considered that the adaptive cruise target corresponding to the target millimeter-wave radar information is stable and real, that is, the adaptive cruise is in the regionId (nearest vehicle in the front area) area. The program can set the regionId (nearest vehicle in the front area) flag position of the adaptive cruise target corresponding to the target millimeter-wave radar information to 1, and the information will be subsequently associated for continuous maintenance and use by the fusion algorithm until the radar output changes and is reset to 0 when the conditions are no longer met.

[0082] Step 103, performing information fusion processing on the visual sensor information and the target millimeter wave radar information to generate fused target information;

[0083] In the specific implementation, the abstract association function can be designed by using C++ (programming language) abstract class, and a set of general processes required for data association can be pre-designed to associate the visual sensor information with the target millimeter wave radar information for information fusion processing. The general process can be roughly as follows: In the first frame of the program, a new fusion target is created based on the visual target. From the second frame, the fusion target position is taken as the midpoint, and the sensor targets that may be matched are preliminarily screened according to the change in the longitudinal distance to form a candidate set. In the candidate set, the Mahalanobis distance between the fusion target and each candidate target is calculated accordingly. The sensor target with the smallest Mahalanobis distance is taken as the optimal candidate target to complete the association step; it should be emphasized that due to the huge differences in the working principles and actual performance of each sensor, the corresponding association matching algorithm and parameters need to be adjusted. For example, the millimeter wave radar has a high longitudinal ranging accuracy, so when using the elliptical threshold screening, the threshold range can be appropriately reduced to achieve fewer targets entering the candidate set; in addition, the visual targets will have overlapping target frames, so when associating, the logic of whether the newly added target overlaps with the existing target can be added. If it overlaps, it is likely to be caused by a visual virtual target, so this situation can be not associated. Each sensor association algorithm can inherit this abstract class respectively, and execute the sensor association algorithm in turn in each cycle, so as to ensure that the general association algorithm is run and each sensor can run its own targeted algorithm and parameters.

[0084] After the association is successful, the target millimeter-wave radar information and visual sensor information can be used for fusion target information, that is, the target millimeter-wave radar information and visual sensor information can be directly fused. The fusion target information will obtain information such as visual type and lateral position, while the longitudinal position, longitudinal speed and other information theoretically mainly use the output information of the millimeter-wave radar. Among them, the fusion can continuously maintain the output of the regionId (nearest car in the front area) flag and pass it directly to the back-end control algorithm for use. In addition to using the position, speed, type, size and other information of the fusion output, the back-end control algorithm can also use the regionId (nearest car in the front area) flag. When this bit is 1, the control algorithm will give priority to the target for adaptive cruise control action, which can greatly shorten the judgment time and has higher stability.

[0085] In some embodiments of the present invention, the step of performing information fusion processing on the visual sensor information and the target millimeter wave radar information to generate fused target information includes:

[0086] Determine the frame number data and the lateral position variance data of the target millimeter-wave radar information, and perform radar data stability judgment processing on the target millimeter-wave radar information according to the frame number data and the lateral position variance data;

[0087] If the result of the radar data stability judgment processing is qualified, the visual sensor information and the target millimeter-wave radar information are subjected to the information fusion processing to generate the fused target information.

[0088] In practical applications, the frame number data and lateral position variance data output by the millimeter-wave radar can be used to perform radar data stability judgment processing on the target millimeter-wave radar information to judge the data stability of the millimeter-wave radar. For example, when the processing result of the radar data stability judgment processing is qualified, the adaptive cruise target corresponding to the target millimeter-wave radar information is considered to be stable and real, that is, the adaptive cruise is in the regionId (nearest vehicle in the front area). The program can set the regionId (nearest vehicle in the front area) flag position of the adaptive cruise target corresponding to the target millimeter-wave radar information to 1, and the information will be subsequently associated for continuous maintenance and use by the fusion algorithm until the radar output changes and no longer meets the conditions and is reset to 0.

[0089] In some embodiments of the present invention, the radar data stability determination process of the target millimeter-wave radar information according to the frame number data and the lateral position variance data includes:

[0090] Determine whether the frame number data is greater than a first preset threshold, and whether the lateral position variance data is less than a second preset threshold;

[0091] If the frame number data is greater than the first preset threshold, and the lateral position variance data is less than the second preset threshold, it is determined that the processing result of the radar data stability determination processing is qualified.

[0092] In practical applications, the frame number data and lateral position variance data output by the millimeter-wave radar can be used to perform radar data stability judgment processing on the target millimeter-wave radar information to judge the data stability of the millimeter-wave radar. For example, when the radar output frame number age is greater than the first preset threshold and the lateral position variance is less than the second preset threshold, the processing result of the radar data stability judgment processing is determined to be qualified, and it is considered that the adaptive cruise target corresponding to the target millimeter-wave radar information is stable and real, that is, the adaptive cruise is in the regionId (nearest vehicle in the front area). The program can set the regionId (nearest vehicle in the front area) flag position of the adaptive cruise target corresponding to the target millimeter-wave radar information to 1, and the information will be subsequently associated for continuous maintenance and use by the fusion algorithm until the radar output changes and is reset to 0 when the above conditions are no longer met.

[0093] In some embodiments of the present invention, the step of performing information fusion processing on the visual sensor information and the target millimeter wave radar information to generate fused target information includes:

[0094] The body sensor information of the vehicle is acquired, and the visual sensor information, the target millimeter-wave radar information and the body sensor information are subjected to the information fusion processing to generate the fusion target information.

[0095] In practical applications, information obtained by other body sensors except visual sensors and millimeter-wave radars can be obtained, and can also be preprocessed through the pre-set information preprocessing module. Specifically, the sensor usually sends out original information such as target position, speed, type, etc., but due to different types and models, the sending format and expression of the above information are different. Therefore, the same data format needs to be pre-defined within the program, and the received original signal is assigned to the corresponding internal data variable according to the content, so as to unify the data format and achieve software and hardware isolation. At the same time, all sensor data are timestamped with the vehicle system time for time synchronization. Since each sensor is installed at a different position on the vehicle body, it uses a different coordinate system for detecting the target. Therefore, the information preprocessing module can also move the measurement point of each sensor to the midpoint of the front bumper of the vehicle by rotation and translation for spatial synchronization.

[0096] Furthermore, the visual sensor information, the target millimeter-wave radar information and the vehicle body sensor information can be subjected to the information fusion processing according to the aforementioned information fusion processing flow to generate the fused target information, thereby further improving the rationality and stability of the fused target information; it should be noted that the vehicle body sensor information can be subjected to information fusion with the visual sensor information and the target millimeter-wave radar information that have completed the association process only after corresponding preprocessing, without the need to perform corresponding association processing on the vehicle body sensor information.

[0097] It should be emphasized that in a multi-sensor system, the output characteristics of each sensor for the detection of the same target are quite different. It is common that a sensor fails to detect the target at a certain moment, while other sensors can detect it. Therefore, a sensor output information effective statistics module can be developed for this characteristic. This module can be used to retrieve the output information of all sensors at the same time. If the output is empty, it means that the corresponding sensor did not detect the target. At this time, the module can mark this sensor as undetected, and the sensor can be directly skipped when the information is subsequently fused, thereby greatly reducing the data indexing time.

[0098] Step 104: performing adaptive cruise control on the vehicle according to the fused target information.

[0099] In the specific implementation, after the association is successful, the target millimeter-wave radar information and visual sensor information can be used for fusion target information. The fusion target information will obtain information such as visual type and lateral position, while the longitudinal position, longitudinal speed and other information theoretically mainly use the output information of the millimeter-wave radar. Among them, the fusion target information can continue to maintain the output of the regionId (nearest car in the front area) flag, and directly pass it to the back-end control algorithm for use in adaptive cruise control. In addition to using the position, speed, type, size and other information output by the fusion target information, the back-end control algorithm can also use the regionId (nearest car in the front area) flag. When this bit is 1, the control algorithm will give priority to the target for adaptive cruise control, which can greatly shorten the judgment time and has higher stability.

[0100] In some embodiments of the present invention, before performing adaptive cruise control on the vehicle according to the fused target information, the method further includes:

[0101] Perform rationality judgment processing on the fusion target information.

[0102] In the specific implementation, the fused target information can be processed for rationality judgment to eliminate outliers and output fused target information whose attributes are consistent with the actual target motion conditions. This ensures that the fused target information is accurate while having a certain amount of information redundancy, providing stable and reliable data input for adaptive cruise conditions, and thus optimizing the user experience.

[0103] The following will be combined Figure 2 The embodiments of the present invention are further described as follows:

[0104] Depend on Figure 2 It can be seen from the content that the overall algorithm framework of the present invention for implementing the above-mentioned adaptive cruise control method includes camera information, millimeter-wave radar information and vehicle body sensor information as the original input information sources of the fusion target, and each original input information source has a corresponding information preprocessing module responsible for converting data of different formats into a unified data format within the fusion algorithm, and is responsible for the spatial alignment of all data; it should be emphasized that the lane line information provided by the camera information can be used to screen and process the millimeter-wave radar information, and when the millimeter-wave radar information is subjected to subsequent preprocessing, in addition to the aforementioned general process, the millimeter-wave radar information can also be subjected to other targeted preprocessing steps such as position stability comparison processing and radar data stability judgment processing in accordance with the aforementioned corresponding steps, so as to comprehensively evaluate whether the adaptive cruise target identified by the radar is accurate and reliable. When the internal setting conditions are met, the regionId (nearest vehicle in the front area) required by the IACC (intelligent adaptive cruise) function is set to 1 and passed to the subsequent association and fusion algorithm. After obtaining the target regionId flag, the back-end algorithm can preferentially perform adaptive cruise control on the adaptive cruise target, such as deceleration, braking, etc., to realize the IACC function. The data association algorithm is responsible for matching the output results of each sensor at the same time, matching different sensor data source targets together for data fusion, and providing a reliable data source for subsequent autonomous driving function modules; after completing the data association of the camera information and the millimeter-wave radar information, the camera information, millimeter-wave radar information and body sensor information that have completed the corresponding processing flow can be fused and rationality judged, and the final fusion target information is output to perform adaptive cruise control on the vehicle; in the embodiment of the present invention, by introducing millimeter-wave radar information fusion into the final output information, it can not only make up for the single data limitations caused by inaccurate visual target position and sudden scene changes, but also limit the uncertainty caused by the susceptibility of millimeter-wave radar to interference through algorithms, thereby comprehensively improving the robustness of the system detection capability and providing the back-end IACC function with a data source with wider scene coverage and better stability.

[0105] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0106] Reference Figure 3 , shows a structural block diagram of an adaptive cruise control device provided in an embodiment of the present invention, which may specifically include the following modules:

[0107] The information acquisition module 301 is used to acquire the visual sensor information and millimeter wave radar information of the vehicle in response to the start-up operation of the vehicle adaptive cruise function; wherein the visual sensor information at least includes lane line information;

[0108] A screening processing module 302 is used to screen the millimeter-wave radar information according to the lane line information to determine the target millimeter-wave radar information;

[0109] A fused target information generating module 303 is used to perform information fusion processing on the visual sensor information and the target millimeter wave radar information to generate fused target information;

[0110] The adaptive cruise control module 304 is used to perform adaptive cruise control on the vehicle according to the fused target information.

[0111] In an optional embodiment of the present invention, the screening processing module 302 includes:

[0112] An adaptive cruise target determination submodule, used to determine, based on the lane line information, an adaptive cruise target that is in the same lane as the vehicle and has the shortest distance from the vehicle;

[0113] The target millimeter-wave radar information determination submodule is used to determine the millimeter-wave radar information corresponding to the adaptive cruise target as the target millimeter-wave radar information.

[0114] In an optional embodiment of the present invention, the fusion target information generation module 303 includes:

[0115] A radar data stability judgment submodule, used to determine the frame number data and the lateral position variance data of the target millimeter-wave radar information, and perform radar data stability judgment processing on the target millimeter-wave radar information according to the frame number data and the lateral position variance data;

[0116] The fused target information generating submodule is used to perform the information fusion processing on the visual sensor information and the target millimeter-wave radar information to generate the fused target information if the processing result of the radar data stability judgment processing is qualified.

[0117] In an optional embodiment of the present invention, the radar data stability judgment submodule includes:

[0118] A threshold judgment unit, used to judge whether the frame number data is greater than a first preset threshold, and whether the horizontal position variance data is less than a second preset threshold;

[0119] The radar data stability result determination unit is used to determine that the processing result of the radar data stability determination processing is qualified if the frame number data is greater than the first preset threshold and the lateral position variance data is less than the second preset threshold.

[0120] In an optional embodiment of the present invention, the target millimeter wave radar information determination submodule includes:

[0121] a position stability comparison unit, used to obtain the historical millimeter-wave radar information corresponding to the adaptive cruise target, and perform position stability comparison processing on the millimeter-wave radar information corresponding to the adaptive cruise target and the historical millimeter-wave radar information;

[0122] The target millimeter-wave radar information determining unit is used to determine the millimeter-wave radar information corresponding to the adaptive cruise target as the target millimeter-wave radar information if the processing result of the position stability comparison processing is qualified.

[0123] In an optional embodiment of the present invention, the fusion target information generating module 303 includes:

[0124] The body sensor information fusion module is used to obtain the body sensor information of the vehicle, and perform the information fusion processing on the visual sensor information, the target millimeter wave radar information and the body sensor information to generate the fused target information.

[0125] In an optional embodiment of the present invention, the device further includes:

[0126] The rationality judgment processing module is used to perform rationality judgment processing on the fusion target information.

[0127] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0128] In addition, an embodiment of the present invention further provides an electronic device, such as Figure 4 As shown, it includes a processor 401, a communication interface 402, a memory 403 and a communication bus 404, wherein the processor 401, the communication interface 402, and the memory 403 communicate with each other through the communication bus 404.

[0129] Memory 403, used for storing computer programs;

[0130] The processor 401 is used to execute the program stored in the memory 403 to implement the following steps:

[0131] In response to a start-up operation of the vehicle adaptive cruise function, obtaining visual sensor information and millimeter wave radar information of the vehicle; wherein the visual sensor information at least includes lane line information;

[0132] Filtering the millimeter-wave radar information according to the lane line information to determine target millimeter-wave radar information;

[0133] Performing information fusion processing on the visual sensor information and the target millimeter wave radar information to generate fused target information;

[0134] Adaptive cruise control is performed on the vehicle according to the fused target information.

[0135] In an optional embodiment of the present invention, the screening of the millimeter-wave radar information according to the lane line information to determine the target millimeter-wave radar information includes:

[0136] Determine, according to the lane line information, an adaptive cruise target that is in the same lane as the vehicle and has the shortest distance from the vehicle;

[0137] The millimeter-wave radar information corresponding to the adaptive cruise target is determined as the target millimeter-wave radar information.

[0138] In an optional embodiment of the present invention, the performing information fusion processing on the visual sensor information and the target millimeter wave radar information to generate fused target information includes:

[0139] Determine the frame number data and the lateral position variance data of the target millimeter-wave radar information, and perform radar data stability judgment processing on the target millimeter-wave radar information according to the frame number data and the lateral position variance data;

[0140] If the result of the radar data stability judgment processing is qualified, the visual sensor information and the target millimeter-wave radar information are subjected to the information fusion processing to generate the fused target information.

[0141] In an optional embodiment of the present invention, the radar data stability judgment processing of the target millimeter-wave radar information according to the frame number data and the lateral position variance data includes:

[0142] Determine whether the frame number data is greater than a first preset threshold, and whether the lateral position variance data is less than a second preset threshold;

[0143] If the frame number data is greater than the first preset threshold, and the lateral position variance data is less than the second preset threshold, it is determined that the processing result of the radar data stability determination processing is qualified.

[0144] In an optional embodiment of the present invention, determining the millimeter-wave radar information corresponding to the adaptive cruise target as the target millimeter-wave radar information includes:

[0145] Acquire historical millimeter-wave radar information corresponding to the adaptive cruise target, and perform position stability comparison processing on the millimeter-wave radar information corresponding to the adaptive cruise target and the historical millimeter-wave radar information;

[0146] If the processing result of the position stability comparison processing is qualified, the millimeter-wave radar information corresponding to the adaptive cruise target is determined as the target millimeter-wave radar information.

[0147] In an optional embodiment of the present invention, the performing information fusion processing on the visual sensor information and the target millimeter wave radar information to generate fused target information includes:

[0148] The body sensor information of the vehicle is acquired, and the visual sensor information, the target millimeter-wave radar information and the body sensor information are subjected to the information fusion processing to generate the fusion target information.

[0149] In an optional embodiment of the present invention, before performing adaptive cruise control on the vehicle according to the fused target information, the method further includes:

[0150] Perform rationality judgment processing on the fusion target information.

[0151] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0152] The communication interface is used for communication between the above terminal and other devices.

[0153] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0154] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0155] like Figure 5 As shown, in another embodiment provided by the present invention, a computer-readable storage medium 501 is also provided, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes an adaptive cruise control method described in the above embodiment.

[0156] In another embodiment of the present invention, a computer program product including instructions is provided. When the computer program product is run on a computer, the computer is enabled to execute an adaptive cruise control method described in the above embodiment.

[0157] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0158] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0159] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0160] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. An adaptive cruise control method, characterized in that: The method comprises: In response to a start-up operation of the vehicle adaptive cruise function, obtaining visual sensor information and millimeter wave radar information of the vehicle; wherein the visual sensor information includes at least lane line information; Filtering the millimeter-wave radar information according to the lane line information to determine target millimeter-wave radar information; Performing information fusion processing on the visual sensor information and the target millimeter wave radar information to generate fused target information; Adaptive cruise control is performed on the vehicle according to the fused target information.

2. The method according to claim 1, characterized in that The screening of the millimeter-wave radar information according to the lane line information to determine the target millimeter-wave radar information includes: Determine, according to the lane line information, an adaptive cruise target that is in the same lane as the vehicle and has the shortest distance from the vehicle; The millimeter-wave radar information corresponding to the adaptive cruise target is determined as the target millimeter-wave radar information.

3. The method according to claim 1, characterized in that The performing information fusion processing on the visual sensor information and the target millimeter wave radar information to generate fused target information includes: Determine the frame number data and the lateral position variance data of the target millimeter-wave radar information, and perform radar data stability judgment processing on the target millimeter-wave radar information according to the frame number data and the lateral position variance data; If the result of the radar data stability judgment processing is qualified, the visual sensor information and the target millimeter-wave radar information are subjected to the information fusion processing to generate the fused target information.

4. The method according to claim 3, characterized in that The radar data stability determination process is performed on the target millimeter-wave radar information according to the frame number data and the lateral position variance data, including: Determine whether the frame number data is greater than a first preset threshold, and whether the lateral position variance data is less than a second preset threshold; If the frame number data is greater than the first preset threshold, and the lateral position variance data is less than the second preset threshold, it is determined that the processing result of the radar data stability determination processing is qualified.

5. The method according to claim 2, characterized in that: The determining the millimeter-wave radar information corresponding to the adaptive cruise target as the target millimeter-wave radar information includes: Acquire historical millimeter-wave radar information corresponding to the adaptive cruise target, and perform position stability comparison processing on the millimeter-wave radar information corresponding to the adaptive cruise target and the historical millimeter-wave radar information; If the processing result of the position stability comparison processing is qualified, the millimeter-wave radar information corresponding to the adaptive cruise target is determined as the target millimeter-wave radar information.

6. The method according to claim 1, characterized in that The performing information fusion processing on the visual sensor information and the target millimeter wave radar information to generate fused target information includes: The body sensor information of the vehicle is acquired, and the visual sensor information, the target millimeter-wave radar information and the body sensor information are subjected to the information fusion processing to generate the fusion target information.

7. The method according to claim 1, characterized in that Before performing adaptive cruise control on the vehicle according to the fused target information, the method further includes: Perform rationality judgment processing on the fusion target information.

8. An adaptive cruise control device, characterized in that: The device comprises: An information acquisition module, configured to acquire visual sensor information and millimeter wave radar information of the vehicle in response to a start-up operation of the vehicle adaptive cruise function; wherein the visual sensor information at least includes lane line information; A screening and processing module, used to screen the millimeter-wave radar information according to the lane line information to determine the target millimeter-wave radar information; A fusion target information generation module is used to perform information fusion processing on the visual sensor information and the target millimeter wave radar information to generate fusion target information; An adaptive cruise control module is used to perform adaptive cruise control on the vehicle according to the fused target information.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to implement the method according to any one of claims 1 to 7 when executing the program stored in the memory.

10. One or more computer-readable media having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform the method of any one of claims 1-7.

Images