An automatic driving method and system based on multi-sensor fusion deployment

Abstract

The present invention relates to the field of process control technology, and specifically to an automatic driving method and system based on multi-sensor fusion deployment. The data to be fused and the driving behavior time series data of multiple sensors in the process of vehicle automatic driving are obtained, and the synchronous processing and real-time control of the data are realized by using a timer. In the early stage of automatic driving, the system quickly adjusts the preset window length according to the data fluctuation of the data to be fused to adapt to environmental changes; when the system is stable, the window length is comprehensively adjusted in combination with the similarity of environmental changes and driving behavior data to enhance the ability to respond to complex environments and emergencies. The method dynamically adjusts the window length and fuses and analyzes the data, improves the accuracy of data fusion and the perception accuracy of automatic driving, and provides a reliable basis for vehicle automatic control strategies.

Description

Technical Field

[0001] The present invention relates to the field of process control technology, and in particular to an automatic driving method and system based on multi-sensor fusion deployment. Background Art

[0002] With the rapid development of the automobile industry, autonomous driving technology has been widely used. Autonomous driving vehicles perceive the surrounding environment by integrating multiple sensors (such as cameras, radars, lidars, ultrasonic sensors, inertial measurement units, etc.) and make corresponding driving decisions based on the perceived data. However, the data from a single sensor often has limitations, such as the limited field of view of the camera in bad weather and the possibility of misjudgment of radar in complex environments. Therefore, multi-sensor fusion technology came into being. This technology improves the perception accuracy and robustness of the autonomous driving system by integrating information from multiple sensors.

[0003] In the process of fusing multi-sensor data, the existing technology usually uses a fixed-length window to fuse multi-sensor data. However, since different sensors have different data preprocessing times, and the environment and driving behavior characteristics will also change dynamically during the autonomous driving process, the use of a fixed-length window will lead to a decrease in the accuracy of data fusion, which in turn affects the perception accuracy of the vehicle's autonomous driving. Summary of the invention

[0004] In order to solve the technical problem that different sensors have different data preprocessing times and the environment and driving behavior characteristics will also change dynamically during the autonomous driving process, the use of a fixed-length window will lead to a decrease in the accuracy of data fusion, thereby affecting the perception accuracy of the vehicle's autonomous driving, the purpose of the present invention is to provide an autonomous driving method based on multi-sensor fusion deployment, and the technical solutions adopted are as follows:

[0005] Obtain the data to be fused from multiple sensors during the vehicle's autonomous driving process and multiple driving behavior time series data;

[0006] Set a timer to determine the environmental risk value at the current triggering moment based on the fluctuation of the data to be fused before the current triggering moment;

[0007] When the number of historical trigger times of the timer does not reach the preset number, the preset window length is adjusted based on the environmental risk value at the current triggering moment to obtain the corrected window length at the current triggering moment;

[0008] When the number of historical trigger times of the timer reaches a preset number, the similarity of data values ​​is analyzed in the driving behavior time series data before and at the current triggering moment to determine the driving behavior risk value at the current triggering moment; based on the data to be fused before the current triggering moment, the environmental risk value and the driving behavior risk value at the current triggering moment are adjusted to obtain a risk level value; based on the risk level value at the current triggering moment, the preset window length is adjusted to obtain a corrected window length at the current triggering moment;

[0009] The data value of each type of data to be fused within the correction window length at the current trigger moment is fused and analyzed to obtain the fusion feature value at the current trigger moment to control the vehicle's automatic driving.

[0010] Furthermore, the types of data to be fused include IMU data and other types of data to be fused.

[0011] Furthermore, the determining of the environmental risk value at the current triggering moment based on the fluctuation of the data to be fused before the current triggering moment includes:

[0012] The data to be fused when the timer is not set before the current trigger moment is set with breakpoints according to preset time intervals, and the average of the data values ​​of each type of data to be fused between two adjacent breakpoints is used as the fusion feature value of each type of data to be fused at the latter breakpoint of the two adjacent breakpoints, and the breakpoints and the trigger moment are collectively referred to as reference moments;

[0013] Among all the data to be fused, the fused feature values ​​of a preset number of reference moments before the current trigger moment are used as the input of the F-OCC model to obtain the vehicle surrounding obstacle ratio sequence corresponding to the current trigger moment; based on the change trend between the data values ​​in the vehicle surrounding obstacle ratio sequence, the first environmental risk factor at the current trigger moment is determined;

[0014] Decomposing and calculating the data values ​​in the IMU data at a preset number of reference moments before the current trigger moment, obtaining the corresponding angular velocity sequence and movement distance sequence; comprehensively analyzing the fluctuation of the data values ​​in the angular velocity sequence and the fluctuation of the data values ​​in the movement distance sequence, and determining the second environmental risk factor at the current trigger moment;

[0015] According to the first environmental risk factor and the second environmental risk factor at the current triggering moment, an environmental risk value at the current triggering moment is obtained, the value of the environmental risk value is a normalized value, and the first environmental risk factor and the second environmental risk factor are both positively correlated with the environmental risk value.

[0016] Furthermore, the preset window length is adjusted based on the environmental risk value at the current triggering moment to obtain a corrected window length at the current triggering moment, including:

[0017] The value obtained by rounding up the product of the environmental risk value and the preset window length is used as the first adjustment parameter, and the difference between the preset window length and the first adjustment parameter is used as the corrected window length at the current triggering moment.

[0018] Furthermore, the method for obtaining the driving behavior risk value includes:

[0019] The absolute value of the difference between the environmental hazard value at each historical triggering moment and the environmental hazard value at the current triggering moment is taken as the environmental hazard difference; the historical triggering moment when the environmental hazard difference is less than the preset difference threshold is taken as the target moment;

[0020] The time period between two adjacent trigger moments is used as the trigger time period corresponding to the latter of the two adjacent trigger moments;

[0021] In the current trigger time period corresponding to the current trigger time and the historical trigger time period corresponding to each target time, the DTW distance between the same type of driving behavior time series data is calculated, and the average of the DTW distances between all types of driving behavior time series data is used as the driving behavior risk factor at the current trigger time;

[0022] All driving behavior risk factors at the current triggering moment are integrated to determine the driving behavior risk value at the current triggering moment.

[0023] Furthermore, the environmental risk value and the driving behavior risk value at the current triggering moment are adjusted based on the data to be fused before the current triggering moment to obtain the risk level value, including:

[0024] In the obstacle ratio sequence around the vehicle corresponding to the current triggering moment, the difference between the last data value and all previous data values ​​is analyzed to obtain the environment similarity feature value corresponding to the current triggering moment, and the value of the environment similarity feature value is a normalized value;

[0025] The environmental risk value and the driving behavior risk value at the current triggering moment are adjusted based on the environmental similarity feature value to obtain a risk degree value.

[0026] Furthermore, the environmental risk value and the driving behavior risk value at the current triggering moment are adjusted based on the environmental similarity feature value to obtain a risk level value, including:

[0027] The product of the environment similarity feature value and the driving behavior risk value is used as the first risk factor, and the product of the value after negative correlation mapping of the environment similarity feature value and the environment risk value is used as the second risk factor;

[0028] The value obtained by normalizing the sum of the first risk factor and the second risk factor is used as the risk level value.

[0029] Furthermore, the preset window length is adjusted based on the danger level value at the current triggering moment to obtain a modified window length at the current triggering moment, including:

[0030] The value obtained by rounding up the product of the risk level value and the preset window length is used as the second adjustment parameter, and the difference between the preset window length and the second adjustment parameter is used as the corrected window length at the current triggering moment.

[0031] Furthermore, the data value of each type of data to be fused within the correction window length at the current trigger moment is subjected to fusion analysis to obtain the fusion feature value at the current trigger moment so as to control the automatic driving of the vehicle, including:

[0032] For each type of data to be fused, the data value within the correction window length at the current trigger moment is fitted based on the least squares method to predict the fusion feature value at the current trigger moment;

[0033] The fusion feature values ​​of all the data to be fused at the current triggering moment are used as the input of the F-OCC model, and the control signal is output to control the vehicle to perform autonomous driving.

[0034] The present invention also proposes an automatic driving system based on multi-sensor fusion deployment, the system comprising:

[0035] A memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any step of the method for autonomous driving based on multi-sensor fusion deployment can be implemented.

[0036] The present invention has the following beneficial effects:

[0037] Through multi-sensor data fusion, the limitations of a single sensor in perceiving the environment can be compensated. Therefore, in the process of multi-sensor fusion deployment, the data to be fused of multiple sensors in the process of vehicle automatic driving are first obtained. In view of the fact that the driving behavior data in the process of vehicle automatic driving can also be used as the basis for subsequent analysis of vehicle automatic control strategies, multiple driving behavior time series data are obtained at the same time. Because different sensors have inconsistent pre-processing times for data, in the present invention, a timer is set, and when the timer is triggered, the synchronous processing of multiple data to be fused is realized, thereby controlling the vehicle automatic driving process in real time. When the number of historical triggers of the timer does not reach the preset number, it is considered that the vehicle's automatic driving time is too short and the driving behavior data is less, so the change of the environment is mainly considered. The change of the environment can be quantified by the fluctuation of various data to be fused. Therefore, based on the fluctuation of the data to be fused before the current trigger moment, the preset window length is adjusted to obtain the corrected window length at the current trigger moment to quickly adapt to environmental changes; and when the number of historical triggers of the timer reaches the preset number, the system is basically stable at this time, and enough driving behavior data has been accumulated for analysis, so the change of the environment is combined with the similarity between the driving behavior time series data, and the change of the environment and the stability of the driving behavior are comprehensively evaluated to adjust the preset window length to obtain the corrected window length at the current trigger moment. By analyzing the changes in the environment and the changes in driving behavior during the vehicle's automatic driving process and dynamically adjusting the window length, the system's ability to respond to complex environments and emergencies can be improved. Finally, each type of data to be fused within the corrected window length is fused and analyzed to improve the accuracy of data fusion. Therefore, when the fusion feature value is obtained to control the vehicle's automatic driving, the perception accuracy of automatic driving can be effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 A method flow chart of an autonomous driving method based on multi-sensor fusion deployment provided by an embodiment of the present invention;

[0040] Figure 2 A method flow chart of a method for obtaining an environmental risk value provided by an embodiment of the present invention;

[0041] Figure 3A method flow chart of a method for obtaining a driving behavior risk value provided by an embodiment of the present invention;

[0042] Figure 4 A method flow chart of a method for obtaining a danger level value provided by an embodiment of the present invention;

[0043] Figure 5 A system block diagram of an autonomous driving system based on multi-sensor fusion deployment provided by an embodiment of the present invention;

[0044] Figure 6 A schematic diagram of the system structure of an autonomous driving system based on multi-sensor fusion deployment provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the specific implementation, structure, features and effects of an autonomous driving method and system based on multi-sensor fusion deployment proposed by the present invention in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0046] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0047] The following is a detailed description of a specific scheme of an autonomous driving method and system based on multi-sensor fusion deployment provided by the present invention in conjunction with the accompanying drawings.

[0048] See also Figure 1 , which shows a method flow chart of an automatic driving method based on multi-sensor fusion deployment provided by an embodiment of the present invention, the method comprising the following steps:

[0049] Step S1: Acquire the data to be fused from multiple sensors during the vehicle's automatic driving process and multiple driving behavior time series data.

[0050] In the exploration and practice of autonomous driving technology, a core challenge is how to achieve comprehensive and accurate perception of the surrounding environment in a complex and changing road environment to formulate driving strategies. The surrounding environment includes road conditions, the dynamics of other vehicles and pedestrians, the status of traffic signs and signal lights, etc. Most of this information cannot be obtained directly through a single sensor, but requires multiple sensors to work together and complement each other to overcome their respective limitations. In addition, the vehicle's own driving behavior data is also an important basis for making safe and efficient driving decisions.

[0051] Therefore, firstly, the data to be fused collected by various sensors in the process of vehicle autonomous driving and various driving behavior time series data are obtained. In the embodiment of the present invention, the types of data to be fused include at least: Inertial Measurement Unit (IMU) data, which can be obtained by the IMU sensor integrated in the autonomous driving vehicle; camera data, which is captured by the high-definition or ultra-high-definition camera installed on the vehicle, and is used to identify vehicles, pedestrians, traffic signs, scene understanding, etc.; radar data, which is obtained by the radar sensor (such as millimeter wave radar) installed on the vehicle, and is used for target tracking, obstacle detection and classification, etc.; LiDAR data, which is obtained by the vehicle-mounted LiDAR, and is used for environmental modeling, etc.; ultrasonic data, which is obtained by ultrasonic sensors, and is used for obstacle identification and positioning, etc.; global navigation satellite data, which is obtained by satellite systems, and is used for navigation and positioning, etc. By fusing all the data to be fused in the subsequent process, more favorable support can be provided for making autonomous driving decisions.

[0052] In addition to the acquisition of the above-mentioned data to be integrated, the vehicle's automatic driving process also needs to obtain various driving behavior time series data during the operation of the vehicle. The types of driving behavior data include at least: steering wheel turning force, brake force, accelerator force, etc. Each type of driving behavior data is a time series sequence and can be obtained through the vehicle's CAN bus.

[0053] At this point, a variety of data to be integrated and a variety of driving behavior time series data in the vehicle's autonomous driving process can be obtained.

[0054] Step S2: Set a timer to determine the environmental hazard value at the current triggering moment based on the fluctuation of the data to be fused before the current triggering moment.

[0055] When the data of multiple sensors in the process of vehicle automatic driving are fused and analyzed, usually, for each data to be fused, the data value in the window of fixed length is fused and analyzed to obtain the fusion characteristic value, and then the fusion characteristic values ​​of all the data to be fused are integrated to perform subsequent operations. In the fusion analysis process, it is necessary to ensure the synchronization of various data to be fused, but due to the different sampling frequencies of different sensors, and the time required for preprocessing the data is also different, for example: for image sensors, the time required for preprocessing increases significantly when the car accelerates, the more noise the image shows, and even ghosting and other phenomena appear, so the amount of data to be calculated becomes larger and the preprocessing time becomes longer; on the contrary, the preprocessing delay of some sensors, such as IMU sensors, is generally very small or it is considered that there is no delay, so in the fusion analysis process, it is necessary to perform time alignment on multiple data to be fused. In the embodiment of the present invention, a timer is set, and when the timer is triggered, the synchronous processing of multiple data to be fused is realized. Given that the primary task of autonomous driving is to ensure the safety of passengers and other road users, it is necessary to analyze the danger of changes in the vehicle's current environment. The danger of environmental changes can be characterized by the fluctuations in the data to be fused. Therefore, the environmental danger value is evaluated based on the fluctuations in the data to be fused before the current trigger moment to judge the stability and potential danger of the vehicle's surrounding environment.

[0056] Preferably, in one embodiment of the present invention, the method for obtaining the environmental risk value includes:

[0057] See also Figure 2 , which shows a method flow chart of a method for obtaining an environmental risk value in one embodiment of the present invention, the method comprising the following steps:

[0058] Step S201: Determine the reference time and the fusion feature value of each reference time.

[0059] Before setting the timer, the fusion method of the data to be fused can be to set breakpoints according to the preset time interval. For each type of data to be fused, the average of the data values ​​between two adjacent breakpoints is used as the fusion feature value at the latter of the two adjacent breakpoints, and all breakpoints and each triggering moment after the timer is set are collectively referred to as reference moments. Among them, the preset time interval is set to the longest preprocessing time required by all sensors, so that the preprocessing operations of all sensors at each breakpoint can be guaranteed to be completed, so that the obtained fusion feature value will be more accurate.

[0060] It should be noted that the time interval of the timer can be set to 20 milliseconds, and the specific interval can be adjusted according to the implementation scenario and is not limited here.

[0061] Step S202: Determine the first environmental risk factor at the current triggering moment based on the change of the fusion feature value at the reference moment before the current triggering moment.

[0062] The fusion feature values ​​of all the data to be fused at the preset number of reference moments before the current trigger moment are used as the input of the F-OCC model to obtain the obstacle ratio sequence around the vehicle corresponding to the current trigger moment. The data values ​​in the obstacle ratio sequence around the vehicle can be used to reflect the change in the ratio of obstacles in the surrounding environment of the vehicle during the automatic driving process. Therefore, the change trend between all data values ​​in the obstacle ratio sequence around the vehicle is analyzed to determine the first environmental risk factor at the current trigger moment. The formula model of the first environmental risk factor can be, for example, as follows: in, Indicates the first environmental risk factor at the current triggering moment; Indicates the number of data values ​​in the obstacle ratio sequence around the vehicle corresponding to the current trigger moment; Indicates the number of obstacles in the vehicle surrounding ratio sequence corresponding to the current triggering moment. data values; Indicates the number of obstacles in the vehicle surrounding ratio sequence corresponding to the current triggering moment. data values; Represents the normalization function.

[0063] In the formula model of the first environmental risk factor, the changing trend of the data values ​​in the obstacle ratio sequence around the vehicle is analyzed, and the changing trend can be represented by the difference between two adjacent data values. When it is a positive number and the larger it is, the data value in the obstacle ratio sequence around the vehicle shows a large increasing trend, which means that during the process of automatic driving of the vehicle, obstacles are increasing, so the degree of danger will also increase; on the contrary, when If it is a negative number and the smaller it is, it means that the data values ​​in the obstacle ratio sequence around the vehicle show a large decreasing trend, which means that during the vehicle's automatic driving process, the obstacles are decreasing, so the degree of danger will also decrease accordingly; all adjacent data values ​​in the obstacle ratio sequence around the vehicle corresponding to the current trigger moment are subjected to the aforementioned difference analysis, and finally all the differences are averaged. Since the mean may be positive or negative, and when it is a positive number, the larger it is, the higher the degree of danger of the vehicle's environment at the current trigger moment; when it is a negative number, the smaller it is, the lower the degree of danger of the vehicle's environment at the current trigger moment, so use Function to mean Normalization is performed so that the value range is between 0 and 1, and the closer it is to 1, the higher the risk.

[0064] It should be noted that obtaining the obstacle ratio sequence around the vehicle based on the F-OCC model is a well-known technology and will not be described in detail here; the preset number is set to 6, and the specific value can be adjusted according to the implementation scenario and is not limited here.

[0065] Step S203: Determine the second environmental risk factor at the current triggering moment based on the fluctuation characteristics of the data value in the IMU data at the reference moment before the current triggering moment.

[0066] IMU is one of the key sensors used to sense the motion state of an autonomous vehicle. It consists of three single-axis accelerometers and three single-axis gyroscopes. Decomposing and calculating the IMU data can obtain the relative posture of the vehicle, that is, obtain information such as angular velocity and movement distance. The fluctuations in the vehicle's angular velocity and movement distance can be used to reflect the complexity of the vehicle's current environment.

[0067] Therefore, the data values ​​in the IMU data at the preset number of reference moments before the current trigger moment are decomposed and calculated to obtain the corresponding angular velocity sequence and movement distance sequence; then the fluctuation of all data values ​​in the angular velocity sequence and the fluctuation of all data values ​​in the movement distance sequence are comprehensively analyzed to determine the second environmental risk factor at the current trigger moment. The formula model of the second environmental risk factor can be, for example, as follows: in, Indicates the second environmental risk factor at the current triggering moment; Represents the variance of all data values ​​in the angular velocity sequence corresponding to the current trigger moment; Represents the variance of all data values ​​in the motion distance sequence corresponding to the current trigger moment; Represents the normalization function.

[0068] In the formula model of the second environmental risk factor, variance is a statistic that measures the degree of data dispersion and fluctuation. When the variance of all data values ​​in the angular velocity sequence is larger, it means that the data values ​​in the angular velocity sequence fluctuate greatly. It can be regarded as that the vehicle has made a sharp turn or suddenly changed lanes in the period before the current trigger moment, which reflects that the environment in which the vehicle is located is more complex, and the danger will increase. Obstacles, traffic flow, road conditions and other factors in the external environment will cause the vehicle to constantly adjust its driving state. This adjustment is reflected in the movement distance sequence, which is the fluctuation of the data value. Therefore, when the variance of all data values ​​in the movement distance sequence is larger, it can be regarded as that the vehicle may have experienced significant acceleration, deceleration and other operations in the period before the current trigger moment to adapt to the changing environment. It can be considered that the environment in which the vehicle is located is more complex and more dangerous. Therefore, the variance of all data values ​​in the angular velocity sequence and the variance of all data values ​​in the movement distance sequence are combined, and the sum of the two values ​​is normalized as the second environmental risk factor corresponding to the current trigger moment. The larger the value, the higher the danger.

[0069] In other embodiments of the present invention, the fluctuation situation can also be characterized by standard deviation, which is not limited here; decomposing and calculating the IMU data to obtain information such as angular velocity and movement distance is a well-known technology and will not be elaborated here.

[0070] Step S204: Determine the environmental risk value at the current triggering moment by integrating the first environmental risk factor and the second environmental risk factor at the current triggering moment.

[0071] Based on the analysis of the above steps, it can be seen that the larger the first environmental risk factor at the current triggering moment, the higher the environmental risk of the vehicle. Similarly, the larger the second environmental risk factor at the current triggering moment, the higher the environmental risk of the vehicle. Therefore, the first environmental risk factor and the second environmental risk factor are positively correlated with the environmental risk value. Therefore, the sum of the first environmental risk factor and the second environmental risk factor at the current triggering moment can be normalized and used as the environmental risk value at the current triggering moment, denoted as , and the larger the environmental danger value is, the more dangerous the vehicle’s environment is at the current triggering moment.

[0072] It should be noted that normalization can be done using Normalization function.

[0073] Step S3: When the historical triggering times of the timer do not reach the preset times, the preset window length is adjusted based on the environmental risk value at the current triggering moment to obtain the corrected window length at the current triggering moment.

[0074] In a period of time just set by the timer, due to the small number of historical triggers, when the preset number is not reached, it is considered that the vehicle's automatic driving time is too short and the vehicle's driving behavior data is small, so the vehicle's automatic driving strategy is mainly formulated based on the dangerous situations that may be caused by environmental changes, and the dangerous situations that may be caused by environmental changes are quantified by the environmental risk value. When the environmental risk value is larger, the real-time performance of data fusion analysis should be improved under the premise of ensuring the accuracy of data fusion, so the degree of reduction of the preset window length should be greater. On the contrary, when the environmental risk value is smaller, the accuracy of data fusion can be further improved, so the degree of reduction of the preset window length should be smaller. Therefore, the preset window length is adjusted based on the environmental risk value at the current triggering moment to obtain the corrected window length at the current triggering moment. In the subsequent process, the data in the corrected window length is fused and analyzed to enable the vehicle to adapt to environmental changes more quickly and formulate a more accurate driving strategy.

[0075] Preferably, in one embodiment of the present invention, adjusting the preset window length based on the environmental risk value at the current triggering moment to obtain a corrected window length at the current triggering moment includes:

[0076] The product of the environmental risk value and the preset window length is rounded up to an integer value as the first adjustment parameter, and the difference between the preset window length and the first adjustment parameter is used as the corrected window length at the current trigger moment. The formula model for correcting the window length includes: in, Indicates the correction window length at the current triggering moment; Indicates the preset window length; Indicates the environmental danger value at the current triggering moment; Represents the ceiling function.

[0077] In the formula model for correcting the window length value, the preset window length is adaptively adjusted based on the environmental risk value. The product of the environmental risk value and the preset window length is rounded up, and the resulting value is the amount by which the preset window needs to be adjusted. When the environmental risk value is greater, in order to ensure the real-time and accuracy of the vehicle's automatic driving strategy, the window length corresponding to the current trigger time should be reduced to a greater extent, that is, On the contrary, when the environmental risk value is smaller, in order to further improve the accuracy of data fusion, the degree of reduction of the window length corresponding to the current trigger time will become smaller, that is, So the preset window length is smaller than The difference is used as the final correction window length at the current trigger moment.

[0078] It should be noted that the preset number of times is set to 10, and the specific value can be adjusted according to the implementation scenario and is not limited here; the preset window length is set to 8, then the preset window length at the current trigger moment includes the data before the current trigger moment and the 8 pre-processed data values ​​that are closest in time to the data value at the current trigger moment. The specific value of the preset window length can be adjusted according to actual conditions and is not limited here.

[0079] Step S4: When the number of historical trigger times of the timer reaches a preset number, the similarity of data values ​​is analyzed in the driving behavior time series data at the current trigger moment and before, and the driving behavior risk value at the current trigger moment is determined; based on the data to be fused before the current trigger moment, the environmental risk value and the driving behavior risk value at the current trigger moment are adjusted to obtain a risk level value; based on the risk level value at the current trigger moment, the preset window length is adjusted to obtain a corrected window length at the current trigger moment.

[0080] When the number of historical triggers of the timer reaches the preset number, the system is basically stable and has accumulated enough driving behavior data for analysis. The similarity between driving behavior data can reflect the stability of the vehicle during automatic driving. Therefore, the changes in the environment are combined with the similarity between the driving behavior time series data, and the data to be fused before the current trigger moment is used to adjust it to obtain the danger level value. The danger level value can comprehensively evaluate the changes in the environment and the stability of driving behavior. Therefore, the preset window length can be adjusted based on the danger level value to obtain the corrected window length at the current trigger moment.

[0081] Driving behavior time series data can reflect the driving status of a vehicle within a certain period of time. Therefore, by analyzing the similarities between driving behavior data, abnormal behaviors that are inconsistent with normal driving status can be identified, thereby evaluating the dangerousness of the driving behavior at the current triggering moment and obtaining the driving behavior risk value.

[0082] Preferably, in one embodiment of the present invention, the method for obtaining the driving behavior risk value includes:

[0083] See also Figure 3 , which shows a method flow chart of a method for obtaining a driving behavior risk value in one embodiment of the present invention, the method comprising the following steps:

[0084] Step S401: Determine similar environmental states and select target moments.

[0085] First, locate the similar environment. By comparing the environmental hazard value at the current trigger moment with the environmental hazard values ​​at all historical trigger moments, select the historical trigger moment with a high degree of similarity to the environmental hazard situation at the current trigger moment as the target moment. This helps to eliminate the interference of differences in environmental hazard factors on driving behavior analysis.

[0086] Therefore, among all historical trigger moments, the absolute value of the difference between the environmental hazard value at each historical trigger moment and the environmental hazard value at the current trigger moment is taken as the environmental hazard difference; the historical trigger moment when the environmental hazard difference is less than the preset difference threshold is taken as the target moment.

[0087] It should be noted that the preset difference threshold is 0.2, and the specific value can be adjusted according to the implementation scenario and is not limited here.

[0088] Step S402: define a trigger time period.

[0089] Then define the trigger time period. By defining the trigger time period, we can ensure that each analysis is based on a continuous and complete data set, avoiding data interruption. The time between two adjacent trigger moments is taken as a time period, and the time period between the current trigger moment and the previous trigger moment is taken as the trigger time period corresponding to the current trigger moment, recorded as the current trigger time period. Similarly, the time period between two adjacent historical trigger moments is taken as the trigger time period corresponding to the latter of the two adjacent historical trigger moments, that is, recorded as the historical trigger time period.

[0090] Step S403: Analyze the similarity of the driving behavior time series data in the current trigger time period corresponding to the current trigger time and the historical trigger time period corresponding to the target time, and determine the driving behavior risk value at the current trigger time.

[0091] In this embodiment of the present invention, similar situations are analyzed by a dynamic time warping algorithm. Since the driving behavior time series data has multiple types, the DTW distance between the same type of driving behavior time series data is first calculated in the current trigger time period corresponding to the current trigger moment and the historical trigger time period corresponding to each target moment, and then the average of the DTW distances between all types of driving behavior time series data is used as the driving behavior risk factor at the current trigger moment. At this time, there is a driving behavior risk factor between the current trigger time period corresponding to the current trigger moment and the historical trigger time period corresponding to each target moment.

[0092] Finally, all driving behavior risk factors at the current triggering moment are integrated to obtain the driving behavior risk value at the current triggering moment. The formula model of the driving behavior risk value can be specifically, for example: in, Indicates the driving behavior risk value at the current triggering moment; Indicates the number of target moments corresponding to the current trigger moment; Indicates the number of types of driving behavior time series data; Indicates the current trigger time period and the The historical trigger time period of the target moment is in DTW distance between the driving behavior time series data; Represents the normalization function.

[0093] In the formula model of driving behavior risk value, the DTW distance between the same type of driving behavior time series data in the current trigger time period and the historical trigger time period corresponding to each target moment is calculated. When the DTW distance value is smaller, it means that the similarity between the driving behavior time series data in the two time periods is higher. It can be considered that the driving behavior in the current trigger time period is similar to the driving behavior in the historical trigger time period, so the possibility of being in an abnormal state is lower, so the risk level of driving behavior will also be lower; conversely, when the DTW value is larger, it means that the similarity between the driving behavior time series data in the two trigger time periods is lower, indicating that the possibility of driving behavior in the current trigger time period being in an abnormal state is higher, so the risk level of driving behavior will also be higher. Therefore, the average of the DTW distances between all types of driving behavior time series data in the current trigger time period and the historical trigger time period corresponding to each target moment is taken as the driving behavior risk factor at the current trigger moment. , the larger the driving behavior risk factor is, the higher the risk level of the driving behavior at the current triggering moment is. Finally, the mean values ​​of all driving behavior risk factors corresponding to the current triggering moment are normalized to obtain the driving behavior risk value at the current triggering moment. The larger the driving behavior risk value is, the more abnormal the driving behavior at the current triggering moment is compared with the history, that is, the more dangerous it is.

[0094] It should be noted that calculating the DTW distance using a dynamic time warping algorithm is a well-known technique and will not be described in detail here.

[0095] During the process of vehicle autonomous driving, since the environment and driving behavior will change dynamically, the similar conditions of the environment in which the vehicle is located during the process of autonomous driving can be used to dynamically adjust the driving behavior risk value and the environmental risk value, so as to accurately assess the danger of the vehicle at the current triggering moment. The similar conditions of the environment in which the vehicle is located during the process of autonomous driving can be analyzed by the data to be fused.

[0096] Therefore, preferably, in one embodiment of the present invention, based on the data to be fused before the current triggering moment, the environmental risk value and the driving behavior risk value at the current triggering moment are adjusted to obtain the risk level value, including:

[0097] See also Figure 4 It shows a method flow chart of a method for obtaining a danger level value in one embodiment of the present invention, and the method comprises the following steps:

[0098] Step S411: Analyze the difference between the last data value and all previous data values ​​in the obstacle ratio sequence around the vehicle corresponding to the current triggering moment, and obtain the environment similarity feature value corresponding to the current triggering moment.

[0099] In the aforementioned step S202, the obstacle ratio sequence of the vehicle surroundings corresponding to the current triggering moment is obtained. The data values ​​in the obstacle ratio sequence of the vehicle surroundings can be used to reflect the change in the ratio of obstacles in the vehicle surroundings during the automatic driving process, and the last data value is closest to the current triggering moment in time. Therefore, according to the difference between the last data value and all previous data values ​​in the obstacle ratio sequence of the vehicle surroundings, the environmental similarity feature value between the current triggering moment and the vehicle's historical driving process is quantified, and the value of the environmental similarity feature value is a normalized value. The formula model of the environmental similarity feature value can be, for example, as follows: in, Indicates the environmental similarity feature value corresponding to the current triggering moment; Indicates the total number of data values ​​in the obstacle ratio sequence around the vehicle corresponding to the current trigger moment; Indicates the number of obstacles in the vehicle surrounding ratio sequence corresponding to the current triggering moment. data values; Indicates the front of the obstacle ratio sequence around the vehicle corresponding to the current trigger moment The mean of the data values; Indicates preset parameters; Represents the normalization function.

[0100] In the formula model of the environment similarity eigenvalue, the previous The mean of the data values ​​represents the average level of the proportion of obstacles in the surrounding environment of the vehicle during its historical driving process. The greater the deviation between the proportion of obstacles closest to the current triggering moment and the aforementioned average level, that is, When the value is larger, it can be considered that the obstacle ratio of the vehicle's environment at the current trigger moment is greatly different from the historical situation, that is, the similarity is lower, so the difference value is used as the denominator to achieve logical relationship correction, and then the obtained ratio is normalized to obtain the environmental similarity feature value corresponding to the current trigger moment. The larger the value, the more similar it is to the environmental conditions in the historical driving process.

[0101] It should be noted that the preset parameters The function of is to prevent the denominator from being 0. Here, the value can be 0.001. The specific value can be adjusted according to the implementation scenario and is not limited here.

[0102] Step S412: adjusting the environmental risk value and driving behavior risk value at the current triggering moment based on the environmental similarity feature value to obtain a risk level value.

[0103] The product of the environmental similarity feature value and the driving behavior risk value is taken as the first risk factor, the product of the value after negative correlation mapping of the environmental similarity feature value and the environmental risk value is taken as the second risk factor, and the value after normalization of the sum of the first risk factor and the second risk factor is taken as the risk degree value. The formula model of the risk degree value includes: in, Indicates the danger level value at the current triggering moment; Indicates the environmental similarity feature value corresponding to the current triggering moment; Indicates the driving behavior risk value at the current triggering moment; Indicates the environmental danger value at the current triggering moment; Represents a normalized function. In the formula model of the danger level value, when the environmental similarity characteristic value corresponding to the current trigger moment is large, it means that the environmental characteristics of the vehicle's current surroundings are similar to the environmental characteristics of the historical driving process, so the driving behavior danger value should be considered more, that is, the proportion of the driving behavior danger value is large; conversely, when the environmental similarity characteristic value is small, it means that the environmental characteristics of the vehicle's current surroundings are very different from the environmental characteristics of the historical driving process, then more environmental factors should be considered, that is, the proportion of the environmental danger value is large. Based on the above logic, the product of the environmental similarity characteristic value and the driving behavior danger value is taken as the first danger factor. , multiply the value after negative correlation mapping of the environmental similarity value by the environmental risk value to obtain the second risk factor , and then the sum of the first risk factor and the second risk factor is used as the risk level value. At this time, in the calculation process of the risk level value, when the environmental similarity feature value is large, the driving behavior risk value accounts for a large proportion. On the contrary, when the environmental similarity feature value is small, the environmental risk value accounts for a large proportion. Since the driving environment and risk level faced by the vehicle during the automatic driving process are dynamically changing, the use of a fixed-length window cannot fully adapt to these changes. In the subsequent process, the fusion analysis of the data will lead to a decrease in fusion accuracy. Therefore, the preset window length can be adjusted based on the risk level value at the current trigger moment to obtain a corrected window length.

[0104] Preferably, in one embodiment of the present invention, adjusting the preset window length based on the danger level value at the current triggering moment to obtain the modified window length at the current triggering moment includes:

[0105] The product of the danger level value and the preset window length is rounded up to an integer value as the second adjustment parameter, and the difference between the preset window length and the second adjustment parameter is used as the corrected window length at the current trigger moment. The formula model for correcting the window length includes:

[0106] in, Indicates the correction window length at the current triggering moment; Indicates the preset window length; Indicates the danger level value at the current triggering moment; Represents the ceiling function.

[0107] In the formula model for correcting the window length value, the preset window length is adaptively adjusted based on the risk level value. The product of the risk level value and the preset window length is rounded up, and the resulting value is the amount by which the preset window needs to be adjusted. When the danger level is higher, in order to ensure the real-time and accuracy of the vehicle's automatic driving strategy, the window length corresponding to the current trigger moment should be reduced to a greater extent. On the contrary, when the danger level is lower, in order to use more data to further improve the accuracy of data fusion, the window length corresponding to the current trigger moment will be reduced to a smaller extent. Therefore, the preset window length and The difference is used as the final correction window length at the current trigger moment.

[0108] Step S5: Perform fusion analysis on the data values ​​of each type of data to be fused within the correction window length at the current trigger moment, and obtain the fusion feature value at the current trigger moment to control the automatic driving of the vehicle.

[0109] In the above process, the correction window length of the current triggering moment in different situations is analyzed with the number of timer triggering times as the dividing line, and the data to be fused within the correction window length is the data that needs to be fused and analyzed to determine the fusion feature value at the current triggering moment. All types of data to be fused contain many aspects of information during the vehicle's driving process. Therefore, by controlling the vehicle's automatic driving based on the fusion feature value of each type of data to be fused, the current driving environment and driving status can be evaluated more accurately and comprehensively, ensuring that the correct driving strategy can be formulated when controlling the vehicle's automatic driving.

[0110] Preferably, in one embodiment of the present invention, a fusion analysis is performed on the data value of each type of data to be fused within the correction window length at the current trigger moment to obtain the fusion feature value at the current trigger moment so as to control the automatic driving of the vehicle, including:

[0111] For each type of data to be fused, the data value within the correction window length at the current trigger moment is fitted based on the least squares method, the data value at the current trigger moment is predicted, and it is used as the fusion feature value at the current trigger moment.

[0112] Then, the fusion feature values ​​of all the data to be fused at the current trigger moment are used as the input of the F-OCC model, and the control signal is output. Finally, the built-in actuators in the autonomous driving system, such as the engine, braking system, steering system, etc., receive the control signal and perform corresponding operations to control the vehicle for autonomous driving.

[0113] It should be noted that the method of fitting data based on the least squares method is a well-known technology and will not be described in detail here; the method of obtaining the vehicle control signal based on the F-OCC model is a well-known technology and will not be described in detail here.

[0114] In order to facilitate calculation, all indicator data involved in the calculation in the embodiment of the present invention are preprocessed to eliminate the dimension effect. The specific means of eliminating the dimension effect are technical means well known to those skilled in the art and are not limited here.

[0115] In an embodiment of the present invention, after obtaining the fusion feature values ​​of all the data to be fused at the current trigger moment, that is, realizing time alignment, they can also be spatially positioned, and then the data after positioning is input into the F-OCC model, so as to obtain a control signal for automatic driving of the vehicle; wherein, the process of spatial positioning is a well-known technology and will not be elaborated here.

[0116] In summary, multi-sensor data fusion can make up for the limitations of a single sensor in perceiving the environment. Therefore, in the multi-sensor fusion deployment process, the data to be fused obtained by multiple sensors during the vehicle's automatic driving process is first obtained. In view of the fact that the driving behavior data during the vehicle's automatic driving process can also be used as the basis for subsequent analysis of the vehicle's automatic control strategy, multiple driving behavior time series data are obtained at the same time. Because different sensors have inconsistent preprocessing times for data, in the present invention, a timer is set, and when the timer is triggered, the synchronous processing of multiple data to be fused is realized, thereby performing real-time control of the vehicle's automatic driving process. When the number of historical triggers of the timer does not reach the preset number, it is considered that the vehicle's automatic driving time is too short and the driving behavior data is less, so the environmental changes are mainly considered. The changes in the environment can be quantified by the fluctuations of various sensor data. Therefore, based on the fluctuations of the data to be fused before the current trigger moment, the preset window length is adjusted to obtain the corrected window length at the current trigger moment to quickly adapt to environmental changes; and when the number of historical triggers of the timer reaches the preset number, the system is basically stable at this time, and enough driving behavior data has been accumulated for analysis, so the changes in the environment are combined with the similarities between the driving behavior time series data, and the changes in the environment and the stability of the driving behavior are comprehensively evaluated to adjust the preset window length to obtain the corrected window length at the current trigger moment. By analyzing the changes in the environment and the changes in driving behavior during the vehicle's automatic driving process and dynamically adjusting the window length, the system's ability to respond to complex environments and emergencies can be improved. Finally, the various data to be fused in the correction window are fused and analyzed to improve the accuracy of data fusion. Therefore, when the fusion feature value is obtained to control the vehicle's automatic driving, the perception accuracy of automatic driving can be effectively improved.

[0117] The present invention also proposes an autonomous driving system based on multi-sensor fusion deployment, please refer to Figure 5 , which shows a system block diagram, including: a data acquisition module 501, used to implement step S1 in the above method; an environmental risk value analysis module 502, used to implement step S2 in the above method; a window length adjustment module 503, used to implement steps S3 and S4 in the above method; an automatic driving control module 504, used to implement step S5 in the above method.

[0118] It should be noted that the system provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above. In addition, the above embodiment provides an autonomous driving system based on multi-sensor fusion deployment and an autonomous driving method embodiment based on multi-sensor fusion deployment, which belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0119] See also Figure 6 , which shows a system structure diagram of an automatic driving system based on multi-sensor fusion deployment provided by an embodiment of the present invention, including a processor 600, a memory 601, a bus 602 and a communication interface 603, wherein the processor 600, the communication interface 603 and the memory 601 are connected via the bus 602; wherein the memory 601 may include a high-speed random access memory, the bus 602 may be an ISA bus, a PCI bus or an EISA bus, etc., and the processor 600 may be an integrated circuit chip with signal processing capabilities; the memory 601 stores at least one instruction, at least one program, a code set or an instruction set, and when the at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor, a step in an automatic driving method based on multi-sensor fusion deployment is implemented.

[0120] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0121] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. An autonomous driving method based on multi-sensor fusion deployment, characterized in that: The method comprises: Obtain the data to be fused from multiple sensors during the vehicle's autonomous driving process and multiple driving behavior time series data; Set a timer to determine the environmental risk value at the current triggering moment based on the fluctuation of the data to be fused before the current triggering moment; When the number of historical trigger times of the timer does not reach the preset number, the preset window length is adjusted based on the environmental risk value at the current triggering moment to obtain the corrected window length at the current triggering moment; When the number of historical trigger times of the timer reaches a preset number, the similarity of data values ​​is analyzed in the driving behavior time series data before and at the current triggering moment to determine the driving behavior risk value at the current triggering moment; based on the data to be fused before the current triggering moment, the environmental risk value and the driving behavior risk value at the current triggering moment are adjusted to obtain a risk level value; based on the risk level value at the current triggering moment, the preset window length is adjusted to obtain a corrected window length at the current triggering moment; Perform fusion analysis on the data value of each type of data to be fused within the correction window length at the current trigger moment, and obtain the fusion feature value at the current trigger moment to control the vehicle's automatic driving; The types of data to be fused include IMU data and other types of data to be fused; The determining of the environmental risk value at the current triggering moment based on the fluctuation of the data to be fused before the current triggering moment includes: The data to be fused when the timer is not set before the current trigger moment is set with breakpoints according to preset time intervals, and the average of the data values ​​of each type of data to be fused between two adjacent breakpoints is used as the fusion feature value of each type of data to be fused at the latter breakpoint of the two adjacent breakpoints, and the breakpoints and the trigger moment are collectively referred to as reference moments; Among all the data to be fused, the fused feature values ​​of a preset number of reference moments before the current trigger moment are used as the input of the F-OCC model to obtain the vehicle surrounding obstacle ratio sequence corresponding to the current trigger moment; based on the change trend between the data values ​​in the vehicle surrounding obstacle ratio sequence, the first environmental risk factor at the current trigger moment is determined; Decomposing and calculating the data values ​​in the IMU data at a preset number of reference moments before the current trigger moment, obtaining the corresponding angular velocity sequence and movement distance sequence; comprehensively analyzing the fluctuation of the data values ​​in the angular velocity sequence and the fluctuation of the data values ​​in the movement distance sequence, and determining the second environmental risk factor at the current trigger moment; According to the first environmental risk factor and the second environmental risk factor at the current triggering moment, an environmental risk value at the current triggering moment is obtained, the value of the environmental risk value is a normalized value, and the first environmental risk factor and the second environmental risk factor are both positively correlated with the environmental risk value.

2. The autonomous driving method based on multi-sensor fusion deployment according to claim 1, characterized in that: The step of adjusting the preset window length based on the environmental risk value at the current triggering moment to obtain the corrected window length at the current triggering moment includes: The value obtained by rounding up the product of the environmental risk value and the preset window length is used as the first adjustment parameter, and the difference between the preset window length and the first adjustment parameter is used as the corrected window length at the current triggering moment.

3. The automatic driving method based on multi-sensor fusion deployment according to claim 1, characterized in that: The method for obtaining the driving behavior risk value includes: The absolute value of the difference between the environmental hazard value at each historical triggering moment and the environmental hazard value at the current triggering moment is taken as the environmental hazard difference; the historical triggering moment when the environmental hazard difference is less than the preset difference threshold is taken as the target moment; The time period between two adjacent trigger moments is used as the trigger time period corresponding to the latter of the two adjacent trigger moments; In the current trigger time period corresponding to the current trigger time and the historical trigger time period corresponding to each target time, the DTW distance between the same type of driving behavior time series data is calculated, and the average of the DTW distances between all types of driving behavior time series data is used as the driving behavior risk factor at the current trigger time; All driving behavior risk factors at the current triggering moment are integrated to determine the driving behavior risk value at the current triggering moment.

4. The autonomous driving method based on multi-sensor fusion deployment according to claim 1, characterized in that: The method of adjusting the environmental risk value and the driving behavior risk value at the current triggering moment based on the data to be fused before the current triggering moment to obtain the risk level value includes: In the obstacle ratio sequence around the vehicle corresponding to the current triggering moment, the difference between the last data value and all previous data values ​​is analyzed to obtain the environment similarity feature value corresponding to the current triggering moment, and the value of the environment similarity feature value is a normalized value; The environmental risk value and the driving behavior risk value at the current triggering moment are adjusted based on the environmental similarity feature value to obtain a risk degree value.

5. The automatic driving method based on multi-sensor fusion deployment according to claim 4, characterized in that: The step of adjusting the environmental risk value and the driving behavior risk value at the current triggering moment based on the environmental similarity feature value to obtain a risk level value includes: The product of the environment similarity feature value and the driving behavior risk value is used as the first risk factor, and the product of the value after negative correlation mapping of the environment similarity feature value and the environment risk value is used as the second risk factor; The value obtained by normalizing the sum of the first risk factor and the second risk factor is used as the risk level value.

6. The autonomous driving method based on multi-sensor fusion deployment according to claim 1, characterized in that: The step of adjusting the preset window length based on the danger level value at the current triggering moment to obtain the corrected window length at the current triggering moment includes: The value obtained by rounding up the product of the risk level value and the preset window length is used as the second adjustment parameter, and the difference between the preset window length and the second adjustment parameter is used as the corrected window length at the current triggering moment.

7. The autonomous driving method based on multi-sensor fusion deployment according to claim 1, characterized in that: The method of performing fusion analysis on the data value of each type of data to be fused within the correction window length at the current triggering moment, and obtaining the fusion feature value at the current triggering moment to control the automatic driving of the vehicle includes: For each type of data to be fused, the data value within the correction window length at the current trigger moment is fitted based on the least squares method to predict the fusion feature value at the current trigger moment; The fusion feature values ​​of all the data to be fused at the current triggering moment are used as the input of the F-OCC model, and the control signal is output to control the vehicle to perform autonomous driving.

8. An autonomous driving system based on multi-sensor fusion deployment, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of an autonomous driving method based on multi-sensor fusion deployment as described in any one of claims 1 to 7 are implemented.

Images