Safety belt micro-motion control method based on environmental perception

Abstract

The application discloses a seat belt micro-motion control method based on environment sensing and relates to the field of vehicle safety prompting, and specifically comprises the following steps: collecting environment data through a vehicle-mounted sensor component to obtain real-time image data, a first data set and a second data set; obtaining a field of view condition classification result through a first identification model and the real-time image data; obtaining a traffic early warning element identification result through a second identification model and the real-time image data; selecting an early warning mode based on the field of view condition classification result and the traffic early warning element identification result; when the early warning mode selects a first safety early warning model, calculating a regional early warning reference value based on the first data set; when the early warning mode selects a second safety early warning model, calculating a predicted collision time based on the second data set; and determining a seat belt micro-motion mode based on the field of view condition classification result, the traffic early warning element identification result, a threshold judgment result of the regional early warning reference value, a threshold judgment result of the predicted collision time and a corresponding included angle.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle safety reminders, and in particular to a seat belt micro-motion control method based on environmental perception. BACKGROUND

[0002] In recent years, thanks to the rapid development of communication technology, sensor technology and artificial intelligence, the automobile field has also made continuous innovation breakthroughs, and the informatization level of automobiles has rapidly improved. Whether it is a new energy vehicle or a traditional fuel vehicle, loading an environmental perception system and adding intelligent auxiliary driving or automatic driving functions has become a new trend. Limited by the current technical level, the auxiliary driving mode and the intelligent driving mode return to the manual driving mode in complex or emergency situations. Even for intelligent vehicles with high informatization level, manual driving is still the main driving mode, so it is important to make good use of the idle environmental perception system to warn the driver in the manual driving state. In addition, when warning the driver, the traditional sound and light reminders are not adaptable enough. When the surrounding environment of the driver is noisy, the sound reminder is difficult for the driver to hear, and the light reminder is easy to cause visual interference. Therefore, it is a highly feasible and promising technical method to remind the driver through a contact method.

[0003] At present, the Chinese invention patent with application number CN202310622816.7 discloses a fatigue warning system based on seat belt tightness, which specifically includes a fatigue perception module, a fatigue warning module, and a deactivation module. Among them: the fatigue perception module perceives the fatigue characteristics of the driver, and divides the fatigue degree or determines the disappearance of the fatigue characteristics according to the fatigue characteristics; the fatigue warning module performs fatigue warning through the repeated tightening and loosening movement of the seat belt with different intensities according to the fatigue degree information transmitted by the fatigue perception module; and the deactivation module stops the repeated tightening and loosening movement of the seat belt according to the fatigue characteristic disappearance information transmitted by the fatigue perception module. The invention accurately and effectively reminds the driver of the fatigue driving state through the repeated tightening and loosening of the seat belt with different intensities, and plays a certain relief role, better reducing the accident rate and ensuring driving safety. Since the invention only warns the driver of fatigue, the warning range is limited, and due to the popularity of intelligent auxiliary driving, the normal activity of the driver has been misidentified as fatigue. In addition, the mode of reminding by using the seat belt is single, and only the seat belt with different intensities is used to loosen and tighten to warn the driver, which is easy to be ignored by the driver and lacks warning differentiation, and there is no perfect mechanism for fully utilizing the seat belt for warning. SUMMARY

[0004] The technical problem solved by the present application is that the traditional technology has a single reminding mode using a safety belt, only relies on safety belts with different forces to loosen and tighten to warn the driver, is easy to be ignored by the driver, lacks warning differentiation, and does not establish a perfect mechanism for fully utilizing the safety belt for warning.

[0005] To solve the above technical problems, the present application provides the following technical solutions:

[0006] Step S1, collecting environmental data of the vehicle driving through a vehicle-mounted sensor assembly, the environmental data including real-time image data and radar data, and calculating a first data set and a second data set based on the radar data;

[0007] Step S2, directly extracting features and abstractly extracting features from the collected real-time image data through a first recognition model, and obtaining a visual field condition classification result through feature fusion and classification;

[0008] Step S3, analyzing and processing the collected real-time image through a second recognition model to obtain a traffic warning element recognition result;

[0009] Step S4, judging the environment type based on the visual field condition classification result output by the first recognition model and the traffic warning element recognition result output by the second recognition model, and selecting a warning mode based on the obtained environment type, the environment type including a first environment type, a second environment type and a third environment type;

[0010] The warning mode includes a first safety warning model and a second safety warning model;

[0011] Step S5, when the warning mode selects the first safety warning model, calculating a regional warning reference value based on the first data set and the first safety warning model;

[0012] When the warning mode selects the second safety warning model, an evaluation distance is calculated based on the second data set and the second safety warning model, and a predicted collision time is calculated based on the evaluation distance;

[0013] Step S6, determining a safety belt micro-motion mode based on the visual field condition classification result output by the first recognition model, the traffic warning element recognition result output by the second recognition model, the threshold judgment result of the regional warning reference value output by the first safety warning model, the threshold judgment result of the predicted collision time and the corresponding included angle output by the second safety warning model.

[0014] As a preferred scheme of the safety belt micro-motion control method based on environment perception, the vehicle-mounted sensor includes a radar assembly and a high-definition camera assembly, and the radar assembly includes a laser radar assembly and a millimeter wave radar assembly.

[0015] The radar data, including the laser radar data and the millimeter wave radar data, are collected by the radar assembly;

[0016] The first data set, including the speed of the target traffic entity, the distance between the vehicle and the target traffic entity, the relative speed and the included angle, is calculated based on the millimeter wave radar data through the Doppler effect and the electromagnetic wave propagation equation;

[0017] The second data set, including the sampling time data, the acceleration of the target traffic entity, the distance between the vehicle and the target traffic entity and the relative speed, is calculated based on the laser radar data through the time difference measurement method and the Kalman filtering;

[0018] The high-definition camera assembly obtains the real-time image data by recording through the vehicle-mounted high-definition camera.

[0019] As a preferred scheme of the safety belt micro-motion control method based on environmental perception, the field of view condition level is obtained based on the collected real-time image data, and the field of view condition classification result is obtained by processing and analyzing the real-time image data through the first identification model, wherein the field of view condition classification result includes good field of view condition and poor field of view condition.

[0020] As a preferred scheme of the safety belt micro-motion control method based on environmental perception, the first identification model is a pre-trained fusion feature convolutional neural network model, and the processing and analysis logic of the first identification model for image data is as follows:

[0021] Direct feature extraction is performed on the collected real-time image data, the real-time image is denoised through Gaussian filtering to obtain a smoothed image, the smoothed image is subjected to first color space conversion to obtain a Lab format image, two-dimensional matrix data of a hue b channel is extracted to obtain first image features, the smoothed image is subjected to second color space conversion to obtain a HIS format image, and two-dimensional matrix data of a saturation S channel is extracted to obtain second image features;

[0022] Abstract feature extraction is performed on the collected real-time image data, a 7*7 convolution kernel selection window receptive field is selected, the real-time image is iteratively traversed in steps and the real-time image pixels in the window receptive field are subjected to convolution calculation to obtain image local features, the image local features are input into a 3*3 maximum pooling layer to obtain maximum pooling results, the maximum pooling results are input into four 3*3 basic residual modules connected in series, and abstract feature extraction is completed through the identity mapping path and the convolution path in the residual connection to obtain third image features;

[0023] Global average pooling processing is performed on the first image feature, the second image feature and the third image feature, vector splicing processing is performed through a fully connected layer to obtain a fusion feature, image classification is completed through the fusion feature and a Softmax function, and a visual field condition classification result is obtained;

[0024] The first identification model is based on a first preset image data set for data enhancement in training, and a training set and a verification set are obtained by division in a ratio of 7:3, the first preset image data set includes sunny road pictures, rainy road pictures, hail road pictures, snowy road pictures, foggy road pictures, sandstorm road pictures and tunnel entrance and exit pictures, and two types of labels including a good visual field condition label and a poor visual field condition label are added to the pictures in the preset image data set according to the driving visual field influence degree.

[0025] As a preferred scheme of the safety belt micro-motion control method based on environmental perception, the second identification model is used to process and analyze the collected real-time images to obtain a traffic warning element identification result, the traffic warning elements include a forked intersection, a pedestrian crosswalk line, a deceleration road line, a parallax road line, a stop and yield sign, a yellow light display of a traffic light, a red light display of a traffic light, an inverted triangular signboard and a front vehicle brake light, and the traffic warning element identification result includes a presence of a warning element and an absence of a warning element.

[0026] As a preferred scheme of the safety belt micro-motion control method based on environmental perception, the second identification model is a pre-trained improved model based on Yolov5, and the processing and analysis logic of the second identification model for real-time image data is that a GhostConv module is used to replace a convolution module in a backbone network, a C3Ghost module is used to replace a C3 module, a Ghost feature is generated by performing convolution and linear combination processing on input real-time image data through the improved backbone network, and high-dimensional image features are obtained by splicing processing based on the generated Ghost feature.

[0027] A GSConv module is used to replace a convolution module in a neck network to extract deep features, a channel mixing strategy is used to mix the high-dimensional image features, feature information is uniformly exchanged on different channels, and a fourth image feature is obtained by performing deep separable convolution calculation.

[0028] In the head network, a decoupling detection head is used to replace an original detection head, and an SE attention mechanism is introduced, global average pooling processing is performed on the fourth image feature to obtain a channel-dimension feature vector, deep features are obtained by weighting calculation based on the channel-dimension feature vector and a weight vector, the deep features are processed by a classifier and a regressor in the decoupling detection head to complete boundary box prediction and category prediction, and a traffic warning element identification result is obtained.

[0029] The second recognition model uses the SioU-LOS boundary loss function instead of the original CioU-LOSS loss function during training. It performs data augmentation based on the second preset image dataset and divides the dataset into training and validation sets in a 7:3 ratio. The second preset image dataset consists of traffic warning element images that have been pre-annotated and bounded, including: images of forked intersections, pedestrian crossing lines, deceleration road lines, optical illusion road lines, stop and yield signs, traffic lights showing yellow, traffic lights showing red, inverted triangle signs, and images of the brake lights of the vehicle in front illuminated.

[0030] As a preferred embodiment of the seat belt micro-motion control method based on environmental perception described in this invention, the environmental type is determined based on the output results of the first recognition model and the second recognition model, and the environmental type includes: a first environmental type, a second environmental type and a third environmental type;

[0031] The early warning method is selected based on the environment type. The early warning method includes a first security early warning model and a second security early warning model. The logic for determining the environment type and selecting the corresponding early warning method is as follows:

[0032] When the visual condition classification result output by the first recognition model is good and the traffic warning element recognition result of the second recognition model is a warning element, it is judged as the first environment type, and the second safety warning model is selected as the warning method.

[0033] When the visual condition classification result output by the first recognition model is "good visual condition" and the traffic warning element recognition result of the second recognition model is "no warning element", it is judged as the second environment type, and the first safety warning model is selected as the warning method.

[0034] When the visual field condition classification result output by the first recognition model is poor, it is judged as the third environment type, and the first safety warning model is selected as the warning method.

[0035] As a preferred embodiment of the seatbelt micro-motion control method based on environmental perception described in this invention, when the warning mode is selected as the first safety warning model, the speed data of the target traffic entity, the distance data between the vehicle and the target traffic entity, the relative speed data, and the included angle data of the target traffic entity are read from the first dataset. The regional warning reference value is calculated through the first safety warning model, and its calculation expression is as follows:

[0036]

[0037] Where SQ represents the warning reference value, d r d represents relative distance. v Indicates the safe distance, 'a' represents the vehicle's acceleration, and 'v' represents the acceleration of the vehicle.a represents absolute speed, v r represents relative speed, represents the included angle formed by the target traffic entity and the vehicle center along the vehicle head direction of the vehicle centerline, represents the corresponding sine value of the included angle formed by the target traffic entity and the vehicle center along the vehicle head direction of the vehicle centerline, represents the corresponding cosine value of the included angle formed by the target traffic entity and the vehicle center along the vehicle head direction of the vehicle centerline, represents an exponential constant, represents an environmental constant, represents a cosine coefficient, represents a linear coefficient, represents a speed index, and represents a mass constant.

[0038] As a preferred solution of the safety belt micro-motion control method based on environmental perception, when the pre-warning mode selects the second safety pre-warning model, the sampling time data, the acceleration of the target traffic entity, the distance between the vehicle and the target traffic entity, and the relative speed in the second data set are read, the evaluation distance is calculated through the relative motion distance equation, and the predicted collision time is calculated based on the evaluation distance, and the calculation expression is:

[0039] D P = D r + [V r + |a l -a g | (nT S + σ) ] nT S + 1 / 2 |a l -a g | (nT S + σ) 2

[0040] wherein D P represents the evaluation distance, D r represents the actual distance, V r represents the relative speed, a l represents the acceleration of the vehicle, a g represents the acceleration of the target traffic entity, represents the T S sampling time, n represents the frequency, σ represents the time constant, and |·| represents the absolute value.

[0041] As a preferred solution of the safety belt micro-motion control method based on environmental perception, when the first identification model outputs the field of view condition classification result from good field of view condition to poor field of view condition, the safety belt enters the first micro-motion mode.

[0042] When the traffic pre-warning element identification result output by the second identification model changes from no pre-warning element to the presence of a pre-warning element, the safety belt enters the first micro-motion mode.

[0043] When the area warning reference value output by the first safety warning model is greater than or equal to the warning reference threshold THR1, the second micro-motion mode is triggered;

[0044] When the predicted collision time output by the second safety warning model is greater than or equal to the predicted collision time threshold THR2, and the included angle between the target traffic entity and the vehicle center along the vehicle head direction of the vehicle centerline is within the range of 0° to 80° and 280° to 360°, the second micro-motion mode is triggered;

[0045] When the predicted collision time output by the second safety warning model is greater than or equal to the predicted collision time threshold THR2, and the included angle between the target traffic entity and the vehicle center along the vehicle head direction of the vehicle centerline is within the range of 80° to 180°, the third micro-motion mode is triggered;

[0046] When the predicted collision time output by the second safety warning model is greater than or equal to the predicted collision time threshold THR2, and the included angle between the target traffic entity and the vehicle center along the vehicle head direction of the vehicle centerline is within the range of 180° to 280°, the fourth micro-motion mode is triggered;

[0047] The first micro-motion mode specifically is that the safety belt is tightened according to a first tightening intensity and a first tightening frequency, and all M vibration units are synchronously vibrated according to a first vibration intensity, and the first micro-motion mode lasts for t1 seconds;

[0048] The second micro-motion mode specifically is that the safety belt is tightened according to a second tightening intensity, and all M vibration units are synchronously vibrated according to a second vibration intensity, and the second micro-motion mode lasts for t2 seconds;

[0049] The third micro-motion mode specifically is that the safety belt is tightened according to a second tightening intensity, N vibration units on the left upper shoulder are continuously vibrated according to a second vibration intensity for t3 seconds, the remaining M-N vibration units are sequentially activated in order from the right lower abdomen to the left upper shoulder, and after all the M-N vibration units are activated, they are stopped for t4 seconds and circulate, and the third micro-motion mode lasts for t3 seconds;

[0050] The fourth micro-motion mode specifically is that the safety belt is tightened according to a second tightening intensity, N vibration units on the right lower abdomen are continuously vibrated according to a second vibration intensity, and the remaining M-N vibration units are sequentially activated in order from the left upper shoulder to the right lower abdomen and circulate, and after all the M-N vibration units are activated, they are stopped for t4 seconds and circulate, and the fourth micro-motion mode lasts for t3 seconds.

[0051] The beneficial effects of the present application: by feature fusion and basic residual unit, the image features extracted by different convolution layers are fully utilized, which is conducive to improving the accuracy of the field of view condition classification result. Through the lightweight improvement of the Yolov5 model, the calculation speed and the accuracy of the traffic warning element identification are considered. Through the field of view condition classification, traffic warning element identification and warning model selection, the environmental factors and traffic condition factors are considered, and the first safety warning model and the second safety warning model are directly calculated to avoid using complex traditional target tracking algorithm, effectively reducing the calculation complexity. Considering that the laser radar has long measurement distance and high accuracy, it is used to directly calculate the predicted collision time between the vehicle and the front and rear direction traffic entities, and the warning speed is fast and accurate. At the same time, considering the characteristics that the laser radar is easily affected by the environment, the characteristics of the millimeter wave radar that the measurement range is close and the measurement angle are used to calculate the regional warning reference value in the bad weather environment with poor field of view, and the advantages are fully utilized. The adoption of four kinds of micro-motion modes is conducive to reminding the driver with distinction, wherein the first micro-motion mode is based on image data to complete the identification of field of view condition deterioration, appearance of bifurcation intersection, traffic light turning red or yellow, appearance of deceleration or stop road sign, appearance of prompt road line and front vehicle brake light, the identification result contains the traffic scene elements that need to be paid attention to in driving, when these traffic scene elements appear, the driver needs to be reminded but the safety risk is low, therefore, low-level vibration intensity is adopted, and the safety belt is alternately tightened and loosened. The second micro-motion mode, the third micro-motion mode and the fourth micro-motion mode have high warning degree, and the safety belt is only tightened but not loosened, which prevents possible impact and enhances the driver's feeling of vibration. In addition, the third micro-motion mode and the fourth micro-motion mode adopt directional reciprocating vibration, which is simple and easy to be recognized by the driver, and is conducive to reminding the driver to pay attention to the warning situation of the left and right rear of the vehicle. By dividing the intensity of the four micro-motion modes, the driver is warned through touch at the same time, and frequent strong vibration reminders are avoided. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 The basic flow diagram of the safety belt micro-motion control method based on environmental perception provided by an embodiment of the present application is shown in the figure.

[0053] Figure 2 The frame diagram of the safety belt micro-motion control method based on environmental perception provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0054] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments.

[0055] Embodiment 1, refer to Figure 1 and Figure 2For an embodiment of the present application, a seat belt micro-control method based on environmental perception is provided, specifically comprising:

[0056] Step S1, collecting environmental data of the vehicle driving through a vehicle-mounted sensor assembly, the environmental data including real-time image data and radar data, and calculating a first data set and a second data set based on the radar data;

[0057] Step S2, directly extracting features and abstractly extracting features from the collected real-time image data through a first recognition model, and obtaining a field of view condition classification result through feature fusion and classification;

[0058] Step S3, analyzing and processing the collected real-time image through a second recognition model to obtain a traffic warning element recognition result;

[0059] Step S4, judging the environmental type based on the field of view condition classification result output by the first recognition model and the traffic warning element recognition result output by the second recognition model, and selecting a warning mode based on the obtained environmental type, the environmental type including a first environmental type, a second environmental type and a third environmental type;

[0060] The warning mode includes a first safety warning model and a second safety warning model;

[0061] Step S5, when the warning mode selects the first safety warning model, calculating a regional warning reference value based on the first data set and the first safety warning model;

[0062] When the warning mode selects the second safety warning model, an evaluation distance is calculated based on the second data set and the second safety warning model, and a predicted collision time is calculated based on the evaluation distance;

[0063] Step S6, determining the seat belt micro-control mode based on the field of view condition classification result output by the first recognition model, the traffic warning element recognition result output by the second recognition model, the threshold judgment result of the regional warning reference value output by the first safety warning model, the threshold judgment result of the predicted collision time and the corresponding included angle output by the second safety warning model.

[0064] In this embodiment, the vehicle-mounted sensor includes a millimeter wave radar assembly, a laser radar assembly and a high-definition camera assembly, radar data is collected through the radar assembly, including laser radar data and millimeter wave radar data;

[0065] The millimeter wave radar assembly is composed of four millimeter wave radars arranged at four corners of the vehicle respectively, and the detection directions form 45°, 135°, 225° and 315° angles with the vehicle center along the head direction of the middle axis, and the first data set is calculated based on the millimeter wave radar data through Doppler effect and electromagnetic wave propagation equation, the first data set including the speed of the target traffic entity, the distance between the vehicle and the target traffic entity, the relative speed and the included angle;

[0066] The laser radar assembly is composed of two laser radars arranged at the central positions of the front bumper and the rear bumper of the vehicle respectively, and the second data set is calculated based on the laser radar data through time difference measurement method and Kalman filtering, the second data set including sampling time data, acceleration of the target traffic entity, distance between the vehicle and the target traffic entity and relative speed;

[0067] The high-definition camera is arranged at the central position above the front windshield of the vehicle, and real-time image data is obtained by video recording through the vehicle-mounted high-definition camera.

[0068] In this embodiment, the field of view condition level is obtained based on the collected real-time image data, including: processing and analyzing the real-time image data through the first recognition model to obtain a field of view condition classification result, the field of view condition classification result including good field of view condition and poor field of view condition.

[0069] In this embodiment, the first recognition model is a pre-trained fusion feature convolutional neural network model, and the processing and analysis logic of the first recognition model for image data is as follows:

[0070] Direct feature extraction is performed on the collected real-time image data, the real-time image is denoised through Gaussian filtering to obtain a smoothed image, the smoothed image is converted into a Lab format image through first color space conversion, a two-dimensional matrix data of the hue b channel is extracted to obtain a first image feature, the smoothed image is converted into a HIS format image through second color space conversion, and a two-dimensional matrix data of the saturation S channel is extracted to obtain a second image feature.

[0071] Abstract feature extraction is performed on the collected real-time image data, a 7*7 convolution kernel selection window receptive field is selected, the real-time image is iteratively traversed and the real-time image pixels in the window receptive field are convolved to obtain image local features, the image local features are input into a 3*3 maximum pooling layer to obtain maximum pooling results, the maximum pooling results are input into four 3*3 basic residual modules connected in series, and abstract feature extraction is completed through the identity mapping path and the convolution path in the residual connection to obtain a third image feature.

[0072] The first image feature, the second image feature and the third image feature are subjected to global average pooling processing, vector splicing processing is performed through a fully connected layer to obtain a fusion feature, image classification is completed through the fusion feature and a Softmax function, and a field of view condition classification result is obtained;

[0073] The first identification model is subjected to data enhancement based on a first preset image data set in training, and a training set and a verification set are obtained by division in a ratio of 7:3, the first preset image data set includes sunny road pictures, rainy road pictures, hail road pictures, snowy road pictures, foggy road pictures, sandstorm road pictures and tunnel entrance and exit pictures, and two types of labels including a good field of view condition label and a poor field of view condition label are added to the pictures in the preset image data set according to driving field of view influence degrees;

[0074] Among them, the hue b channel and the saturation S channel data can best reflect the visibility level and have low calculation complexity, feature fusion is conducive to accurate classification of real-time images based on the field of view visibility of the model, and the introduction of the basic residual unit fully utilizes the features extracted by different convolution layers, which is conducive to expressing image features of different scales and levels and improves the accuracy of the model.

[0075] In the embodiment, the traffic warning element recognition result is obtained by processing and analyzing the collected real-time image through the second identification model, the traffic warning element includes a fork intersection, a pedestrian crosswalk line, a speed reduction line, a misperception road line, a stop and yield sign, a traffic light displaying yellow, a traffic light displaying red, an inverted triangular sign and a front vehicle brake light, and the traffic warning element recognition result includes the presence of a warning element and the absence of a warning element.

[0076] In the embodiment, the second identification model is a pre-trained improved model based on Yolov5, and the processing and analysis logic of the second identification model for real-time image data is that a GhostConv module is used instead of a convolution module in a backbone network, a C3Ghost module is used instead of a C3 module, a Ghost feature is generated by convolution and linear combination processing of the input real-time image data through the improved backbone network, and high-dimensional image features are obtained by splicing processing based on the generated Ghost feature.

[0077] A GSConv module is used instead of a convolution module in a neck network to extract deep features, and the high-dimensional image features are subjected to channel shuffling processing through a shuffle mixing strategy, the feature information is uniformly exchanged on different channels, and the fourth image feature is obtained through deep separable convolution calculation;

[0078] In the head network, the decoupling detection head is used instead of the original detection head, and the SE attention mechanism is introduced. The fourth image feature is globally averaged and pooled to obtain a feature vector in the channel dimension. The deep feature is obtained by weighting calculation based on the feature vector in the channel dimension and the weight vector. The deep feature is processed by the parallel classifier and regressor in the decoupling detection head to complete the boundary box prediction and class prediction, and the traffic warning element recognition result is obtained.

[0079] The second identification model uses the SioU-LOS boundary loss function instead of the original CioU-LOSS loss function during training, and performs data enhancement based on a second preset image data set. The training set and the validation set are obtained by dividing according to the ratio of 7:3. The second preset image data set is composed of traffic warning element pictures that are pre-labeled and framed, including: split intersection pictures, pedestrian crosswalk line pictures, deceleration road line pictures, parallax road line pictures, stop and yield sign pictures, traffic light showing yellow light pictures, traffic light showing red light pictures, inverted triangular sign pictures and front vehicle brake light on pictures.

[0080] Among them, through the lightweight improvement of the Yolov5 model, the image features are fully extracted, and the calculation speed and recognition accuracy are considered.

[0081] In this embodiment, the output results of the first identification model and the second identification model are used to judge the environment type, and the environment type includes: a first environment type, a second environment type and a third environment type.

[0082] The warning mode is selected based on the environment type, and the warning mode includes a first safety warning model and a second safety warning model. The logic for judging the environment type and selecting the corresponding warning mode is as follows:

[0083] When the first identification model outputs a visual field condition classification result of good visual field condition and the second identification model traffic warning element recognition result exists a warning element, it is judged as the first environment type, and the warning mode selects the second safety warning model.

[0084] When the first identification model outputs a visual field condition classification result of good visual field condition and the second identification model traffic warning element recognition result is no warning element, it is judged as the second environment type, and the warning mode selects the first safety warning model.

[0085] When the first identification model outputs a visual field condition classification result of poor visual field condition, it is judged as the third environment type, and the warning mode selects the first safety warning model.

[0086] In this embodiment, the relative distance data, the relative speed data and the angle data between the vehicle and the target traffic entity are obtained based on the first data set, and the regional warning reference value is calculated by the first safety warning model. The calculation expression is:

[0087]

[0088] wherein SQ represents a warning reference value, d r represents a relative distance, d v represents a safety distance, a represents a vehicle acceleration, v a represents an absolute speed, v r represents a relative speed, θ represents an included angle formed by a target traffic entity and a vehicle center along a vehicle head direction of a vehicle centerline, sinθ represents a corresponding sine value of the included angle formed by the target traffic entity and the vehicle center along the vehicle head direction of the vehicle centerline, cosθ represents a corresponding cosine value of the included angle formed by the target traffic entity and the vehicle center along the vehicle head direction of the vehicle centerline, γ represents an exponential constant, ε represents an environmental constant, δ represents a cosine coefficient, k represents a linear coefficient, μ represents a speed index, and c represents a mass constant.

[0089] In the embodiment, the calculation of the second safety warning model is based on the lidar data, and the relative distance data, the relative speed data, and the sampling time data between the vehicle and the target traffic entity are obtained based on the lidar data. The evaluation distance is calculated through a relative motion distance equation, and the predicted collision time is calculated based on the evaluation distance. The calculation expression is as follows:

[0090] D P = D r + [V r + |a l -a g | (nT S + σ) ] nT S + 1 / 2 |a l -a g | (nT S + σ) 2

[0091] wherein D P represents the evaluation distance, D r represents the actual distance, V r represents the relative speed, a l represents the vehicle acceleration, a g represents the acceleration of the target traffic entity, nT S represents the sampling time, n represents the frequency, σ represents the time constant, and |·| represents the absolute value.

[0092] The consideration of the environmental factors and the traffic condition factors is completed through the field of view condition classification, the traffic early warning element identification and the early warning model selection, the first safety early warning model and the second safety early warning model are directly calculated to avoid the use of a complex traditional target tracking algorithm, and the calculation complexity is effectively reduced. Considering that the laser radar has a long measuring distance and is accurate, the laser radar is used to directly calculate the predicted collision time between the vehicle and the front and rear direction traffic entities, the early warning speed is fast and accurate. Meanwhile, considering that the laser radar is easily affected by the environment, the millimeter wave radar is used to calculate the regional early warning reference value in the poor weather environment with poor field of view, and the advantages are fully utilized and the disadvantages are avoided.

[0093] In the embodiment, when the field of view condition classification result output by the first identification model changes from good to poor, the safety belt enters the first micro-motion mode;

[0094] When the traffic early warning element identification result output by the second identification model changes from no early warning element to the presence of early warning elements, the safety belt enters the first micro-motion mode;

[0095] When the regional early warning reference value output by the first safety early warning model is greater than or equal to the early warning reference threshold THR1, the second micro-motion mode is triggered;

[0096] When the predicted collision time output by the second safety early warning model is greater than or equal to the predicted collision time threshold THR2, and the included angle formed by the target traffic entity and the vehicle center along the vehicle center axis in the vehicle head direction belongs to the range of 0° to 80° and 280° to 360°, the second micro-motion mode is triggered;

[0097] When the predicted collision time output by the second safety early warning model is greater than or equal to the predicted collision time threshold THR2, and the included angle formed by the target traffic entity and the vehicle center along the vehicle center axis in the vehicle head direction is in the range of 80° to 180°, the third micro-motion mode is triggered;

[0098] When the predicted collision time output by the second safety early warning model is greater than or equal to the predicted collision time threshold THR2, and the included angle formed by the target traffic entity and the vehicle center along the vehicle center axis in the vehicle head direction is in the range of 180° to 280°, the fourth micro-motion mode is triggered;

[0099] The first micro-motion mode specifically is that the safety belt is tightened according to a first level tightening intensity and a first level tightening frequency, and all M vibration units are synchronously vibrated according to a first level vibration intensity, and the first micro-motion mode duration is t1 seconds;

[0100] The second micro-motion mode specifically is that the safety belt is tightened according to a second level tightening intensity, and all M vibration units are synchronously vibrated according to a second level vibration intensity, and the second micro-motion mode duration is t2 seconds;

[0101] The third micro-motion mode is specifically that the safety belt is tightened according to the secondary tightening strength, the N vibration units on the left upper shoulder continuously vibrate according to the secondary vibration strength for t3 seconds, the remaining M-N vibration units are started in turn in the order from the right lower abdomen to the left upper shoulder, all the M-N vibration units are stopped for t4 seconds and circulate after being started, the circulation is repeated, and the third micro-motion mode lasts for t3 seconds.

[0102] The fourth micro-motion mode is specifically that the safety belt is tightened according to the secondary tightening strength, the N vibration units on the right lower abdomen continuously vibrate according to the secondary vibration strength, the remaining M-N vibration units are started in turn in the order from the left upper shoulder to the right lower abdomen and circulate, all the M-N vibration units are stopped for t4 seconds and circulate after being started, and the fourth micro-motion mode lasts for t3 seconds.

[0103] Among them, the four micro-motion modes are beneficial to reminding the driver in a differentiated manner. The first micro-motion mode is based on image data to complete the identification of the poor visual field condition, the appearance of a fork in the road, the traffic light turning red or yellow, the appearance of a speed reduction or stop sign, the appearance of a prompt road line, and the brake light of the front vehicle. The identification result contains the traffic scene elements that need to be paid attention to in driving. When these traffic scene elements appear, they need to be reminded, but the safety risk is low, so low-level vibration strength is adopted, and the safety belt is alternately tightened and loosened. The second micro-motion mode, the third micro-motion mode and the fourth micro-motion mode have high warning degree, the safety belt is only tightened but not loosened, which prevents possible impact and enhances the driver's feeling of vibration. In addition, the third micro-motion mode and the fourth micro-motion mode adopt reciprocating vibration with direction, which is simple and easy to be recognized by the driver, and is beneficial to reminding the driver to pay attention to the warning situation existing in the left and right rear of the vehicle.

[0104] Those skilled in the art will understand that embodiments of the present invention can provide methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0105] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A seatbelt micro-motion control method based on environmental perception, characterized in that, include: Step S1: Collect environmental data of the vehicle while it is in motion through the vehicle-mounted sensor components. The environmental data includes real-time image data and radar data. Calculate the first dataset and the second dataset based on the radar data. Step S2: The first recognition model performs direct feature extraction and abstract feature extraction on the acquired real-time image data, and obtains the field of view classification result through feature fusion and classification. The visual field condition classification results include good visual field condition and poor visual field condition; Step S3: The collected real-time images are analyzed and processed by the second recognition model to obtain the traffic warning element recognition results; The traffic warning element identification results include whether warning elements exist or not. Step S4: Based on the visual condition classification results output by the first recognition model and the traffic warning element recognition results output by the second recognition model, determine the environment type, and select the warning method based on the obtained environment type. The environment type includes a first environment type, a second environment type and a third environment type. The early warning methods include a first security early warning model and a second security early warning model; Step S5: When the first security early warning model is selected as the early warning method, the regional early warning reference value is calculated based on the first dataset and the first security early warning model. When the second safety warning model is selected as the warning method, the evaluation distance is calculated based on the second dataset and the second safety warning model, and the predicted collision time is calculated based on the evaluation distance. Step S6: Determine the seat belt micro-motion mode based on the field of vision classification results output by the first recognition model, the traffic warning element recognition results output by the second recognition model, the threshold judgment results of the regional warning reference values ​​output by the first safety warning model, and the threshold judgment results of the predicted collision time and corresponding included angle output by the second safety warning model. Among them, the environment type is determined based on the output results of the first recognition model and the second recognition model, and the environment type includes the first environment type, the second environment type and the third environment type; The early warning method is selected based on the environment type. The early warning method includes a first security early warning model and a second security early warning model. The logic for determining the environment type and selecting the corresponding early warning method is as follows: When the visual condition classification result output by the first recognition model is good and the traffic warning element recognition result of the second recognition model is that there are warning elements, it is judged as the first environment type, and the warning method is selected as the second safety warning model. When the visual condition classification result output by the first recognition model is "good visual condition" and the traffic warning element recognition result of the second recognition model is "no warning element", it is judged as the second environment type, and the first safety warning model is selected as the warning method. When the visual field condition classification result output by the first recognition model is poor, it is judged as the third environment type, and the first safety warning model is selected as the warning method. The first recognition model is a pre-trained convolutional neural network model with fused features. The processing and analysis logic of the first recognition model for image data is as follows: Direct feature extraction is performed on the acquired real-time image data. The real-time image is denoised by Gaussian filtering to obtain a smooth image. The smooth image is then converted to a first color space to obtain a Lab format image. The two-dimensional matrix data of the hue b channel is extracted to obtain the first image feature. The smooth image is then converted to a second color space to obtain a HIS format image. The two-dimensional matrix data of the saturation S channel is extracted to obtain the second image feature. Abstract feature extraction is performed on the acquired real-time image data. The receptive field of the window is selected by the convolution kernel. The real-time image is traversed step by step and the real-time image pixels in the receptive field are convolved to obtain the local image features. The local image features are input into the max pooling layer to obtain the max pooling result. The max pooling result is input into four cascaded basic residual modules. The abstract feature extraction is completed through the identity mapping path and convolution path in the residual connection to obtain the third image feature. Global average pooling is applied to the first image features, the second image features, and the third image features. Vector concatenation is performed through a fully connected layer to obtain fused features. Image classification is then performed using the fused features and the Softmax function to obtain the field of view classification result. The first recognition model is trained by performing data augmentation based on a first preset image dataset, and the training set and validation set are obtained by dividing the dataset according to a certain ratio. The first preset image dataset includes: sunny road images, rainy road images, hail road images, snowy road images, foggy road images, sandstorm road images, and tunnel entrance and exit images. The images in the preset image dataset are labeled with two types of labels according to the degree of impact on driving vision, including labels with good vision conditions and labels with poor vision conditions. The second recognition model is a pre-trained improved model based on Yolov5. The processing and analysis logic of the second recognition model for real-time image data is as follows: in the backbone network, the GhostConv module is used to replace the convolution module, and the C3Ghost module is used to replace the C3 module. The improved backbone network performs convolution and linear combination processing on the input real-time image data to generate Ghost features. Based on the generated Ghost features, high-dimensional image features are obtained by splicing. In the neck network, the GSConv module is used instead of the convolution module for deep feature extraction. The high-dimensional image features are processed by channel shuffling through a shuffle mixing strategy. Feature information is uniformly exchanged on different channels and the fourth image feature is obtained by depth-separable convolution. In the head network, a decoupled detection head is used to replace the original detection head and an SE attention mechanism is introduced. The fourth image features are processed by global average pooling to obtain the channel dimension feature vector. The deep features are obtained by weighted calculation based on the channel dimension feature vector and the weight vector. The deep features are processed by the classifier and regressor connected in parallel in the decoupled detection head to complete the bounding box prediction and category prediction, and the traffic warning element recognition result is obtained. The second recognition model uses the SioU-LOS boundary loss function instead of the original CioU-LOSS loss function during training. It performs data augmentation based on the second preset image dataset and divides it into training and validation sets according to a certain ratio. The second preset image dataset consists of traffic warning element images that have been pre-annotated and bounded, including images of intersections, pedestrian crossing lines, deceleration road lines, optical illusion road lines, stop and yield signs, traffic lights showing yellow lights, traffic lights showing red lights, inverted triangle signs, and images of the brake lights of the vehicle in front illuminated. When the first safety warning model is selected as the warning method, the speed data of the target traffic entity, the distance data between the vehicle and the target traffic entity, the relative speed data, and the included angle data of the target traffic entity are read from the first dataset. The regional warning reference value is calculated using the first safety warning model, and its calculation expression is as follows: Where SQ represents the warning reference value, Indicates relative distance. This indicates the safe distance, and 'a' represents the vehicle's acceleration. Represents absolute velocity. Represents relative velocity. This represents the angle formed between the target traffic entity and the centroid of the vehicle along the vehicle's centerline in the direction of its frontal movement. This represents the sine value of the angle formed between the target traffic entity and the centroid of the vehicle along the vehicle's centerline in the direction of the vehicle's front. This represents the cosine value of the angle formed between the target traffic entity and the centroid of the vehicle along the vehicle's centerline in the direction of its frontal movement. Represents an exponential constant. Represents environmental constants. Let k denote the cosine coefficient, and k denote the linear coefficient. Indicates the speed index, Indicates the mass constant; When the second safety warning model is selected as the warning method, the sampling time data, the acceleration of the target traffic entity, the distance and relative speed between the vehicle and the target traffic entity are read from the second dataset. The assessment distance is calculated through the relative motion distance equation, and the predicted collision time is calculated based on the assessment distance. The calculation expression is as follows: in, Indicates the distance being assessed. Indicates the actual distance. Represents relative velocity. This indicates that the vehicle is accelerating. Represents the acceleration of the target traffic entity, representing Sampling time, Indicates frequency. Represents the time constant. Indicates taking the absolute value; The logic for determining the seatbelt micro-motion model is as follows: When the visual field condition classification result output by the first recognition model changes from good visual field condition to poor visual field condition, the seat belt enters the first micro-motion mode. When the traffic warning element identification result output by the second identification model changes from no warning element to having a warning element, the seat belt enters the first micro-motion mode. When the regional warning reference value output by the first security warning model is greater than or equal to the warning reference threshold At that time, the second micro-motion mode is triggered; When the predicted collision time output by the second safety warning model is greater than or equal to the predicted collision time threshold Furthermore, the angle formed between the target traffic entity and the centroid of the vehicle along the vehicle's centerline in the direction of the vehicle's front end belongs to... and When the range is reached, the second micro-motion mode is triggered; When the predicted collision time output by the second safety warning model is greater than or equal to the predicted collision time threshold And the angle formed by the target traffic entity and the centroid of the vehicle along the front direction of the vehicle's centerline is When the range is reached, the third micro-motion mode is triggered; When the predicted collision time output by the second safety warning model is greater than or equal to the predicted collision time threshold And the angle formed by the target traffic entity and the centroid of the vehicle along the front direction of the vehicle's centerline is When the range is reached, the fourth micro-motion mode is triggered; The first micro-motion mode specifically involves: the seatbelt tightening according to a first-level tightening intensity and a first-level releasing frequency; all M vibration units vibrating synchronously according to a first-level vibration intensity; and the duration of the first micro-motion mode is... Second; The second micro-motion mode specifically involves: the seatbelt tightening at a secondary tightening intensity, and all M vibration units vibrating synchronously at a secondary vibration intensity. The duration of the second micro-motion mode is... Second; The third micro-motion mode specifically involves: the seatbelt tightening at a secondary tightening intensity, and N vibration units in the upper left shoulder area continuously vibrating at a secondary vibration intensity. After a few seconds, the remaining MN vibration units will start vibrating sequentially from the lower right abdomen to the upper left shoulder, and will stop vibrating after all MN vibration units have started. The cycle repeats in seconds, and the duration of the third micro-motion mode is... Second; The fourth micro-motion mode is as follows: the seat belt is tightened according to the second-level tightening intensity; N vibration units in the lower right abdomen vibrate continuously according to the second-level vibration intensity; the remaining MN vibration units are activated sequentially from the upper left shoulder to the lower right abdomen and cycle through the vibration. The vibration stops after all MN vibration units are activated. The cycle repeats in seconds, and the duration of the fourth micro-motion mode is... Second.

2. The seatbelt micro-motion control method based on environmental perception as described in claim 1, characterized in that: The vehicle-mounted sensor includes a radar assembly and a high-definition camera assembly, and the radar assembly includes a lidar assembly and a millimeter-wave radar assembly. Radar data, including lidar data and millimeter-wave radar data, is acquired through radar components. Based on millimeter-wave radar data, a first dataset is calculated using the Doppler effect and electromagnetic wave propagation equation. The first dataset includes the speed of the target traffic entity, the distance between the vehicle and the target traffic entity, the relative speed, and the included angle. Based on lidar data, a second dataset is obtained by time difference measurement and Kalman filtering. The second dataset includes sampling time data, acceleration of the target traffic entity, distance and relative speed between the vehicle and the target traffic entity. The high-definition camera assembly obtains real-time image data by recording video through an in-vehicle high-definition camera.

3. The seatbelt micro-motion control method based on environmental perception as described in claim 1, characterized in that: The visual field status level is obtained based on the acquired real-time image data, and the visual field status classification result is obtained by processing and analyzing the real-time image data through the first recognition model.

4. The seatbelt micro-motion control method based on environmental perception as described in claim 1, characterized in that: The second recognition model processes and analyzes the collected real-time images to obtain the traffic warning element recognition results. The traffic warning elements include road forks, pedestrian crossing lines, deceleration road lines, optical illusion road lines, stop and yield signs, traffic lights showing yellow, traffic lights showing red, inverted triangle signs, and the brake lights of the vehicle in front illuminating.

Images