Auxiliary driving system for identifying dangerous dynamic target

Abstract

The invention discloses an auxiliary driving system for identifying a dangerous dynamic target. The auxiliary driving system comprises a high-low quality data set generation module and an identification model, the high and low quality data set generation module is used for generating a high quality data set, a low quality data set, a time mask and a space mask; the low-quality data set is obtained by performing space mask S and time mask ST on the high-quality data set; the identification model is used for identifying dangerous dynamic targets in the low-quality data; and training the recognition model by using the high-quality data set and the low-quality data set to obtain a trained recognition model. According to the method, the dangerous target can be identified under the condition that the video quality is reduced due to the figure jittering of a driver under the condition of a relatively low speed in a city. According to the method, on the basis of fully utilizing the dangerous target in the original Yolo-fast detection single-frame image, the dangerous dynamic target is further analyzed for unstable video data.

Description

Technical Field

[0001] The present invention relates to a target recognition algorithm, and in particular to an auxiliary driving system for recognizing dangerous dynamic targets. Background of the Invention

[0003] Existing motorcycle helmets have the function of passively protecting the driver's head safety during driving, but do not have the ability to actively detect danger and warn the driver.

[0004] The miniaturization of existing cameras, the miniaturization of computing hardware, the lightweight algorithm software, and the engineering of VR\AR technology can already constitute a smart helmet system for assisted driving. In the scenario where a smart helmet is used, unlike the stable assisted driving system on a car, it is mounted on the driver's head and is an unstable system. The data obtained from the camera is also unstable and of low quality, which seriously affects the subsequent work of dangerous target identification. In this case, if only single-frame static image recognition is used, first, inaccurate target recognition may occur; second, dangerous actions of the target cannot be predicted.

[0005] After analysis, it is found that for this problem, under relatively low-speed conditions in the city, the redundant data of 30 frames per second can be used, that is, the continuous and stable partial features can be found from the information of multiple consecutive frames of unstable images, that is, the time redundancy can be fully utilized to complete the feature reconstruction task, so that the encoder can learn useful features. Then, recognition is performed to identify the dangerous action targets that need to be identified during assisted driving and provide them to the driver. Summary of the invention

[0006] The technical problem to be solved by the present invention is to identify targets when the input data is unstable; the present invention proposes an assisted driving system for identifying dangerous dynamic targets, including a high- and low-quality data set generation module and a recognition model; the high- and low-quality data set generation module is used to generate a high-quality data set, a low-quality data set, a time mask and a spatial mask; the low-quality data set is obtained by performing a spatial mask MaskS and a temporal mask MaskST on the high-quality data set; the recognition model is used to identify dangerous dynamic targets in low-quality data; the recognition model is trained using the high-quality data set and the low-quality data set to obtain a trained recognition model.

[0007] Furthermore, the low-quality data refers to the feature loss rate MismatchT of each frame in the video. k All meet the 3S data quality standards, the specific formula is as follows;

[0008] VMT+3*DVMT>MismatchT k >VMT-3*DVMT

[0009] Among them, VMT represents the average feature loss rate of the entire video, and DVMT represents the variance of the feature loss rate of the entire video.

[0010] Furthermore, the process of obtaining high-quality videos is as follows:

[0011] Step 1: Collect video data to form a dataset Dataset1, where the video includes dangerous dynamic targets;

[0012] Step 2: Translate and cut the video data in the dataset Dataset1 into small videos with each segment including Tmax frame images; identify and segment the dangerous dynamic targets in the small video images, calculate the ORB features of the dangerous dynamic targets in each small video, and calculate the ORB feature matching ratio value Tmatch between the current frame k and the previous frame image of the dangerous dynamic targets k , and further obtain the feature loss rate MismatchT between adjacent frames k ;

[0013] The feature loss rate MismatchT of the k-th frame k = 1 - the feature matching ratio value Tmatch k ;

[0014] Step 3: Calculate the average feature loss rate VMT and the variance DVMT of the feature loss rate of each small video respectively;

[0015]

[0016] Step 4: In each small video, count the number of frames F k that meet the following conditions for the feature loss rate MismatchT h ;

[0017] MismatchT k < VMT - 3 * DVMT, when

[0018] When the ratio of the number of frames F h to the total number of frames of this small video is greater than MisF, extract this segment of video and add it to the high-quality data set DatasetHigh.

[0019] Furthermore, the process of obtaining low-quality videos is as follows:

[0020] Step 1: Each segment of video in the high-quality data set DatasetHigh has Tmax frame images; add a random mask to the Tmax frame images to obtain the data set Dataset1 after spatial masking, and the mask in the spatial dimension used is MaskS;

[0021] Step 2: For the images in the dataset Dataset1, randomly lose them in the time dimension T according to the set ratio to obtain a low-quality dataset DatasetLow after time loss, and obtain a mask MaskST of the time dimension.

[0022] Furthermore, in step 1, a random mask is added to the Tmax frame image to obtain a spatially masked dataset Dataset1, the specific contents of which are as follows:

[0023] Yolo-fastest is used to extract image features, specifically the 17th low-level feature layer, and converted into a feature heat map Where FW is the width of the feature heat map, FH is the height of the feature heat map, Ti is the time dimension, and the value range of Ti is from 1 to Tmax; in the spatial dimension, Gaussian noise is added to the spatial part of FW×FH so that the image after adding noise is MismatchT with the previous frame. k The videos that meet the 3S data quality standard are added to the low-quality dataset Dataset2, and the mask MaskS of the spatial dimension is also obtained.

[0024] Furthermore, the recognition model includes an encoder and a decoder; Yolo-fastest is used to extract features from the low-quality dataset DatasetLow, which is input into the encoder to capture the relationship between consecutive frames and output a continuous encoding vector with a time length of Tmax.

[0025] Among them, FL represents the feature length extracted by Yolo-fastest;

[0026] Concat the continuous encoding vector Xo and its corresponding mask MaskS by position to get Xin; construct the codebook Dic=[d1,d2,…,d cmi ,…d CM ];

[0027] Among them, FLT is the vector length of each data, FLT = FL × Tmax; CM represents the size of the codebook Dic; remove the part of the encoding vector feature Xin that is equal to 1, compare the distance between the encoding vector feature Xin and the class in the codebook, and find the mapping between the continuous vector Xin and the target class cmj on the codebook Dic;

[0028] cmj=Argmin||Xin-d cmj ||

[0029] Among them, cmj is the index of the codebook vector closest to Xin, including dangerous dynamic targets, safety classes, and other classes;

[0030] Quantify the current feature Xin to the feature X q Set to d cmj ;

[0031] X q =d cmj

[0032] The quantized feature X q Input to decoder, output reconstructed video

[0033] Furthermore, the recognition model reconstruction loss is defined as:

[0034]

[0035] Where w1 and w2 are weights;

[0036] It refers to the proportion of all Xreal data that are not correctly assigned to cmj;

[0037] is the meaning of not being correctly classified; ∑ is the sum, is the difference between the restored video and the real image Vhi, ‖ ‖ 1 Indicates that the 1-norm distance calculation method is used.

[0038] Furthermore, the dangerous dynamic targets include people, cars, animals, electric vehicles, bicycles, floating obstacles in the air, and ground obstacles.

[0039] Furthermore, the video data collection environment includes urban expressways, urban low-speed roads, and roads within residential areas; the video data collection environment is represented by m, where m=1, 2, and 3, respectively representing urban expressways, urban low-speed roads, and roads within residential areas;

[0040] The recognition models are trained using high-quality data and low-quality data in different environments, and recognition models suitable for different environments are obtained, which are recorded as Model1, Model2, and Model3 respectively.

[0041] The method for identifying dangerous dynamic targets by the auxiliary driving system of the present invention comprises the following steps:

[0042] Step 1, identify the current environment, call the corresponding model Model1, Model2 or Model3, if the recognition result is not in the acquisition environment m, call Model3;

[0043] Step 2: Use Yolo-fastest to identify video frames and mark dangerous dynamic targets;

[0044] Step 3, input the video into the recognition model, the video has a total of Tmax frames;

[0045] If the same dangerous dynamic target appears less than or equal to T1 times, and the image center distance between each frame in the video and the same dangerous target in the previous frame is less than T2% of the image width and height pixels, go to step 4;

[0046] If the same dangerous dynamic target appears more than T1 times, and the distance difference is less than or equal to T2% of the image width and height pixels, then the target is directly set as a dangerous dynamic target and marked;

[0047] Otherwise, no processing will be performed; in the present invention, T1 is set to 10 and T2 is set to 20;

[0048] Step 4: Compare with the codebook Dic to obtain its predicted value cmj and mark the dangerous dynamic target.

[0049] Beneficial effects: The present invention proposes an intelligent helmet-assisted driving system for identifying dangerous targets. The system is based on the camera data on the intelligent driving helmet and can identify dangerous targets under relatively low-speed conditions in cities, where the motorcyclist's body shakes and reduces the video quality. It makes full use of the original Yolo-fastest to detect dangerous targets in single-frame images, and further analyzes dangerous dynamic targets for unstable video data. The advantages are: first, it improves the accuracy of target recognition on unstable single-frame static images; second, it can combine video information to predict dangerous actions of targets; third, the overall computational complexity is not large, and it can be used on a small computer with a GPU.

[0050] The present invention adopts the scheme of encoder and decoder and proposes a method of vector quantization in latent space to improve the feature representation ability and the generalization ability of the encoder, thereby solving the problem of dangerous dynamic target recognition under the premise of unstable input data. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0052] Figure 1 is a flowchart of establishing a data set Dataset according to a specific embodiment of the present invention;

[0053] Figure 2 is a schematic diagram of the structure of a recognition model according to a specific embodiment of the present invention;

[0054] Figure 3 It is a flow chart of the model application of a specific embodiment of the present invention. DETAILED DESCRIPTION

[0055] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0056] The present invention provides an assisted driving system for identifying dangerous dynamic targets. Specifically, it is an algorithm for identifying dangerous dynamic targets in videos based on camera data on an intelligent driving helmet under relatively low-speed conditions in cities and under conditions where the motorcycle driver's body shaking reduces the video quality.

[0057] In the present invention, the discrete space of the video (i.e., multiple frames of images) is used to replace the original continuous space of the video, that is, the continuous representation space is introduced into the discrete representation space to improve the model's modeling ability for data. In the quantization process, each codeword represents the characteristic data of a class of dangerous targets in the discrete latent space codebook. Through encoding and decoding, the model can learn the feature representation of the input data and also learn the potential relationship between these features. By automatically generating more diverse new samples that conform to the distribution of the input data, sufficient training can be carried out when the actual sample size is not large. In addition, this discrete representation method can not only help the model learn more general feature representations, but also improve the model's generalization ability for low-quality data. This discrete feature representation has stronger robustness and better interpretability, and can reduce the impact of noise, that is, the impact of data instability, to a certain extent.

[0058] An assisted driving system for identifying dangerous dynamic targets of the present invention includes high- and low-quality data set generation modules, model training, and model application.

[0059] 1. High and low quality dataset generation module

[0060] The function of the high-quality and low-quality dataset generation module is to calculate the time and space loss ratio of the features of various dynamic targets in different environments, and to generate low-quality datasets uniformly based on the high-quality data according to the feature loss ratio. In the model training step, the model is trained by using the time and space correspondence between the high-quality and low-quality datasets and the known artificially added degradation information, thereby improving the adaptability of the model. Its structure is as follows: Figure 1 shown.

[0061] 1.1, Collect data set materials.

[0062] The environment is divided into three categories: urban expressways, urban low-speed roads, and roads within residential areas;

[0063] The invented assisted driving system is set in the smart driving helmet. Wearing the helmet, driving a motorcycle in the above three types of environments, collect preliminary real videos of actual operation. Define dangerous dynamic targets (including 7 categories such as people, cars, animals, electric vehicles, bicycles, floating obstacles in the air, and ground obstacles), and use a Yolo-fastest algorithm to automatically detect and segment dangerous dynamic targets in a single frame image. After collecting 24 hours of video of each of the 7 types of targets in the 3 types of environments, set it as Dataset1.

[0064] 1.2. Use a Yolo-fastest and videos of the above different types of environments to train a model MS that can recognize three types of environments: urban expressways, urban low-speed roads, and residential roads.

[0065] 1.3, in different categories of environments, a Yolo-fastest algorithm is used to automatically detect and segment dynamic targets in video images of each type of environment. Then, artificial semi-supervised learning is performed to remove the labels of parts that are not dangerous dynamic targets. For each picture of the above video part, refer to https: / / blog.csdn.net / lwx309025167 / article / details / 80365075, calculate the ORB (Oriented FAST and Rotated BRIEF) feature, and match the current frame with the previous frame for ORB features to obtain the ORB matching ratio value Tmatch of each previous and next frame, and then subtract Tmatch from 1 to obtain the feature loss rate MismatchT of the previous and next frames in 3 different categories of environments.

[0066] 1.4. For videos in three types of environments, namely urban expressways, urban low-speed roads, and roads within residential communities, steps 1.2 to 1.3 are used to obtain the feature loss rate of the previous and next frames of each frame. Then, the average loss rates VMT1 (urban expressways), VMT2 (urban low-speed roads), and VMT3 (roads within residential communities) of the three types of environments and their corresponding variances DVMT1 (urban expressways), DVMT2 (urban low-speed roads), and DVMT3 (roads within residential communities) are calculated.

[0067] 1.5, then, in order to improve greater adaptability, for the videos in the three types of environments, all videos are translated and cut into small segments of video each including Tmax frame images, and the feature loss rate MismatchT of the previous and next frames in each video is analyzed, for the parts that are lower than the respective 3S standards;

[0068] For example, in the video on the urban expressway, for the part where MismatchT1 < VMT1 - 3 * DVMT1, calculate the proportion MisF1 (urban expressway), MisF2 (urban low-speed road), and MisF3 (intra-village road) of the total number of frames in this type of video it occupies. For these short videos that are below their respective 3S standards, they constitute the so-called original high-quality data in the present invention. MismatchT1 represents the feature loss rate between consecutive frames in the urban expressway environment, MismatchT2 represents the feature loss rate between consecutive frames in the urban low-speed road environment, and MismatchT3 represents the feature loss rate between consecutive frames in the intra-village road environment.

[0069] 1.6, and then adopt the method of artificially adding random masks to increase its feature loss rate MismatchT.

[0070] Specifically, given a set of so-called original high-quality data (continuous video) Vid1, including Tmax frame images, use Yolo-fastest to extract features, specifically the low-level feature layer of the 17th layer. After converting to a feature heat map, the result is a set of feature heat maps for every Tmax frame pictures. Where FW is the width of the feature heat map, FH is the height of the feature heat map, and Ti is the time dimension. Ti ranges from 1 to Tmax, that is, the longest number of frames in this video.

[0071] 1.7, then, in the spatial dimension, add Gaussian noise to the FW×FH spatial part. The low-quality data (continuous video) restored from this frame feature heat map (added with noise) is called Vid2. The feature loss rates before and after adding Gaussian noise to Vid2 in the present invention meet the 3S data quality standard, that is:

[0072] VMT + 3 * DVMT > MismatchT > VMT - 3 * DVMT

[0073] Obtain Dataset2. The feature loss rates in all the data in Dataset2 reach the standard lower limit. At the same time, obtain a mask MaskS in this spatial dimension. In this mask, the positions of the added parts are set to 1, and the positions of the parts retaining the original information are set to 0.

[0074] 1.8, then, for the newly generated datasets that all meet the 3S data quality standard requirements, for the part of the time dimension T, randomly lose according to the proportions of MisF1 (urban expressway), MisF2 (urban low-speed road), and MisF3 (intra-village road) to achieve the effect of time loss. Obtain DatasetLow. Both the time and space in all the data in DatasetLow reach the standard lower limit.

[0075] In this way, we obtain the standard DatasetLow of unified low-quality data sets, thereby improving the adaptability of the model trained by it. DatasetLow still includes three types of data: urban expressways, urban low-speed roads, and roads within residential areas.

[0076] At the same time, a mask MaskST of this time-space dimension is obtained. In this mask, the spatial part is MaskS in step 1.7, and the time part is set to 1 for the position of the lost part and set to 0 for the position that retains the original information.

[0077] 1.9, then, in the Dataset1 dataset, find the part that meets the 3S data quality standard (the set of Vid1 in step 1.6), and directly obtain an original high-quality standard dataset DatasetHigh.

[0078] DatasetLow, DatasetHigh and mask constitute the data set Dataset that needs to be learned by the entire present invention. They all have a unified timestamp. Each low-quality image frame of DatasetLow can find the corresponding high-quality image frame on DatasetHigh. However, compared with DatasetHigh, DatasetLow has lost some frames due to step 1.8. The lost frames can be viewed through MaskST in step 1.8.

[0079] 2. Model training

[0080] The model structure is as follows Figure 2 As shown, it mainly includes an asymmetric encoder and decoder, and its principle is:

[0081] Using the two datasets, DatasetLow and DatasetHigh, we take advantage of the redundant information in time from the low-quality dataset so that the results of the model reconstruction learned from the DatasetLow dataset can be close to the DatasetHigh dataset.

[0082] In the learning phase, the encoder extracts the feature representation in DatasetLow, and then the decoder reconstructs the data based on the variables obtained during the encoding process. Then, overall, the error between the feature data of DatasetHigh and the reconstructed feature data obtained from DatasetLow is minimized, so that the learning ability of the model's feature representation is continuously improved.

[0083] 2.1 Encoder

[0084] Use the standard DatasetLow of low-quality datasets as the input of the encoder. Figure 2As shown, the encoder also uses Transformer to capture the relationship between the features extracted by Yolo-fastest in consecutive frames (specifically the 17th low-level feature layer), so that it can encode and represent the standard DatasetLow of the low-quality dataset. The input of the encoder is a set of videos Vli from DatasetLow with a length of Tmax frames (which has a corresponding part Vhi of the high-quality dataset in step 1.9, and the corresponding mask is MaskSTli). The output of the encoder is a set of features encoded vectors with a continuous time length of Tmax. FL is the feature length extracted by Yolo-fastest (specifically, the data length of the 17th low-level feature layer), and the part of the continuous coding vector that is 1 is set to 1.001. Because 1 in MaskSTli represents a modified value, before performing Concat on the coded value and MaskSTli, modify it to prevent the next step from mistaking it for a modified value.

[0085] Then, the feature Xo and the corresponding mask MaskSTli are concat-combined according to the position to obtain Xin, where the 1 part is the modified value and the 0 part is the original value.

[0086] 2.2, Feature Discrete

[0087] In order to improve the feature reconstruction feature data in low-quality videos and directly identify the type of danger, thereby reducing the amount of calculation, further reducing the calculation requirements for smart helmets and increasing their applicability, the present invention uses the idea of ​​discrete feature space to find the code of the dangerous dynamic target code book closest to the original code vector to determine what type of danger it is.

[0088] The advantage of this approach is that it can be directly identified during the reconstruction process, reducing the amount of calculation; the disadvantage is that when the data set is small and the dangerous target code book does not contain much content (i.e., there are not many classifications of dangerous targets), non-dangerous targets may be mistakenly identified as dangerous targets, but this is acceptable in the early stages of assisted driving. The principle is:

[0089] The present invention obtains a continuous set of encoding vector features Xin from the output of the encoder in the standard DatasetLow of the original low-quality data set according to a set of encoding books, and maps it to a discrete vector d cmj, where cmj means danger type. In this way, by using vector quantization technology, the continuous feature space is mapped to the discrete potential space, which helps to learn more meaningful feature representations and improve the model's ability to represent data. In this way, the encoder can extract key feature information from the input video data for valuable mapping in the discrete space, reducing the amount of calculation. In addition, through discrete quantization, redundant information is also removed, the same type of data will be more compact, and the anti-interference ability will be enhanced.

[0090] Specifically, for step 2.1, a set of encoded vector features The recognition model of the present invention learns a codebook Dic=[d1,d2,…,d cmi ,…d CM ], where FLT is the vector length of each data, which is composed of FLT = FL × Tmax, cmi is 1 ~ CM, CM is the size of this codebook, that is, how many classes there are, and the encoding vector feature Xin is compared with these classes in the feature space. The present invention adopts the nearest neighbor search algorithm to find the mapping between the continuous vector Xin and the target class cmj on the codebook Dic. When calculating, it is necessary to remove the part of Xin that is equal to 1, that is, the part that is modified in the mask in step 2.1.

[0091] cmj=Argmin||Xin-d cmj ||

[0092] Among them, cmj is the index of the codebook vector closest to Xin, which includes 7 categories, including people, cars, animals, electric vehicles, bicycles, floating obstacles in the air, ground obstacles, as well as safety and other categories, a total of 9 categories. Then the quantized feature X of the current Xin feature is q Set to d cmj .

[0093] X q =d cmj

[0094] Here, the actual class of Xin's manual labeling is Xreal, and cmj is the predicted class of the result of this classification. cmj may not be consistent with Xreal, that is, it may be inaccurate.

[0095] Here, the codebook Dic can be obtained by referring to the method in the paper, Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova, "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding" http: / / arxiv.org / pdf / 1810.04805.

[0096] Its input data is the high and low quality data sets in the present invention, and its output is the codebook Dic.

[0097] 2.3 Decoder

[0098] During the training process, the present invention requires a decoder. After training, in practical applications, the decoder is optional.

[0099] In the present invention, the decoder consists of a Transformer module and an output layer consisting of a fully connected layer. The input of the decoder is the quantized feature X q , the output is a video with a frame length of Tmax It corresponds to Vli in the low-quality dataset Datasetlow of the encoder input in step 2.1. In step 1.9, the data corresponding to Vli in the high-quality dataset DatasetHigh is Vhi.

[0100] The overall task is to maintain the reconstructed dataset sequence It is consistent with the corresponding Vhi in the original high-quality standard dataset DatasetHigh.

[0101] 2.4 Pre-training

[0102] The definition of the reconstruction loss of the recognition model of the present invention is:

[0103]

[0104] Where w1 and w2 are weights, It refers to the proportion of all Xreal data that is not correctly distributed to cmj. It means that there is no correct classification, ∑ is the sum, which is an expression about whether the judgment is correct. is the difference between the restored video and the real image, indicating the specific overall distortion effect, ‖‖ 1 Indicates that the 1-norm distance calculation method is used.

[0105] Then, the model with the smallest verification loss is saved as the best model. In the present invention, three models Model1, Model2, and Model3 are trained in three types of environments, namely, urban expressways, urban low-speed roads, and roads in residential areas in step 1.1.

[0106] For the specific pre-training process, please refer to the paper Y.Chen, L.Zhao, J.Yuan, Y.Tian, ​​Z.Xia, S.Geng, L.Han, DNMetaxas, "Hierarchically self-supervised transformer for humanskeleton representation learning", European Conference on Computer Vision, Springer, 2022, pp.185-202.

[0107] 3. Identification model application

[0108] like Figure 3 As shown, the present invention uses the innovative part of the present invention on the basis of making full use of Yolo-fastest.

[0109] 3.1. Use the MS model in step 1.1 to identify the current environment and call out the corresponding models Model1, Model2, and Model3. If the recognition result is not in the third environment, call Model3.

[0110] 3.2, Yolo-fastest is used to identify video frames and mark possible dangerous dynamic targets.

[0111] 3.3, set Tmax in step 2.1 to 15, that is, 15 frames. Then, in these 15 frames, if the same dangerous dynamic target appears 1 to 10 times, and the maximum difference in image center distance from the same dangerous target in the previous frame is only 20% of the image width and height pixels, then it is determined to be content that needs dynamic recognition, and go to step 3.5.

[0112] 3.4, if the same dangerous dynamic target appears more than 10 times within 15 frames, and the distance difference is less than or equal to 20% of the image width and height pixels, then the target is directly set as a dangerous dynamic target and go to step 3.6. Other cases are likely to be too high jitter and will not be processed.

[0113] 3.5, using step 2.2, extract and classify the target features, obtain its predicted value cmj, and proceed to step 3.6.

[0114] 3.6. Use the smart helmet to mark dangerous targets and display them directly to the user, then return to step 3.1 and repeat the work.

[0115] In this way, by implementing the present invention, users can obtain camera data for smart driving helmets, and can identify dangerous targets under relatively low-speed conditions in cities, and under difficult conditions where the motorcyclist's body shakes and reduces the video quality. It makes full use of the original Yolo-fastest to detect dangerous targets in single-frame images, and further analyzes dangerous dynamic targets for unstable video data. Its advantages are: first, it improves the accuracy of target recognition on unstable single-frame static images; second, it can combine video information to predict dangerous actions of targets; third, the overall computational complexity is not large, and it can be used on a small computer with a GPU.

Claims

1. An auxiliary driving system for identifying dangerous dynamic targets, characterized in that: Includes high and low quality dataset generation modules and recognition models; The high- and low-quality data set generation module is used to generate a high-quality data set, a low-quality data set, a time mask, and a space mask; The low-quality data set is obtained by performing spatial masking MaskS and temporal masking MaskST on the high-quality data set; The recognition model is used to identify dangerous dynamic targets in low-quality data; The recognition model is trained using high-quality data sets and low-quality data sets to obtain a trained recognition model.

2. The driver assistance system for identifying dangerous dynamic targets according to claim 1, characterized in that: The low-quality data refers to the feature loss rate MismatchT of each frame in the video. k All meet the 3S data quality standards, the specific formula is as follows; VMT+3*DVMT>MismatchT k >VMT-3*DVMT Among them, VMT represents the average feature loss rate of the entire video, and DVMT represents the variance of the feature loss rate of the entire video.

3. The driver assistance system for identifying dangerous dynamic targets according to claim 1, characterized in that: The process of obtaining high-quality video is as follows: Step 1, collecting video data to form a data set Dataset1, wherein the video includes a dangerous dynamic target; Step 2: Translate and cut the video data in the dataset Dataset1 into continuous small segments, each of which includes Tmax frame images; identify and segment the dangerous dynamic targets in the small video images, calculate the ORB features of the dangerous dynamic targets in each small video, and calculate the ORB feature matching ratio value Tmatch between the current frame k and the previous frame image of the dangerous dynamic targets k , and further obtain the feature loss rate of adjacent frames MismatchT k ; The feature loss rate of the kth frame MismatchT k =1-feature matching ratio value Tmatch k ; Step 3, calculate the average feature loss rate VMT and the variance DVMT of the feature loss rate for each short video segment respectively; Step 4: In each short video segment, calculate the feature loss rate MismatchT k The number of frames F that meet the following conditions h ; MismatchT k <When VMT - 3 * DVMT, When the frame number F h When the ratio of the total number of frames of the short video segment is greater than MisF, the video segment is extracted and added to the high-quality data set DatasetHigh.

4. The driver assistance system for identifying dangerous dynamic targets according to claim 2, characterized in that: The process of obtaining low-quality video is as follows: Step 1, each video in the high-quality data set DatasetHigh has Tmax frame images; add a random mask to the Tmax frame images to obtain a spatially masked data set Dataset1, and the mask of the spatial dimension used is MaskS; Step 2: For the images in the dataset Dataset1, randomly lose them in the time dimension T according to the set ratio to obtain a low-quality dataset DatasetLow after time loss, and obtain a mask MaskST of the time dimension.

5. The driver assistance system for identifying dangerous dynamic targets according to claim 4, characterized in that: In step 1, a random mask is added to the Tmax frame image to obtain a spatially masked dataset Dataset1, the specific contents of which are as follows: Yolo-fastest is used to extract image features, specifically the 17th low-level feature layer, and converted into a feature heat map Where FW is the width of the feature heat map, FH is the height of the feature heat map, Ti is the time dimension, and the value range of Ti is from 1 to Tmax; in the spatial dimension, Gaussian noise is added to the spatial part of FW×FH so that the image after adding noise is MismatchT with the previous frame. k The videos that meet the 3S data quality standard are added to the low-quality dataset Dataset2, and the mask MaskS of the spatial dimension is also obtained.

6. The driver assistance system for identifying dangerous dynamic targets according to claim 4, characterized in that: The recognition model includes an encoder and a decoder; Use Yolo-fastest to extract features from the low-quality dataset DatasetLow, input it into the encoder, capture the relationship between consecutive frames, and output a continuous encoding vector with a time length of Tmax Among them, FL represents the feature length extracted by Yolo-fastest; Concat the continuous encoding vector Xo and its corresponding mask MaskS by position to get Xin; construct the codebook Dic=[d1,d2,…,d cmi ,…d CM ]; Among them, FLT is the vector length of each data, FLT = FL × Tmax; CM represents the size of the codebook Dic; remove the part of the encoding vector feature Xin that is equal to 1, compare the distance between the encoding vector feature Xin and the class in the codebook, and find the mapping between the continuous vector Xin and the target class cmj on the codebook Dic; <h2 style=";text-align:left;direction:ltr">cmj = Argmin || Xin-d<h2 style=";text-align:left;direction:ltr"> cmj <h2 style=";text-align:left;direction:ltr"> || Among them, cmj is the index of the codebook vector closest to Xin, including dangerous dynamic targets, safety classes, and other classes; Quantify the current feature Xin to the feature X q Set to d cmj ; X q =d cmj The quantized feature X q Input to decoder, output reconstructed video 7. The driver assistance system for identifying dangerous dynamic targets according to claim 6, characterized in that: The recognition model reconstruction loss is defined as: Where w1 and w2 are weights; It refers to the proportion of all Xreal data that are not correctly assigned to cmj; is the meaning of not being correctly classified; ∑ is the sum, is the difference between the restored video and the real image Vhi, ‖‖ 1 Indicates that the 1-norm distance calculation method is used.

8. The driver assistance system for identifying dangerous dynamic targets according to claim 1, characterized in that: The dangerous dynamic targets include people, cars, animals, electric vehicles, bicycles, floating obstacles in the air, and ground obstacles.

9. The driver assistance system for identifying dangerous dynamic targets according to claim 6, characterized in that: The video data collection environment includes urban expressways, urban low-speed roads, and roads within residential areas. The video data collection environment is represented by m, where m=1, 2, and 3, respectively representing urban expressways, urban low-speed roads, and roads within residential areas. The recognition models are trained using high-quality data and low-quality data in different environments, and recognition models suitable for different environments are obtained, which are recorded as Model1, Model2, and Model3 respectively.

10. The method for identifying dangerous dynamic targets by an auxiliary driving system as claimed in claim 6, characterized in that: The steps include: Step 1, identify the current environment, call the corresponding model Model1, Model2 or Model3, if the recognition result is not in the acquisition environment m, call Model3; Step 2: Use Yolo-fastest to identify video frames and mark dangerous dynamic targets; Step 3, input the video into the recognition model, the video has a total of Tmax frames; If the same dangerous dynamic target appears less than or equal to T1 times, and the image center distance between each frame in the video and the same dangerous target in the previous frame is less than T2% of the image width and height pixels, go to step 4; If the same dangerous dynamic target appears more than T1 times, and the distance difference is less than or equal to T2% of the image width and height pixels, then the target is directly set as a dangerous dynamic target and marked; Otherwise, no processing will be performed; in the present invention, T1 is set to 10 and T2 is set to 20; Step 4: Compare with the codebook Dic to obtain its predicted value cmj and mark the dangerous dynamic target.

Images