Multi-stage alarm method and system for vehicle

Abstract

The invention provides a multi-level alarm method and system for a vehicle. The method comprises the following steps: obtaining a video stream of a monitoring area; preprocessing the picture; amplifying or reducing the picture; a target detection module is used for analysis; tracking, filtering, screening and classifying object position coordinates and feature information through a tracking and classifying module; according to the invention, the system can automatically identify and classify images, improve the accuracy and robustness of the system, detect a plurality of targets at the same time, improve the accuracy and robustness of the system, improve the accuracy and robustness of the system, and improve the accuracy and robustness of the system, thereby improving the accuracy and robustness of the system, and improving the accuracy and robustness of the system. And secondly, risk assessment is carried out according to the distance, the speed and the moving track of the obstacle, and multi-level alarm prompt is given. The detected target is continuously tracked, dynamic environment information and risk assessment are provided, and the real-time performance and accuracy of the alarm system are improved.

Description

Technical Field

[0001] The present invention relates to the field of vehicle alarms, and in particular to a multi-level alarm method and system for a vehicle. Background Art

[0002] Existing technologies mainly rely on millimeter wave sensors to detect vehicles, pedestrians and obstacles on the side and rear of the vehicle. It consists of the following modules: 1. Millimeter wave radar sensor, which is the sensing component of the system, responsible for transmitting and receiving millimeter wave signals. When the millimeter wave encounters an obstacle, it will be reflected. The radar analyzes the reflected signal through the Kepler effect to determine the surrounding environment, including the distance, speed and direction of the obstacle. 2. Signal processing unit, the received millimeter wave signal needs to be processed before it can be converted into useful information. This unit is responsible for filtering, amplifying, digitizing and other processing of the original signal to extract key parameters such as the position and speed of the target. 3. Control unit, based on the information provided by the signal processing unit, the control unit determines whether there is a potential collision risk and decides whether to warn the driver. 4. Human-computer interaction module, when the system determines that there is danger in the blind spot, it will remind the user through visual, auditory or tactile means, and attract the driver's attention by flashing icons on the dashboard and sounding alarms.

[0003] However, the existing technology has obvious defects, mainly including the following points: 1. The technology of millimeter wave radar is complex and the manufacturing cost is relatively high, especially in the high-frequency 77GHz radar system. Although the resolution of millimeter wave radar is high, it still has certain limitations in close-range and small target detection. For example, small targets such as pedestrians and bicycles cannot be well detected. 2. Millimeter wave radar may produce false alarms or missed alarms in some cases. For example, stationary metal objects may be mistaken for moving obstacles, and some non-metallic objects cannot be well reflected by radar waves and thus effectively detected. 3. Millimeter wave radar may be affected by multipath effects in complex environments, resulting in inaccurate detection results. Multipath effects refer to radar signals reaching the receiver after multiple reflections, causing signal interference and distortion. 4. The complexity and high cost of the system. The amount of data generated by millimeter wave radar is large, requiring high-performance processors for real-time processing and analysis. At the same time, the field of view of millimeter wave radar is relatively narrow and cannot cover the entire area around the vehicle. In order to achieve all-round and multi-angle detection, multiple millimeter wave radars are usually required to work together. 5. Unable to classify targets. Any object detected within the monitoring range of the millimeter wave is regarded as an obstacle and an alarm is issued. It is impossible to achieve classified and partitioned detection and alarm. Summary of the invention

[0004] The purpose of the present invention is to achieve low cost, high reliability and low complexity of vehicle side multi-level warning.

[0005] In order to achieve the above object, the present invention adopts the following technical scheme:

[0006] A multi-level alarm method for a vehicle comprises the following steps:

[0007] S1: Obtain the real-time video stream of the monitored area and store the real-time video stream data in the storage module;

[0008] S2: The acquired real-time video stream data is preprocessed by a preprocessing module, wherein the preprocessing includes but is not limited to stitching of multiple camera images, adjusting brightness and saturation, correcting lens distortion, noise reduction and cropping. The preprocessing module performs geometric correction, distortion elimination, contrast enhancement, brightness adjustment, and noise removal on the image, mainly to improve the image quality and visual effect of the image;

[0009] S3: binarizing, enlarging or reducing the image preprocessed in step S2 through a processing module, and adjusting the image size to adapt to different application scenarios or algorithm input requirements;

[0010] S4: Analyze the image processed in step S3 using the target detection module, identify predefined objects, and calculate the position coordinates of these objects in the image;

[0011] S5: adjusting the object position coordinates according to the screen zoom ratio in step S3, drawing a rectangular wireframe in the screen to identify the identified object, and then tracking, filtering, screening and classifying the object position coordinates and feature information through the tracking and classification module;

[0012] S6: Calculate the distance between the target and the vehicle based on the target position coordinates and size obtained in step S5, and combine the speed of the vehicle and the motion state of the target to obtain the target risk assessment index F2, and give an alarm prompt according to the level, including but not limited to sound prompts and visual warnings;

[0013] S7: The post-processing module respectively displays the image pre-processed in step S2 and the image processed in step S5 through the display module for real-time viewing by the user.

[0014] Preferably, the target detection module is embedded with a target detection algorithm ( YOLO ), the target detection algorithm ( YOLO ) performs real-time detection on each frame of image, and the specific steps are as follows:

[0015] S401: Input image, divide the image into SxS grids according to the image data with required resolution corresponding to different inputs of the model to form a feature map (X);

[0016] S402: Bounding box prediction: for each input grid, predict B bounding boxes and the confidence of the target of each bounding box and the probability of each bounding box area in multiple categories, including bounding box information (x, y, h, w) and confidence values;

[0017] S403: Category prediction: predict a category information for each bounding box, recorded as category C;

[0018] S404: Calculate a comprehensive score, perform a comprehensive score calculation to remove target windows with low probability;

[0019] S405: Non-maximum suppression, performing maximum suppression (NMS) to remove redundant windows;

[0020] S406: Output the result, output a tensor tensor containing information SxSx(5*B+C).

[0021] Preferably, the target detection algorithm ( YOLO ) In the backbone part of the model, the attention mechanism CSPNet is introduced. The CSPNet divides the feature map (X) into two parts, and divides (X) into (X1) and (X2), where the number of channels of (X1) is αC, and the number of channels of (X2) is (1-α)C. α is a hyperparameter, usually set to 0.5; (X2) is processed through a series of convolutional layers, convolution, batch normalization, and activation functions to generate a feature map (F(X2)); the processed feature map (F(X2)) is spliced ​​with the directly transmitted feature map (X1) to generate the final output feature map (Y). The specific calculation formula is as follows:

[0022] X1∈R H×W×64

[0023] X2∈R H×W×64

[0024] F(X2)=ReLU(BatchNorm(Conv(X2)))

[0025] Y∈R H×W×64

[0026] Among them, the symbol R represents a set of real numbers, X1, X2, and Y are all three-dimensional tensors with a shape of H×W×64, all elements are real numbers, ReLU() is the activation function, Conv is the convolutional layer, and BatchNorm is the batch normalization layer.

[0027] Preferably, the specific steps of the tracking and classification module for tracking, filtering, screening and classifying the object coordinates and feature information are as follows:

[0028] S501: Divide the detection frames of the drawn rectangular wireframe into a high-resolution frame set and a low-resolution frame set, then create corresponding Tracks, and use Kalman filtering to predict the position of the track in the current frame;

[0029] S502: Perform IOU matching on the target detection frame of the frame and the rectangular frame predicted by Tracks in the previous frame one by one, and then calculate the cost matrix based on the result of IOU matching, and the calculation method is 1-IOU;

[0030] S503: using all the cost matrices obtained in step S502 as inputs of the Hungarian algorithm to obtain linear matching results;

[0031] S504: Loop step S502 and step S503 until the video frame ends.

[0032] Preferably, the IOU Match in step S502 uses DIoU-NMS instead of traditional NMS. DIoU-NMS not only considers the intersection over union (IoU) between the prediction boxes, but also considers the distance between the center points of the two boxes, so as to better handle targets with small overlapping areas but close distances, reduce false positive detections, and improve detection accuracy. The specific calculation formula is as follows:

[0033]

[0034] Where S i is the classification confidence, ε is the NMS threshold, M is the detection box with the highest confidence, and B i is the i-th detection box, ρ represents the Euclidean distance between the two center points, b represents the center point of the prediction box, and b gt represents the center point of the real box, c is the diagonal length of the smallest rectangular box containing the predicted box and the target box, and the IoU value range is between 0 and 1.

[0035] Preferably, the linear matching results obtained in step S503 include the following three types:

[0036] The first is Unmatched Tracks, which directly deletes the unmatched Tracks.

[0037] The second type is Unmatched Detections, which initializes such Detections as a new Track.

[0038] The third is that the detection box and the predicted box are paired successfully, indicating that the previous frame and the next frame are tracked successfully, and the corresponding Detections are updated through the Kalman filter to update the corresponding Tracks variables.

[0039] Preferably, the calculation formula of the risk assessment index F2 is as follows:

[0040]

[0041] F2=F1×Time

[0042] Among them, Precision is the accuracy, which is the proportion of correctly detected risk objects to all detected objects, Recall is the recall rate, which is the proportion of correctly detected risk objects to all actually existing risk objects, F1 is an indicator that comprehensively considers precision and recall rate, TP is the true positive example, that is, the number of samples correctly predicted by the model as positive. FP is the false positive example, that is, the number of samples incorrectly predicted by the model as positive, FN is the false negative example, that is, the number of samples incorrectly predicted by the model as negative, FN is the false negative example, that is, the number of samples incorrectly predicted by the model as negative, F2 is the time taken for the target to cross the dangerous area in the positive direction, and F2 is the risk assessment indicator.

[0043] Preferably, the present invention also provides a system of a vehicle-based multi-level alarm method, comprising an image acquisition module, a preprocessing module, a processing module, a target detection module, a tracking and classification module, a post-processing module, an alarm module, a display module and a storage module, wherein the image acquisition module acquires real-time video streams within the vehicle monitoring area, and performs multi-screen splicing, brightness saturation modification, distortion correction, noise reduction and cropping through the preprocessing module, and then the preprocessed screen is enlarged or reduced through the processing module, and each frame of the image is detected in real time using the built-in YOLO algorithm of the target detection module, and the coordinates of the target in the screen and the corresponding target category are output, the output results of the target detection module are tracked, filtered and screened through the tracking and classification module, and finally, according to the position and size of the target, the distance between it and the vehicle is calculated, and the potential danger level is evaluated in combination with the speed of the vehicle and the motion state of the target, and an alarm prompt is issued through the alarm module.

[0044] Preferably, the target detection module also includes an input module, a prediction module, a category prediction module, a calculation module and an output module. The input module divides the image data into parts according to the resolution requirements of different models and passes them to the prediction module. The prediction module is used to predict border information and confidence. The category prediction module predicts the border category. The calculation module is used to calculate the comprehensive score and non-maximum suppression. The output module outputs a tensor tensor.

[0045] The beneficial effects of the present invention are:

[0046] 1. Multiple cameras can be set up in the monitoring area, using the camera image as a sensor to provide a wide field of view, intuitive visual display and alarm content, helping the driver to clearly understand the situation around the vehicle, and achieving a 360-degree panoramic view to eliminate blind spots.

[0047] 2. The system can automatically recognize and classify images, improve the accuracy and robustness of the system, and can detect multiple targets at the same time, such as vehicles, pedestrians, bicycles, etc., improving the versatility of the system. Risk assessment is performed based on the distance, speed and movement trajectory of obstacles, and different levels of alarms are issued according to the assessment level. Secondly, through algorithms such as Kalman filters, the detected targets can be continuously tracked, dynamic environmental information and risk assessment can be provided, and the real-time and accuracy of the alarm system can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a schematic diagram of the workflow of the present invention;

[0049] Figure 2 It is a detailed schematic diagram of the process of S4 of the present invention;

[0050] Figure 3 It is a detailed schematic diagram of the process of S5 of the present invention;

[0051] Figure 4 It is a schematic diagram of the process of the system of the present invention;

[0052] Figure 5 It is a flow chart of the target detection module of the present invention;

[0053] Figure 6 It is a flow chart of the tracking and classification module of the present invention; DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0055] In the description of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside" and "outside" etc., indicating directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as a limitation on the present invention. In the description of the present invention, "multiple" means two or more, unless otherwise clearly and specifically defined.

[0056] Embodiment 1

[0057] See also Figure 1 , a multi-level alarm method for a vehicle, comprising the following steps:

[0058] S1: Get the real-time video stream of the monitored area and store the real-time video stream data in the storage module; according to the type of camera, it is divided into cameras with built-in image processing units and cameras without image processing units. The former can be AHD\CVBS\USB cameras, which need ADC module conversion and then connected to DVP interface or MIPI interface, while the latter is a sensor module that can be directly connected to DVP interface or MIPI interface. The image acquisition module is the above two DVP interfaces and MIPI interface.

[0059] S2: The acquired real-time video stream data is preprocessed through a preprocessing module, and the preprocessing includes but is not limited to stitching multiple camera images, adjusting brightness and saturation, correcting lens distortion, noise reduction and cropping; the preprocessing module is mainly used to improve image quality and improve the visual effect of the image.

[0060] S3: binarize, enlarge or reduce the image preprocessed in step S2 through a processing module, improve the recognition rate (mAP) and detection delay of the target detection module through image processing, and adjust the image size to adapt to different application scenarios or algorithm input requirements;

[0061] S4: Analyze the image processed in step S3 using the target detection module, identify predefined objects, and calculate the position coordinates of these objects in the image;

[0062] The target detection module uses the YOLO algorithm to detect each frame of the image in real time. The processing content includes: image preprocessing, feature extraction, target detection, and non-maximum suppression. The output detection result is the coordinates of the target in the picture and the corresponding target category, such as vehicles, pedestrians, and other obstacles.

[0063] S5: adjusting the object position coordinates according to the screen zoom ratio in step S3, drawing a rectangular wireframe in the screen to identify the identified object, and then tracking, filtering, screening and classifying the object position coordinates and feature information through the tracking and classification module;

[0064] S6: Calculate the distance between the target and the vehicle based on the target position coordinates and size obtained in step S5, and combine the speed of the vehicle and the motion state of the target to obtain the target risk assessment index F2, and give an alarm prompt according to the level, including but not limited to sound prompts and visual warnings;

[0065] S7: The post-processing module respectively displays the image pre-processed in step S2 and the image processed in step S5 through the display module for real-time viewing by the user.

[0066] Specifically, if Figure 2 As shown, the target detection module is implanted with a target detection algorithm ( YOLO ), the target detection algorithm ( YOLO ) performs real-time detection on each frame of image, and the specific steps are as follows:

[0067] S401: Input image, divide the image into SxS grids according to the image data with required resolution corresponding to different inputs of the model to form a feature map (X);

[0068] S402: Bounding box prediction: for each input grid, predict B bounding boxes and the confidence of the target of each bounding box and the probability of each bounding box area in multiple categories, including bounding box information (x, y, h, w) and confidence values;

[0069] S403: Category prediction: predict a category information for each bounding box, recorded as category C;

[0070] S404: Calculate a comprehensive score, perform a comprehensive score calculation to remove target windows with low probability;

[0071] S405: Non-maximum suppression, performing maximum suppression (NMS) to remove redundant windows;

[0072] S406: Output the result, output a tensor tensor containing information SxSx(5*B+C).

[0073] The target detection algorithm ( YOLO ) The attention mechanism CSPNet is introduced in the backbone part of the model. The attention mechanism CSPNet is consistent with the principle of the cross-region local network mechanism in the referenced document A NEW BACKBONE THAT CAN ENHANCE LEARNING CAPABILITY OF CNN. This design reduces computational redundancy and improves computational efficiency. By directly transferring some feature maps, CSPNet enhances feature reuse, enabling the network to better learn and retain useful feature information. The introduction of CSPNet helps balance the gradient flow, reduce the problem of gradient disappearance and explosion, and thus improve the training stability of the network.

[0074] The CSPNet divides the feature map (X) into two parts, and divides (X) into (X1) and (X2), where the number of channels of (X1) is αC, and the number of channels of (X2) is (1-α)C, α is a hyperparameter, usually set to 0.5; (X2) is processed through a series of convolutional layers, convolution, batch normalization, and activation functions to generate a feature map (F(X2)); the processed feature map (F(X2)) is spliced ​​with the directly transferred feature map (X1) to generate the final output feature map (Y), and the specific calculation formula is as follows:

[0075] X1∈R H×W×64

[0076] X2∈R H×W×64

[0077] F(X2)=ReLU(BatchNorm(Conv(X2)))

[0078] Y∈R H×W×64

[0079] Among them, the symbol R represents a set of real numbers, X1, X2, and Y are all three-dimensional tensors with a shape of H×W×64, all elements are real numbers, ReLU() is the activation function, Conv is the convolutional layer, and BatchNorm is the batch normalization layer.

[0080] Specifically, if Figure 3 As shown, the specific steps of the tracking and classification module for tracking, filtering, screening and classifying object coordinates and feature information are as follows:

[0081] S501: Divide the detection frames of the drawn rectangular wireframe into a high-resolution frame set and a low-resolution frame set, then create corresponding Tracks, and use Kalman filtering to predict the position of the track in the current frame;

[0082] S502: Perform IOU matching on the target detection frame of the frame and the rectangular frame predicted by Tracks in the previous frame one by one, and then calculate the cost matrix based on the result of IOU matching, and the calculation method is 1-IOU;

[0083] S503: using all the cost matrices obtained in step S502 as inputs of the Hungarian algorithm to obtain linear matching results;

[0084] S504: Loop step S502 and step S503 until the video frame ends.

[0085] The IOU Match in step S502 uses DIoU-NMS instead of traditional NMS. DIoU-NMS not only considers the intersection over union (IoU) between the predicted boxes, but also considers the distance between the center points of the two boxes, so as to better handle targets with small overlapping areas but close distances, reduce false positive detections, and improve detection accuracy. The specific calculation formula is as follows:

[0086]

[0087] Where S i is the classification confidence, ε is the NMS threshold, M is the detection box with the highest confidence, and B i is the i-th detection box, ρ represents the Euclidean distance between the two center points, b represents the center point of the prediction box, and b gt represents the center point of the real box, c is the diagonal length of the smallest rectangular box containing the predicted box and the target box, and the IoU value range is between 0 and 1.

[0088] When IoU is 1, it means that the two bounding boxes completely overlap; when IoU is 0, it means that the two bounding boxes do not overlap at all.

[0089] Specifically, if Figure 5 As shown, the linear matching results obtained in step S503 include the following three types:

[0090] The first is Unmatched Tracks, which directly deletes the unmatched Tracks.

[0091] The second type is Unmatched Detections, which initializes such Detections as a new Track.

[0092] The third is that the detection box and the predicted box are paired successfully, indicating that the previous frame and the next frame are tracked successfully, and the corresponding Detections are updated through the Kalman filter to update the corresponding Tracks variables.

[0093] The calculation formula of the risk assessment index F2 is as follows:

[0094]

[0095] F2=F1×Time

[0096] Among them, Precision is the accuracy, which is the proportion of correctly detected risk objects to all detected objects, Recall is the recall rate, which is the proportion of correctly detected risk objects to all actually existing risk objects, F1 is an indicator that comprehensively considers precision and recall rate, TP is the true positive example, that is, the number of samples correctly predicted by the model as positive. FP is the false positive example, that is, the number of samples incorrectly predicted by the model as positive, FN is the false negative example, that is, the number of samples incorrectly predicted by the model as negative, FN is the false negative example, that is, the number of samples incorrectly predicted by the model as negative, F2 is the time taken for the target to cross the dangerous area in the positive direction, and F2 is the risk assessment indicator.

[0097] The specific steps of the present invention are:

[0098] Assuming that 50 people are identified, the system provided by the present invention defines a rectangular danger zone Wa / Wb. The danger zones are close to each other. When the algorithm only considers the danger zone, 40 people are not in the danger zone, 10 people are in the danger zone, of which 3 people are defined as A\B\C moving towards the danger zone, and 7 people are uniformly defined as D in the danger zone but not close to it. Among them, it takes 3 seconds for A to approach from the Wa zone to the Wb zone, 2 seconds for B to approach from the Wa zone to the Wb zone, and 1 second for C to approach from the Wa zone to the Wb zone. Then TP is 40, FP is 7, FN is 3, then Precision is approximately equal to 0.85, approximately equal to 0.93. F1 is approximately equal to 0.88.

[0099] For A, F2 = 2.64, for B, F2 = 1.76. For C, F2 = 0.88, it is defined that less than or equal to F1 is a level 1 alarm, greater than F1 and less than or equal to twice F1 is a level 2 alarm, and greater than 2 times F1 is a level 3 alarm. In this way, it is divided into 3 levels of alarm, and according to the different alarm levels of each level, the system can give different alarm prompts.

[0100] Specifically, if Figure 4 As shown, the present invention also provides a system of a vehicle-based multi-level alarm method, including an image acquisition module, a preprocessing module, a processing module, a target detection module, a tracking and classification module, a post-processing module, an alarm module, a display module and a storage module. The image acquisition module will collect real-time video streams in the vehicle monitoring area, and perform multi-screen splicing, brightness saturation modification, distortion correction, noise reduction, and cropping through the preprocessing module, and then the preprocessed screen is enlarged or reduced through the processing module, and the YOLO algorithm built in the target detection module is used to detect each frame of the image in real time, and the coordinates of the target in the screen and the corresponding target category are output, and the output results of the target detection module are tracked, filtered and screened through the tracking and classification module, and finally, according to the position and size of the target, the distance between it and the vehicle is calculated, and the potential danger level is evaluated in combination with the speed of the vehicle and the motion state of the target, and an alarm prompt is issued through the alarm module.

[0101] Specifically, if Figure 5 As shown, the target detection module also includes an input module, a prediction module, a category prediction module, a calculation module and an output module. The input module divides the image data according to the resolution requirements of different models and passes it to the prediction module. The prediction module is used to predict the border information and confidence. The category prediction module predicts the border category. The calculation module is used to calculate the comprehensive score and non-maximum suppression. The output module will output a tensor tensor, which is the coordinate of the target in the picture and the corresponding target category, such as vehicles, pedestrians, and other obstacles.

[0102] The above are only preferred embodiments of the present invention and are not intended to limit the present invention in other forms. Any technician familiar with the profession may use the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A multi-level alarm method for a vehicle, characterized in that: The steps include: S1: Obtain the real-time video stream of the monitored area and store the real-time video stream data in the storage module; S2: preprocessing the acquired real-time video stream data through a preprocessing module, wherein the preprocessing includes but is not limited to stitching multiple camera images, adjusting brightness and saturation, correcting lens distortion, noise reduction and cropping; S3: binarizing, enlarging or reducing the image preprocessed in step S2 through a processing module, and adjusting the image size to adapt to different application scenarios or algorithm input requirements; S4: Analyze the image processed in step S3 using the target detection module, identify predefined objects, and calculate the position coordinates of these objects in the image; S5: adjusting the object position coordinates according to the screen zoom ratio in step S3, drawing a rectangular wireframe in the screen to identify the identified object, and then tracking, filtering, screening and classifying the object position coordinates and feature information through the tracking and classification module; S6: Calculate the distance between the target and the vehicle based on the target position coordinates and size obtained in step S5, and combine the speed of the vehicle and the motion state of the target to obtain the target risk assessment index F2, and give an alarm prompt according to the level, the alarm prompt includes but is not limited to sound prompts and visual warnings; S7: The post-processing module respectively displays the image pre-processed in step S2 and the image processed in step S5 through the display module for real-time viewing by the user.

2. A multi-level alarm method for a vehicle according to claim 1, characterized in that: The target detection module is embedded in the target detection algorithm ( YOLO ), the target detection algorithm ( YOLO ) performs real-time detection on each frame of image, and the specific steps are as follows: S401: Input image, divide the image into SxS grids according to the image data with required resolution corresponding to different inputs of the model to form a feature map (X); S402: Bounding box prediction: for each input grid, predict B bounding boxes and the confidence of the target of each bounding box and the probability of each bounding box area in multiple categories, including bounding box information (x, y, h, w) and confidence values; S403: Category prediction: predict a category information for each bounding box, recorded as category C; S404: Calculate a comprehensive score, perform a comprehensive score calculation to remove target windows with low probability; S405: Non-maximum suppression, performing maximum suppression (NMS) to remove redundant windows; S406: Output the result, output a tensor tensor containing information SxSx(5*B+C).

3. A multi-level alarm method for a vehicle according to claim 2, characterized in that: The target detection algorithm ( YOLO ) In the backbone part of the model, the attention mechanism CSPNet is introduced. The CSPNet divides the feature map (X) into two parts, and divides (X) into (X1) and (X2), where the number of channels of (X1) is αC, and the number of channels of (X2) is (1-α)C. α is a hyperparameter, usually set to 0.5; (X2) is processed through a series of convolutional layers, convolution, batch normalization, and activation functions to generate a feature map (F(X2)); the processed feature map (F(X2)) is spliced ​​with the directly transmitted feature map (X1) to generate the final output feature map (Y). The specific calculation formula is as follows: X1∈R H×W×64 X2∈R H×W×64 F(X2)=ReLU(BatchNorm(Conv(X2))) Y∈R H×W×64 Among them, the symbol R represents a set of real numbers, X1, X2, and Y are all three-dimensional tensors with a shape of H×W×64, all elements are real numbers, ReLU() is the activation function, Conv is the convolutional layer, and BatchNorm is the batch normalization layer.

4. A multi-level alarm method for a vehicle according to claim 1, characterized in that: The specific steps of the tracking and classification module for tracking, filtering, screening and classifying object coordinates and feature information are as follows: S501: Divide the detection frames of the drawn rectangular wireframe into a high-resolution frame set and a low-resolution frame set, then create corresponding Tracks, and use Kalman filtering to predict the position of the track in the current frame; S502: Perform IOU matching on the target detection frame of the frame and the rectangular frame predicted by Tracks in the previous frame one by one, and then calculate the cost matrix based on the result of IOU matching, and the calculation method is 1-IOU; S503: using all the cost matrices obtained in step S502 as inputs of the Hungarian algorithm to obtain linear matching results; S504: Loop step S502 and step S503 until the video frame ends.

5. A multi-level alarm method for a vehicle according to claim 4, characterized in that: The IOU Match in step S502 uses DIoU-NMS instead of traditional NMS. DIoU-NMS not only considers the intersection over union (IoU) between the predicted boxes, but also considers the distance between the center points of the two boxes, so as to better handle targets with small overlapping areas but close distances, reduce false positive detections, and improve detection accuracy. The specific calculation formula is as follows: Where S i is the classification confidence, ε is the NMS threshold, M is the detection box with the highest confidence, and B i is the i-th detection box, ρ represents the Euclidean distance between the two center points, b represents the center point of the prediction box, and b gt represents the center point of the real box, c is the diagonal length of the smallest rectangular box containing the predicted box and the target box, and the IoU value range is between 0 and 1.

6. A multi-level alarm method for a vehicle according to claim 4, characterized in that: The linear matching results obtained in step S503 include the following three types: The first is Unmatched Tracks, which directly deletes the unmatched Tracks. The second type is Unmatched Detections, which initializes such Detections as a new Track. The third is that the detection box and the predicted box are paired successfully, indicating that the previous frame and the next frame are tracked successfully, and the corresponding Detections are updated through the Kalman filter to update the corresponding Tracks variables.

7. A multi-level alarm method for a vehicle according to claim 1, characterized in that: The calculation formula of the risk assessment index F2 is as follows: F2=F1×Time Among them, Precision is the accuracy, which is the proportion of correctly detected risk objects to all detected objects, Recall is the recall rate, which is the proportion of correctly detected risk objects to all actually existing risk objects, F1 is an indicator that comprehensively considers precision and recall rate, TP is the true positive example, that is, the number of samples correctly predicted by the model as positive. FP is the false positive example, that is, the number of samples incorrectly predicted by the model as positive, FN is the false negative example, that is, the number of samples incorrectly predicted by the model as negative, FN is the false negative example, that is, the number of samples incorrectly predicted by the model as negative, F2 is the time taken for the target to cross the dangerous area in the positive direction, and F2 is the risk assessment indicator.

8. A multi-level alarm system for a vehicle, characterized in that: According to a multi-level alarm method for a vehicle according to any one of claims 1 to 7, the system includes an image acquisition module, a preprocessing module, a processing module, a target detection module, a tracking and classification module, a post-processing module, an alarm module, a display module and a storage module. The image acquisition module will collect real-time video streams in the vehicle monitoring area, and perform multi-screen splicing, brightness saturation modification, distortion correction, noise reduction, and cropping through the preprocessing module, and then the preprocessed screen is enlarged or reduced through the processing module, and the built-in YOLO algorithm of the target detection module is used to detect each frame of the image in real time, and the coordinates of the target in the screen and the corresponding target category are output, and the output results of the target detection module are tracked, filtered and screened through the tracking and classification module, and finally, according to the position and size of the target, the distance between it and the vehicle is calculated, and the potential danger level is evaluated in combination with the speed of the vehicle and the motion state of the target, and an alarm prompt is issued through the alarm module.

9. The multi-level warning system for a vehicle according to claim 8, characterized in that: The target detection module also includes an input module, a prediction module, a category prediction module, a calculation module and an output module. The input module divides the image data and passes it to the prediction module according to the resolution requirements of different models. The prediction module is used to predict border information and confidence. The category prediction module predicts the border category. The calculation module is used to calculate the comprehensive score and non-maximum suppression. The output module outputs a tensor tensor.

Images