Safety belt unfastening detection method and device and electronic equipment

Abstract

The invention discloses a safety belt unfastening detection method and device and electronic equipment. According to the unfastened safety belt detection method and device and the electronic equipment, the video frame of the unfastened safety belt detection object is obtained, the video frame is input into the human head detection model to obtain the human head detection frame coordinate, and the safety belt detection area is determined according to the human head detection frame coordinate; a double-threshold method is adopted for judgment, and compared with single-threshold judgment, the judgment is more accurate; and meanwhile, an alarm result is obtained by combining alarm logic with continuous multi-frame behavior results, so that the influence when the detection area is shielded can be effectively reduced, and the accuracy and recall rate of detection are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision image processing, and in particular to a method, device and electronic equipment for detecting not fastening a seat belt. Background Art

[0002] Thanks to the rapid development of cutting-edge deep learning technology, the intelligent detection technology of not wearing seat belts based on computer vision has developed rapidly in recent years. The existing detection method of not wearing seat belts uses deep learning technology to detect seat belts. Through repeated iterations of massive image data, the classification model can accurately learn the characteristics of seat belts, so as to accurately judge whether the seat belt is fastened for a single frame of image, with extremely high accuracy.

[0003] However, when the human chest, i.e., the seat belt wearing area, is blocked, the seat belt wearing features are not obvious, and the existing seat belt unfastening detection methods are prone to false detection or missed detection, and the accuracy of the detection results needs to be improved. Summary of the invention

[0004] The purpose of the present invention is to provide a method, device and electronic device for detecting whether a seat belt is not fastened, so as to solve the problem that when the human chest, i.e., the seat belt wearing area, is blocked, the seat belt wearing features are not obvious, the existing seat belt unfastening detection method is prone to cause false detection or missed detection, and the accuracy of the detection result needs to be improved.

[0005] In a first aspect, the present invention provides a method for detecting a person not wearing a seat belt, comprising: Obtain a video frame of the object not wearing a seat belt; Input the video frame into a head detection model to obtain the coordinates of a head detection frame; Determine a seat belt detection area according to the coordinates of the head detection frame; The seat belt detection area is changed to an input size of a classification model and then input into the classification model to obtain a prediction value p of fastening the seat belt; Compare the predicted value p with threshold k1 and threshold k2; If p≤k1, outputting the determination result of the video frame as not fastening the seat belt, and issuing a not fastening the seat belt alarm; If p≥k2, outputting the determination result of the video frame as fastening the seat belt; If k1 < p < k2, according to the predicted value p of each video frame, the detection result p' of each video frame is obtained, where p' = 1 when p ≤ k1; p' = 0 when p ≥ k2; p' = y' when k1 < p < k2, and y' is the alarm percentage threshold; store the detection result p' of each video frame into a queue, the maximum length of the queue is the alarm time threshold t1, if the queue exceeds the maximum queue length, discard the detection result p' of the oldest frame; sum the detection results p' in the queue and divide by the queue length to obtain the queue ratio; determine whether the queue ratio is greater than the alarm percentage threshold y' and the detection result p' of the current video frame > 0; If the queue ratio is greater than the alarm percentage threshold y' and the detection result p' of the current video frame > 0, issue an unfastened seat belt alarm.

[0006] Further, in the step of obtaining the video frame of the unfastened seat belt detection object, the video frame of the unfastened seat belt detection object is collected by a camera, and the camera is installed in the following manner: The camera is installed on the instrument panel at the front end of the steering wheel. The installation position of the camera is on the same straight line as the central axis of the steering wheel, or is installed within 10 cm on the left and right sides of the center line of the steering wheel. The front and rear installation position of the camera is 50 cm away from the center of the steering wheel, and the position deviation is within 5 cm in the front and rear directions; Further, after the camera is installed, it is calibrated in the following manner: Adjust the height of the camera so that the steering wheel does not appear in the picture; Adjust the angle of the camera so that the height of the driver's head is at the auxiliary frame in the display picture. The vertical line of the auxiliary frame divides the picture width into three equal parts. Taking the upper left corner as the image origin, the left side of the frame is at 1 / 3 of the picture width, and the right side of the frame is at 2 / 3 of the picture width; the horizontal line is at 1 / 6 of the picture height, and the lower side of the frame is at 2 / 3 of the picture height.

[0007] Further, determining the seat belt detection area according to the head detection frame coordinates includes: The two-point method representation coordinates of the head detection frame are [x1, y1, x2, y2]. The horizontal coordinates of the seat belt detection area are determined according to x1 and x2. Expand 0.7 times the width of the head frame to the left and right directions based on x1 and x2 to obtain the horizontal coordinates belt_x1 and belt_x2 of the seat belt detection area: when belt_x1 is less than 0, take 0, and when belt_x2 is greater than the image width, take the image width value; The vertical coordinates of the seat belt detection area are determined according to y1 and the image height. The vertical coordinate belt_y1 of the seat belt detection area expands 50 pixels upward based on y1, and the vertical coordinate belt_y2 of the seat belt detection area takes the bottom of the picture.

[0008] Furthermore, in the step of comparing the predicted value p with the threshold value k1 and the threshold value k2, the method for selecting the threshold value k1 and the threshold value k2 is as follows: A batch of images are selected from the vehicle operation video and classified into seat belt fastening, seat belt unfastening, and occlusion; Use the classification model to obtain the classification scores of the above three types of pictures respectively; According to the classification results, the distribution curves of the classification scores of the three categories of images are obtained, where category 1, category 2, and category 3 represent the categories of seat belt fastening images, unseat belt fastening images, and occlusion images, respectively; According to the distribution curve of the classification results, the threshold k1 and threshold k2 are obtained, where k1 is the intersection of the distribution curves of category 2 and category 3, and k2 is the intersection of the distribution curves of category 1 and category 3. When the image score is less than k1, the probability of not fastening the seat belt is the highest. When the image score is greater than k2, the probability of fastening the seat belt is the highest. When the image score is between k1 and k2, the maximum probability of the image is an occluded image.

[0009] Furthermore, if the queue ratio is greater than the alarm percentage threshold y' and the detection result p' of the current video frame>0, the method further includes: Determine whether the difference between the current video frame time and the alarm timestamp is greater than the preset alarm time interval; If the difference between the current video frame time and the alarm timestamp is greater than the preset alarm time interval, a seat belt unfastened alarm is issued and the alarm timestamp is updated.

[0010] In a second aspect, the present invention provides a seat belt unfastened detection device, comprising: An acquisition unit, used for acquiring a video frame of a detection object not wearing a seat belt; A detection unit, used to input the video frame into a head detection model to obtain the coordinates of a head detection frame; A determination unit, configured to determine a seat belt detection area according to the coordinates of the head detection frame; A prediction unit, used for changing the seat belt detection area to an input size of a classification model and then inputting the input into the classification model to obtain a prediction value p of fastening the seat belt; A comparison unit, used for comparing the predicted value p with a threshold value k1 and a threshold value k2; an alarm unit, configured to output, when p≤k1, a determination result of the video frame as not fastening a seat belt, and issue a not fastening a seat belt alarm; and, when p≥k2, output a determination result of the video frame as fastening a seat belt; A storage unit is used to obtain the detection result p' of each video frame according to the predicted value p of each video frame when k1 < p < k2, where p' = 1 when p ≤ k1; p' = 0 when p ≥ k2; p' = y' when k1 < p < k2, and y' is the alarm percentage threshold; store the detection result p' of each video frame into a queue, the maximum length of the queue is the alarm time threshold t1, and if the queue exceeds the maximum queue length, discard the detection result p' of the oldest frame; A calculation unit is used to sum the detection results p' in the queue and then divide by the queue length to obtain a queue ratio; A judgment unit is used to judge whether the queue ratio is greater than the alarm percentage threshold y' and the detection result p' of the current video frame > 0; The alarm unit is further used to issue a seat belt not fastened alarm when the queue ratio is greater than the alarm percentage threshold y' and the detection result p' of the current video frame > 0.

[0011] In a third aspect, the present invention provides an electronic device, including: a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to execute the steps of the method described in the first aspect above.

[0012] Advantages of the present invention: The seat belt not fastened detection method, device and electronic device provided by the present invention obtain the video frames of the seat belt not fastened detection object, input the video frames into the head detection model to obtain the head detection frame coordinates, determine the seat belt detection area according to the head detection frame coordinates, and adopt a double-threshold method to judge for the problem of occlusion in the seat belt detection area, which is more accurate than a single-threshold judgment; at the same time, the alarm result is obtained by combining the alarm logic with the behavior results of multiple consecutive frames, which can effectively reduce the influence when the detection area is occluded, thereby improving the detection accuracy and recall rate. Description of the Drawings

[0013] In order to further understand the features and technical content of the present invention, please refer to the following detailed description and drawings of the present invention. However, the drawings are only provided for reference and illustration purposes and are not used to limit the present invention. Drawings

[0014] Figure 1 is a flowchart of the seat belt not fastened detection method of the present invention; Figure 2 is a schematic diagram of the camera installation position; Figure 3 is a schematic diagram of the camera imaging; Figure 4 is a schematic diagram of the obtained seat belt detection area; Figure 5 is a flowchart of the selection method of the threshold k1 and the threshold k2; Figure 6 is the classification score distribution diagram; Figure 7 It is the occlusion map of the seat belt detection area; Figure 8 It is a flow chart of an implementation method of a method for detecting not fastening a seat belt according to the present invention; Fig. 9 It is a block diagram of the device for detecting the unfastened seat belt of the present invention; Fig.10 It is a block diagram of an electronic device of the present invention. DETAILED DESCRIPTION

[0015] To further illustrate the technical means and effects of the present invention, the following is a detailed description in conjunction with the preferred embodiments of the present invention and the accompanying drawings.

[0016] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0017] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first" and "second" are used only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0018] In this application, the word "exemplary" is used to mean "serving as an example, illustration, or description". Any embodiment described in this application as "exemplary" is not necessarily to be construed as being preferred or advantageous over other embodiments. The following description is given to enable any technician in the field to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present application can be implemented without using these specific details. In other instances, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.

[0019] When a driver does not wear a seat belt, there are the following dangers during driving: Injury to the driver and passengers. In the event of a car accident, the driver and passengers who do not wear seat belts will have a secondary collision with hard objects, steering wheels, windshields, seat backs and other objects in the car, which can easily cause injury to the occupants. Reduce the driver's concentration. Not wearing a seat belt will reduce the driver's concentration when driving, which can easily cause safety accidents. The driver and passengers are thrown out of the car. When the car collides or encounters an unexpected emergency brake, the driver and passengers who do not wear seat belts will be thrown out of the seat or thrown out of the car due to the huge inertia force, which is very dangerous. Therefore, it is necessary to detect the driver for not wearing a seat belt. At present, the existing technology uses deep learning technology to detect seat belts. Through repeated iterations of massive image data, the classification model can accurately learn the characteristics of the seat belt, so as to accurately judge whether the seat belt is fastened for a single frame image. However, in the actual seat belt failure detection process, the driver's seat belt may be blocked by the driver's arms, collar and other external environments, resulting in an inability to accurately identify whether the driver is wearing a seat belt. The present invention designs a detection method for this scenario, which can effectively reduce the impact of occlusion on the results and improve the algorithm's precision and recall rate.

[0020] See also Figure 1 The embodiment of the present invention provides a method for detecting a person not wearing a seat belt, comprising: S101, obtaining a video frame of a detection object not wearing a seat belt.

[0021] In this embodiment, the video frame of the unfastened seat belt detection object is collected by a camera, and the camera collects a video stream. The video stream is composed of a series of continuous image frames and is played at a specific playback speed to present a dynamic visual effect. According to different encoding methods, the video stream can be divided into an original stream and an encoded stream. The original stream is an uncompressed video stream, which is composed of images in YUV format; the encoded stream is a compressed and encoded video stream, such as a video stream compressed based on compression / encoding algorithms such as H.264. A video frame is a single image in a video and is also one of the basic units that make up a video sequence. Usually, the parameters of a video frame include frame rate, resolution, bit rate, etc. The video stream is composed of continuous video frames, and a dynamic video effect is presented by quickly switching and playing continuous image frames. The unfastened seat belt detection method of the present invention can detect whether the driver is wearing a seat belt in a video frame. First, the camera needs to be installed and calibrated according to the following method.

[0022] See also Figure 2 , which is an exemplary installation position of the camera. The camera needs to be installed on the dashboard in front of the steering wheel. The best installation position should be in a straight line with the central axis of the steering wheel. If this cannot be met, it can be installed within 10 cm to the left and right of the center line of the steering wheel. The closer to the central axis, the better. The installation steps are as follows: The horizontal position of the installation finds the center line through the central axis of the steering wheel, and the installation position deviation is within 10 cm to the left and right of the horizontal center line. The front and rear positions of the installation are 50 cm away from the center of the steering wheel, and the position deviation is within 5 cm before and after.

[0023] See also Figure 3 After the camera is installed, you need to adjust and verify that the camera image meets the algorithm analysis requirements. The adjustment steps are as follows: Adjust the appropriate height, requiring that the steering wheel cannot appear in the image. If this cannot be met, add a bracket to raise the camera position. Adjust the appropriate angle, requiring the driver's head height to be at the auxiliary frame on the display screen. After determining the angle, tighten the camera angle fixing screws. The image should be as follows: Figure 3As shown. In the figure, the vertical line of the auxiliary frame is divided into three equal parts for cutting the picture width, with the upper left corner as the image origin, the left side of the frame at 1 / 3 of the picture width, and the right side of the frame at 2 / 3 of the picture width. The horizontal line is at 1 / 6 of the picture height, and the bottom of the frame is at 2 / 3 of the picture height. The image resolution obtained by the camera of the present invention is 1080*720, and the resolution of 1080P means that there are 1080 pixels per line, a total of 720 lines, and nearly 800,000 pixels can be presented. This high-pixel resolution can provide clearer, more delicate picture details and richer color performance, which is convenient for unfastened seat belt detection. In addition, the resolution of 1080P is widely used in the field of high-definition televisions and displays, and the resolution of most screens can match or exceed 1080P. Therefore, using this resolution can ensure that the image maintains the best clarity and detail performance when displayed. The resolution of 1080P is one of the standards for high-definition video and is widely used in various high-definition video applications and devices. This high compatibility enables 1080P resolution images to be widely used in various fields. Although the resolution of 1080P is relatively high, it can still reduce the storage space requirement while ensuring image quality. Compared with images with higher resolutions, the file size of 1080P images is more moderate and suitable for storage and transmission. Therefore, in the unfastened seat belt detection method of the present invention, a camera with a picture resolution of 1080*720 is used.

[0024] S102, inputting the video frame into a head detection model to obtain the coordinates of a head detection frame.

[0025] The front image of the driver while driving is obtained through the above-mentioned camera installation position and calibration method, and then the video frame is input into the head detection model to obtain the head detection frame coordinates, which refer to the rectangular frame coordinates of the head position detected in the image or video. Usually, the head detection frame coordinates include the coordinate values ​​of the upper left corner and the lower right corner, so that the seat belt detection area is determined according to the head detection frame coordinates. This method has less interference and difficulty in obtaining the detection area.

[0026] The head detection model can use YOLOv5, which is composed of multiple parts. First, the model uses the YOLOv5 algorithm, which is a popular target detection algorithm. It achieves target detection by dividing the image into multiple grids, each of which is responsible for predicting one or more objects. Secondly, there are several preset anchor boxes in each grid, which have preset width, height, coordinates and confidence. During the training process, the anchor box in which grid the midpoint of the real manual annotation box falls will "grow" or "shrink" towards the real box, and set the confidence to 1, indicating that the grid has an object, and the confidence of other grids without predicted boxes is 0. In addition, in order to enhance the performance of the network, YOLOv5 adopts the Focus structure and CSP structure. The Focus structure can fuse feature maps of different scales, thereby enhancing the network's detection ability for targets of different scales. The CSP structure can separate and compress the feature information in the network, thereby improving the performance of the network. Finally, in order to improve the multi-scale detection ability of the network, YOLOv5 also adopts the FPN (Feature Pyramid Network) structure. This structure can achieve this goal by upsampling and concatenating the previous layers. Through the above structures, the head detection model of YOLOv5 can effectively detect the head in the image.

[0027] The general steps for head detection using Yolov5 include: Prepare the dataset: Collect image or video datasets containing human heads and divide them into training and test sets. Preprocess the dataset: Preprocess the dataset, including image scaling, cropping, grayscale, etc. Train the model: Train the model using the YOLOv5 model and the training dataset. This involves defining the structure of the model and then training it using the training data. Test the model: Evaluate and test the trained model using the test set to determine the accuracy and performance of the model. Apply the model: Apply the trained model to the actual scene. It should be noted that when using Yolov5 for head detection, the model needs to be properly adjusted and optimized to meet the needs of the actual scene. For example, parameters such as the number of categories, category names, and pre-trained models of the model can be adjusted to improve the accuracy of the model. In addition, it should be noted that processing large amounts of image or video data may require high-performance computer hardware and sufficient computing resources.

[0028] S103: Determine a seat belt detection area according to the coordinates of the head detection frame.

[0029] Specifically, the method of determining the seat belt detection area [belt_x1, belt_y1, belt_x2, belt_y2] is as follows: The two-point method of the head detection frame represents the coordinates as [x1, y1, x2, y2]. The horizontal coordinates of the seat belt detection area are determined based on x1 and x2. Based on x1 and x2, the horizontal coordinates of the seat belt detection area are expanded to the left and right directions by 0.7 head frame widths to obtain belt_x1 and belt_x2: belt_x1=x1-0.7*(x2-x1); belt_x1=0 if belt_x1<0; belt_x2=x2+0.7*(x2-x1); belt_x2=1080 if belt_x2>1080.

[0030] When belt_x1 is less than 0, it is taken as 0. When belt_x2 is greater than the image width, it is taken as the image width, where 1080 is the image width.

[0031] The vertical coordinate of the seat belt detection area is determined according to y1 and the image height. The vertical coordinate of the seat belt detection area, belt_y1, is 50 pixels upward based on y1. The vertical coordinate of the seat belt detection area, belt_y2, is the bottom of the image, where 720 is the image height: belt_y1=y1-50; belt_y2=720.

[0032] The seat belt detection area obtained according to the present invention is as follows: Figure 4 shown.

[0033] S104, changing the seat belt detection area to an input size of a classification model and then inputting the area into the classification model to obtain a prediction value p for fastening the seat belt.

[0034] Specifically, a classification model based on machine learning is used: the model uses an existing unfastened seat belt data set, uses a machine learning method to train the data, establishes a classification model, and then uses the model to analyze new data to determine whether there is unfastened seat belt behavior. When establishing an unfastened seat belt classification model, it is necessary to fully consider the reliability, accuracy, real-time performance of the data set, and at the same time, fully consider the scope of application and application scenarios of the model. The input size of the classification model can be adjusted, but usually a specific input size is selected, such as 224x224 pixels. This is because at this size, the image information is relatively rich, and at the same time, the computational burden of the model will not be increased too much. The specific choice may depend on the performance requirements and computing resources of the model. Some models may choose a larger input size to obtain more image information, thereby improving the classification accuracy. However, this approach will also increase the computational workload and storage requirements of the model. Therefore, when selecting the input size of the classification model, it is necessary to weigh various factors, including classification accuracy, computing resources, storage requirements, etc. After the present invention changes the seat belt detection area to the input size of the classification model, it is input into the classification model to obtain the predicted value p of fastening the seat belt.

[0035] Furthermore, the above classification model can be trained and predicted using a deep learning algorithm and a convolutional neural network (CNN). Data collection and preprocessing: Collect image or video datasets containing people who are not wearing seat belts and people who are not wearing seat belts, and annotate and preprocess the data, such as adjusting the image size, normalizing pixel values, etc. Feature extraction: Use a deep learning algorithm, such as a convolutional neural network (CNN), to extract features associated with not wearing seat belts from images or videos. These features may include human posture, clothing, background, etc. Model training: Use the extracted features as input and train with data with known labels. During the training process, continuously adjust the model parameters to improve the accuracy and generalization ability of the model so that the model can correctly classify whether someone is not wearing a seat belt. Model evaluation: Use a test set to evaluate the model, and calculate indicators such as the model's precision and recall rate to determine the performance of the model. Model deployment: Deploy the trained model to actual application scenarios, such as vehicle-mounted systems or monitoring systems, to detect in real time whether someone is not wearing a seat belt in an image or video.

[0036] When building a classification model for not wearing seat belts, the following points should be noted: Data quality: The quality of the data set directly affects the performance of the model. It is necessary to select a high-quality data set, including sufficient sample size and diversity. Feature selection: Selecting features related to not wearing seat belts is the key. It is necessary to deeply analyze the data and mine effective features to improve the accuracy of the model. Model selection: According to the characteristics of the data set and the actual application requirements, select the appropriate deep learning algorithm and model architecture. Training and optimization: During the training process, the model needs to be optimized and the model parameters need to be adjusted to improve the accuracy and generalization ability of the model.

[0037] Real-time: In real-time detection scenarios, it is necessary to ensure the detection speed and efficiency of the model to avoid affecting system performance.

[0038] S105, comparing the predicted value p with threshold k1 and threshold k2.

[0039] The present invention designs a dual-threshold selection method for detecting area occlusion, compares the predicted value p with thresholds k1 and k2, and thus obtains a single-frame detection result, which is more accurate than a single threshold judgment.

[0040] In this example, see Figure 5 , the selection method of threshold k1 and threshold k2 is as follows: S1051, select a batch of images from the vehicle operation video and classify them according to whether the vehicle is wearing a seat belt, not wearing a seat belt, and being blocked.

[0041] Among them, the seat belt category is for those who can clearly see that the seat belt is fastened, the unbelted category is for those who are not fastened, and the pictures that are obscured and difficult to judge are classified as the obscured category.

[0042] S1052, using the classification model to obtain classification scores for the above three types of images respectively.

[0043] S1053, obtaining distribution curves of classification scores of three categories of pictures according to the classification results, where category 1, category 2, and category 3 represent seatbelt-fastened picture categories, unseatbelt-fastened picture categories, and occluded picture categories, respectively.

[0044] The classification results of the present invention are distributed as follows Figure 6 shown.

[0045] S1054, based on the distribution curve of the classification result, thresholds k1 and k2 are obtained, where k1 is the intersection of the distribution curves of category 2 and category 3, and k2 is the intersection of the distribution curves of category 1 and category 3; when the image score is less than k1, the probability of not wearing a seat belt is the highest; when the image score is greater than k2, the probability of wearing a seat belt is the highest; when the image score is between k1 and k2, the image is most likely to be an occluded image. Figure 7 as shown Figure 7 On the left side in it is the occluded image without wearing a seat belt, and on the right side is the occluded image with a seat belt worn

[0046] S106, if p ≤ k1, output the determination result of the video frame as not wearing a seat belt, and issue a warning for not wearing a seat belt

[0047] S107, if p ≥ k2, output the determination result of the video frame as wearing a seat belt

[0048] S108, if k1 < p < k2, obtain the detection result p' of each video frame according to the predicted value p of each video frame, where p' = 1 when p ≤ k1; p' = 0 when p ≥ k2; p' = y' when k1 < p < k2, and y' is the alarm percentage threshold

[0049] When k1 < p < k2, no judgment is made temporarily, but S108 and subsequent steps are executed for further judgment

[0050] S109, store the detection result p' of each video frame into a queue, the maximum length of the queue is the alarm time threshold t1, if the queue exceeds the maximum queue length, discard the detection result p' of the oldest frame

[0051] Initialize the queue to be empty, preset the alarm time threshold t1, set the maximum queue length to the alarm time threshold length, set the alarm percentage threshold y', and set the alarm time interval

[0052] S110, sum the detection results p' in the queue and divide by the queue length to obtain the queue ratio

[0053] S111, determine whether the queue ratio is greater than the alarm percentage threshold y' and the detection result p' of the current video frame > 0

[0054] If the queue ratio is greater than the alarm percentage threshold y' and the detection result p' of the current video frame > 0, execute S112 to issue a warning for not wearing a seat belt. If the queue ratio is greater than the alarm percentage threshold y' and the detection result p' of the current video frame > 0, repeat the execution of S109

[0055] Among them, in S108, when k1 < p < k2, p' = y'. Finding reasonable k1 and k2 and setting the frame results with scores between k1 and k2 to the alarm percentage is the key to eliminating the occlusion influence. This operation weakens the influence of the uncertainty of the model prediction score on the result queue to the greatest extent, and the uncertainty of the model prediction score mainly comes from the occlusion of the seat belt detection area. The principle is as follows: ............ Equation 1.1 ...........Formula 1.2 ...........Formula 1.3 When all the blocks are blocked during a period of time, the predicted value of each frame is is between the thresholds k1 and k2. ', then the final result ratio of the queue is equal to the alarm percentage, as shown in formula 1.1. When there is occlusion in a certain period of time but it is not always occluded, there is an unoccluded picture frame In addition to k1 and k2, if the image frame is not wearing a seat belt, the predicted value is is less than the threshold k1, then there exists , then the final queue ratio is greater than the alarm percentage, and an alarm is generated, as shown in formula 1.2. If it is a picture frame of a person wearing a seat belt, then the predicted value is greater than the threshold k2, then there exists 0, the final queue ratio is less than the alarm percentage and no alarm is generated, as shown in Formula 1.3. Compared with a single threshold, the occluded image is judged as either fastened or unfastened according to the threshold. When the number of occluded results in the queue is greater than the number of unoccluded results, false alarms and missed alarms will occur. The logic of the present invention avoids this problem.

[0056] See also Figure 8 As an optional implementation, S111, determine whether the queue ratio is greater than the alarm percentage threshold y' and the detection result p' of the current video frame is > 0. If the queue ratio is greater than the alarm percentage threshold y' and the detection result p' of the current video frame is > 0, S112, before issuing a seat belt failure alarm, the method further includes: S801, determining whether the difference between the current video frame time and the alarm timestamp is greater than a preset alarm time interval.

[0057] S802, if the difference between the current video frame time and the alarm timestamp is greater than the preset alarm time interval, issue a seat belt unfastened alarm and update the alarm timestamp. If the difference between the current video frame time and the alarm timestamp is less than or equal to the preset alarm time interval, repeat S109.

[0058] In this embodiment, the alarm timestamp refers to the time point when the unfastened seat belt behavior is detected. In practical applications, the alarm timestamp can be recorded in a time format, such as year, month, day, hour, minute, second, etc., and can be accurate to the second level. This timestamp records when the alarm event occurs. The form of the unfastened seat belt alarm can include but is not limited to sound alarms, light prompts, vibration prompts, and voice prompts.

[0059] Sound alarm: When the vehicle detects that the driver or occupant is not wearing a seat belt, a sound alarm can be issued on the dashboard or seat. This alarm is usually a beep or a prompt tone. Light prompt: When the seat belt is not fastened, a flashing light will light up on the dashboard or seat of the vehicle, usually a red or yellow light, to remind the driver and occupant to fasten the seat belt. Vibration prompt: The seats or steering wheels of some vehicles can vibrate to remind the driver and occupant that the seat belt is not fastened. This prompt method is relatively gentle and will not cause excessive interference to the driver and occupant. Voice prompt: A voice prompt system can be configured. When the driver and occupant are not wearing a seat belt, the system will inform the driver through voice prompts that they need to fasten the seat belt. Setting an appropriate alarm time interval can improve the performance of the alarm system. For example, a short alarm interval may cause a large consumption of system resources, thereby reducing the service life of the system; while a longer alarm interval can reduce the consumption of system resources and improve the stability and service life of the system.

[0060] From the above, it can be seen that the unfastened seat belt detection method provided by the embodiment of the present invention obtains the detection area through the calibration method of the camera installation position and the head detection model, which has less interference and difficulty in obtaining the detection area than other methods; by obtaining the video frame of the unfastened seat belt detection object, the video frame is input into the head detection model to obtain the head detection frame coordinates, and the seat belt detection area is determined according to the head detection frame coordinates. In order to solve the problem of occlusion of the seat belt detection area, a double threshold method is used for judgment, which is more accurate than a single threshold judgment; at the same time, the alarm result is obtained by combining the alarm logic with the behavior results of multiple consecutive frames, which can effectively reduce the impact when the detection area is blocked, thereby improving the detection accuracy and recall rate.

[0061] See also Fig. 9 , an embodiment of the present invention provides a seat belt unfastened detection device, comprising: An acquisition unit 91 is used to acquire a video frame of a detection object not wearing a seat belt; A detection unit 92, configured to input the video frame into a head detection model to obtain the coordinates of a head detection frame; A determination unit 93, configured to determine a seat belt detection area according to the coordinates of the head detection frame; A prediction unit 94, configured to change the seat belt detection area into an input size of a classification model and then input the input into the classification model to obtain a prediction value p of fastening the seat belt; A comparison unit 95, configured to compare the predicted value p with a threshold k1 and a threshold k2; The alarm unit 96 is used to output the determination result of the video frame as not fastening the seat belt and issue a not fastening seat belt alarm when p≤k1; and output the determination result of the video frame as fastening the seat belt when p≥k2; Storage unit 97 is used to obtain the detection result p' of each video frame according to the predicted value p of each video frame when k1 < p < k2, where p' = 1 when p ≤ k1; p' = 0 when p ≥ k2; p' = y' when k1 < p < k2, and y' is the alarm percentage threshold; store the detection result p' of each video frame into a queue, and the maximum length of the queue is the alarm time threshold t1. If the queue exceeds the maximum queue length, discard the detection result p' of the oldest frame; Calculation unit 98 is used to sum the detection results p' in the queue and divide by the queue length to obtain the queue ratio; Judgment unit 99 is used to judge whether the queue ratio is greater than the alarm percentage threshold y' and the detection result p' of the current video frame > 0. The alarm unit 96 is further used to issue a seatbelt not fastened alarm when the queue ratio is greater than the alarm percentage threshold y' and the detection result p' of the current video frame > 0.

[0062] As can be seen from the above, the seatbelt not fastened detection device provided by the embodiment of the present invention obtains the video frame of the seatbelt not fastened detection object, inputs the video frame into the head detection model to obtain the head detection frame coordinates, determines the seatbelt detection area according to the head detection frame coordinates, and adopts a dual-threshold method to judge for the problem of occlusion in the seatbelt detection area, which is more accurate than a single-threshold judgment; at the same time, the alarm result is obtained by combining the warning logic with the behavior results of multiple consecutive frames, which can effectively reduce the influence when the detection area is occluded, thereby improving the detection accuracy and recall rate.

[0063] Please refer to Fig.10 The embodiment of the present invention further provides an electronic device, a memory 100 and a processor 200. The memory stores a computer program. When the computer program is executed by the processor, the processor executes the steps of the above method.

[0064] The embodiment of the present invention further provides a storage medium. The storage medium stores a computer program. When the computer program is executed by a processor, it implements some or all of the steps in the seatbelt not fastened detection methods provided by the embodiments of the present invention. The storage medium can be a magnetic disk, an optical disk, a read-only memory (abbreviation: ROM) or a random access memory (abbreviation: RAM), etc.

[0065] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present invention or certain parts of the embodiments.

[0066] In this specification, the same or similar parts between the various embodiments can be referred to each other. In particular, for the embodiment of the unfastened seat belt detection device, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiment.

[0067] The above-described embodiments of the present invention do not limit the protection scope of the present invention.

Claims

1. A method for detecting a person not wearing a seat belt, characterized in that: Including: Obtain a video frame of an object without a seat belt fastened; Input the video frame into a head detection model to obtain the coordinates of the head detection box; Determine the seat belt detection area according to the coordinates of the head detection box; Change the seat belt detection area to the input size of a classification model and then input it into the classification model to obtain the predicted value p of whether the seat belt is fastened; Compare the predicted value p with a threshold k1 and a threshold k2; If p ≤ k1, output the determination result of the video frame as not wearing a seat belt and issue an alarm for not wearing a seat belt; If p ≥ k2, output the determination result of the video frame as wearing a seat belt; If k1 < p < k2, according to the predicted value p of each video frame, obtain the detection result p' of each video frame, where p' = 1 when p ≤ k1; p' = 0 when p ≥ k2; p' = y' when k1 < p < k2, and y' is the alarm percentage threshold; Store the detection result p' of each video frame in a queue, and the maximum length of the queue is the alarm time threshold t1. If the queue exceeds the maximum queue length, discard the detection result p' of the oldest frame; Sum up the detection results p' in the queue and divide by the queue length to obtain the queue ratio; Judge whether the queue ratio is greater than the alarm percentage threshold y' and the detection result p' of the current video frame > 0; If the queue ratio is greater than the alarm percentage threshold y' and the detection result p' of the current video frame > 0, issue an alarm for not wearing a seat belt.

2. The method for detecting not fastening a seat belt as claimed in claim 1, characterized in that: In the step of obtaining a video frame of an object without a seat belt fastened, the video frame of the object without a seat belt fastened is collected by a camera, and the camera is installed in the following manner: The camera is installed on the instrument panel at the front end of the steering wheel. The installation position of the camera and the central axis of the steering wheel are on the same straight line, or it is installed within 10 cm to the left and right of the center line of the steering wheel. The front and rear position of the camera is 50 cm away from the center of the steering wheel, and the position deviation is within 5 cm in the front and rear directions.

3. The method for detecting not fastening a seat belt as claimed in claim 2, characterized in that: After the camera is installed, it is calibrated in the following manner: Adjust the height of the camera so that the steering wheel does not appear in the picture; Adjust the angle of the camera so that the height of the driver's head is at the auxiliary frame in the display picture. The vertical line of the auxiliary frame divides the picture width into three equal parts for cutting. Taking the upper left corner as the image origin, the left side of the frame is at 1 / 3 of the picture width, and the right side of the frame is at 2 / 3 of the picture width; the horizontal line is at 1 / 6 of the picture height, and the lower side of the frame is at 2 / 3 of the picture height.

4. The method for detecting not wearing a seat belt according to claim 1, characterized in that: Determining the seat belt detection area according to the coordinates of the head detection box includes: The two-point method representation coordinates of the head detection box are [x1, y1, x2, y2]. The horizontal coordinates of the seat belt detection area are determined according to x1 and x2. Expand 0.7 times the width of the head box to the left and right directions based on x1 and x2 to obtain the horizontal coordinates belt_x1 and belt_x2 of the seat belt detection area: when belt_x1 is less than 0, take 0, and when belt_x2 is greater than the image width, take the image width value; The vertical coordinate of the seat belt detection area is determined based on y1 and the image height. The vertical coordinate belt_y1 of the seat belt detection area is expanded 50 pixels upward with y1 as the reference, and the vertical coordinate belt_y2 of the seat belt detection area is taken at the bottom of the picture.

5. The method for detecting not fastening a seat belt as claimed in claim 1, characterized in that: In the step of comparing the predicted value p with the threshold values k1 and k2, the selection methods of the threshold values k1 and k2 are as follows: Select a batch of pictures from the vehicle operation video and classify them according to whether the seat belt is fastened, not fastened, or blocked. Use the classification model to obtain the classification scores of the above three types of pictures respectively. Obtain the distribution curves of the classification scores of the three types of pictures according to the classification results. Category 1, Category 2, and Category 3 represent the pictures of fastened seat belts, pictures of unfastened seat belts, and pictures of occlusion respectively. Obtain the threshold values k1 and k2 according to the distribution curves of the classification results. Among them, k1 is the intersection point of the distribution curves of Category 2 and Category 3, and k2 is the intersection point of the distribution curves of Category 1 and Category 3. When the picture score is less than k1, the probability of not fastening the seat belt is the highest. When the picture score is greater than k2, the probability of fastening the seat belt is the highest. When the picture score is between k1 and k2, the picture is most likely an occluded picture.

6. The method for detecting not fastening a seat belt as claimed in claim 1, characterized in that: If the queue ratio is greater than the alarm percentage threshold y' and the detection result p' of the current video frame > 0, the method further includes: Determine whether the difference between the current video frame time and the alarm timestamp is greater than the preset alarm time interval. If the difference between the current video frame time and the alarm timestamp is greater than the preset alarm time interval, issue an alarm for not fastening the seat belt and update the alarm timestamp.

7. A seat belt unfastened detection device, characterized in that: Includes: An acquisition unit for acquiring video frames of the detection object not fastening the seat belt. A detection unit for inputting the video frames into a head detection model to obtain the head detection box coordinates. A determination unit for determining the seat belt detection area according to the head detection box coordinates. A prediction unit for changing the seat belt detection area to the input size of the classification model and then inputting it into the classification model to obtain the predicted value p of fastening the seat belt. A comparison unit for comparing the predicted value p with the threshold values k1 and k2. An alarm unit for outputting the determination result of the video frame as not fastening the seat belt and issuing an alarm for not fastening the seat belt when p ≤ k1. Outputting the determination result of the video frame as fastening the seat belt when p ≥ k2. A storage unit for obtaining the detection result p' of each video frame according to the predicted value p of each video frame when k1 < p < k2. Among them, p' = 1 when p ≤ k1; p' = 0 when p ≥ k2; p' = y' when k1 < p < k2, and y' is the alarm percentage threshold. Store the detection result p' of each video frame in a queue. The maximum length of the queue is the alarm time threshold t1. If the queue exceeds the maximum queue length, discard the detection result p' of the oldest frame. A calculation unit for summing the detection results p' in the queue and then dividing by the queue length to obtain the queue ratio. A judgment unit for judging whether the queue ratio is greater than the alarm percentage threshold y' and the detection result p' of the current video frame > 0. The alarm unit is further configured to issue a seat belt not-fastened alarm when the queue ratio is greater than an alarm percentage threshold y' and the detection result p' of the current video frame is >0.

8. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 6.

Images