Seat belt testing methods and devices

Abstract

This invention provides a seatbelt detection method and apparatus. The method includes: acquiring driving images within a preset time period; acquiring a seatbelt mask based on each frame of the driving images, and determining whether the two boundaries of the seatbelt are parallel based on the seatbelt mask to obtain a parallelism judgment result; segmenting the seatbelt mask to obtain a first seatbelt and a second seatbelt corresponding to each frame of the driving images; determining whether the first seatbelt and the second seatbelt are continuous to obtain a continuity judgment result; obtaining a seatbelt wearing judgment result for each frame of the driving images based on the parallelism judgment result and the continuity judgment result; and generating and sending an alarm message based on the sum of the seatbelt wearing judgment results for each frame of the driving images being greater than a first preset threshold. This invention improves the accuracy of seatbelt detection by determining the parallelism of the seatbelt boundaries within the driving images and the continuity of the first and second seatbelts.

Description

Technical Field

[0001] This invention relates to the field of vehicle technology, and in particular to a method and apparatus for detecting seat belts. Background Technology

[0002] With economic development and rapid urban construction, the number of cars is increasing year by year. This continued rapid growth in the number of cars places a huge burden on urban transportation systems. Illegal and unsafe driving behaviors persist, and drivers not wearing seat belts directly endanger their own lives and the lives of others. Seat belt detection is an important component of vehicle violation detection in the field of intelligent transportation, effectively assisting traffic enforcement officers in supervising drivers' seat belt use.

[0003] Traditional seatbelt detection is triggered by a contact switch inside the buckle. When the driver enters the cab and turns on the ignition key, the circuit of the seatbelt audible alert device is activated if the seatbelt is not inserted into the buckle, prompting the driver to fasten the seatbelt. After the seatbelt is fastened, the contact switch inside the seatbelt buckle is in the closed state, the power supply to the seatbelt reminder system is disconnected, and the audible alert stops.

[0004] However, since the detection system only targets the inside of the buckle, it has certain limitations. Drivers can bypass the seatbelt reminder by using the seatbelt buckle or by inserting the seatbelt directly into the buckle from behind, thus driving without wearing a seatbelt. Summary of the Invention

[0005] This invention provides a seat belt detection method and apparatus to address the shortcomings of existing technologies where drivers insert their seat belts into the buckles to block seat belt detection when they are not wearing them. This invention enables real-time and accurate detection of the driver's seat belt wearing status, thereby improving the reliability of seat belt detection.

[0006] This invention provides a seatbelt detection method, comprising: acquiring driving images within a preset time period; acquiring a seatbelt mask based on each frame of the driving images, and determining whether the two boundaries of the seatbelt are parallel based on the seatbelt mask to obtain a parallelism judgment result; segmenting the seatbelt mask to obtain a first seatbelt and a second seatbelt corresponding to each frame of the driving images; wherein, the first seatbelt is the part of the seatbelt that contacts the driver's body, and the second seatbelt is the part of the seatbelt that does not contact the driver's body; determining whether the first seatbelt and the second seatbelt are continuous to obtain a continuity judgment result; obtaining a seatbelt wearing judgment result for each frame of the driving images based on the parallelism judgment result and the continuity judgment result; and generating and sending an alarm message based on the sum of the seatbelt wearing judgment results for each frame of the driving images being greater than a first preset threshold.

[0007] According to a seat belt detection method provided by the present invention, the step of determining whether the two boundaries of the seat belt are parallel based on the seat belt mask includes: obtaining the linear equations of the two boundaries of the seat belt using Hough transform based on the seat belt mask; calculating the slopes corresponding to the two boundaries based on the linear equations of the two boundaries, and comparing the deviation of the slopes corresponding to the two boundaries with a first preset range; if the deviation is within the first preset range, then the two boundaries of the seat belt are parallel.

[0008] According to a seatbelt detection method provided by the present invention, the step of obtaining the coordinate equations of the two boundaries of the seatbelt based on the seatbelt mask and using Hough transform includes: performing edge detection on the seatbelt mask to obtain the edge points of the seatbelt; for each edge point, converting all straight lines passing through the edge point into points in polar coordinate space to form a curve corresponding to each edge point; obtaining multiple intersection points based on the curves corresponding to all edge points; selecting two intersection points according to the number of curves passing through the intersection points, and calculating the straight line equations in their corresponding rectangular coordinate space respectively to obtain the straight line equations of the two boundaries of the seatbelt.

[0009] According to a seat belt detection method provided by the present invention, before obtaining the linear equations of the two boundaries of the seat belt using Hough transform based on the seat belt mask, the method further includes: subjecting the seat belt mask to corrosion and expansion treatment.

[0010] According to a seat belt detection method provided by the present invention, the step of determining whether the first seat belt and the second seat belt are continuous includes: identifying the seat belt mask to obtain the boundary connection result of the first seat belt and the second seat belt; if the boundary connection result shows that they are connected, locating the middle region of the first seat belt and the second seat belt, and extracting the first gray value of the first seat belt and the second gray value of the second seat belt located in the middle region; comparing the deviation between the first gray value and the second gray value with a second preset range, and if it is within the second preset range, then the first seat belt and the second seat belt are continuous.

[0011] According to a seatbelt detection method provided by the present invention, after acquiring driving images within a preset time period, the method includes: for each frame of the driving image, recognizing a face and locating a shoulder based on the recognized face; based on the located shoulder position, detecting whether the seatbelt overlaps with the shoulder position; recognizing a steering wheel area for each frame of the driving image and detecting whether the overlap area between the seatbelt and the steering wheel area is greater than a preset threshold; based on the overlap between the seatbelt and the shoulder position and the fact that the overlap area is greater than the preset threshold, determining whether the two boundaries of the seatbelt are parallel based on the seatbelt mask.

[0012] The present invention also provides a seat belt detection device, comprising: an image acquisition module for acquiring driving images within a preset time period; a parallel judgment module for acquiring a seat belt mask based on each frame of driving images, and judging whether the two boundaries of the seat belt are parallel based on the seat belt mask; a segmentation module for segmenting the seat belt mask to obtain a first seat belt and a second seat belt corresponding to each frame of driving images; wherein the first seat belt is the part of the seat belt in contact with the driver's body, and the second seat belt is the part of the seat belt not in contact with the driver's body; a continuity judgment module for judging whether the first seat belt and the second seat belt are continuous; a detection module for obtaining a seat belt wearing judgment result for each frame of driving images based on the parallel judgment result of the two boundaries of the seat belt and the continuity judgment result of the first seat belt and the second seat belt; and a control module for generating and sending alarm information based on the sum of the seat belt wearing judgment results of each frame of driving images being greater than a first preset threshold.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the seat belt detection methods described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the seatbelt detection methods described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the seat belt detection methods described above.

[0016] The seat belt detection method and apparatus provided by this invention determine whether a valid seat belt exists in the corresponding driving image by judging the parallelism of the seat belt boundary and the continuity of the first and second seat belts. This avoids the situation where the seat belt detection accuracy is poor due to the obstruction of objects, and effectively improves the accuracy of seat belt detection. By statistically analyzing the seat belt judgment results of each frame of driving image, the inaccuracy of the judgment results of a single frame of driving image is avoided, thereby further improving the accuracy of seat belt detection. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the seatbelt detection method provided by the present invention;

[0019] Figure 2 This is a schematic diagram of the structure of the seat belt detection device provided by the present invention;

[0020] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0022] Figure 1 This diagram illustrates a flow chart of a seatbelt detection method according to the present invention. The method includes:

[0023] S01, acquire driving images within a preset time period;

[0024] S02, Based on each frame of driving image, obtain the seat belt mask, and determine whether the two boundaries of the seat belt are parallel based on the seat belt mask, and obtain the parallel judgment result;

[0025] S03, segment the seat belt mask to obtain the first seat belt and the second seat belt corresponding to each frame of driving image; wherein, the first seat belt is the part of the seat belt that contacts the driver's body, and the second seat belt is the part of the seat belt that does not contact the driver's body;

[0026] S04, determine whether the first seat belt and the second seat belt are connected, and obtain the connection judgment result;

[0027] S05, based on the parallel judgment results and the continuous judgment results, obtain the seat belt wearing judgment result for each frame of driving image;

[0028] S06, based on the fact that the sum of the seat belt wearing judgment results of each frame of driving image is greater than the first preset threshold, generate and send alarm information.

[0029] It should be noted that S0N in this specification does not represent the order of the seat belt testing methods. The seat belt testing method of the present invention will be described in detail below.

[0030] Step S01: Obtain driving images within a preset time period.

[0031] In this embodiment, acquiring driving images within a preset time period includes: acquiring a video stream of the driving area; and extracting driving images within the preset time period based on the video stream. Specifically, extracting driving images within the preset time period based on the video stream includes: acquiring driving images of a certain number of frames before and after the current frame driving image, for example, acquiring a preset number of driving images before and after the current frame driving image, to avoid poor detection results due to subsequent judgment based on a single frame driving image. The specific number of driving images acquired can be set according to actual detection needs and is not further limited here; for example, it can be five driving images before and after the current frame driving image. In other embodiments, acquiring driving images within a preset time period includes: acquiring multiple consecutively captured driving images within the preset time period.

[0032] It should be noted that the driving images are taken from the driving area within the vehicle. The vehicle can be a car, ship, or airplane used to carry people or goods. The vehicle can be a private car or a commercial vehicle, such as a car-sharing service, ride-hailing service, taxi, bus, school bus, truck, coach, train, subway, and tram.

[0033] In one optional embodiment, a face-tracking camera can be used to acquire driving images of the driving area. The face-tracking camera is positioned in front of the driver's area to capture images of the driving area in the direction of the driver's face. It should be noted that the camera includes at least one of a high-resolution infrared camera, a DMS camera, an OMS camera, a video camera, a video recorder, and electronic devices with a camera. The electronic device can be a mobile terminal device such as a mobile terminal, computer, camera, or tablet computer.

[0034] In one optional embodiment, after acquiring driving images within a preset time period, the method includes: for each frame of the driving image, recognizing a face and locating the shoulder based on the recognized face; based on the located shoulder position, detecting whether the seat belt overlaps with the shoulder position; recognizing the steering wheel area for each frame of the driving image and detecting whether the overlap area between the seat belt and the steering wheel area is greater than a preset threshold; based on the overlap between the seat belt and the shoulder position and the fact that the overlap area is greater than the preset threshold, determining whether the two boundaries of the seat belt are parallel based on a seat belt mask. It should be noted that the preset threshold can be determined based on the driver's body type, the size of the steering wheel, and the area of ​​the seat belt located at the steering wheel position and in contact with the driver's body; no further limitations are imposed here.

[0035] Step S02: Based on each frame of driving image, obtain the seat belt mask, and determine whether the two boundaries of the seat belt are parallel based on the seat belt mask, and obtain the parallel judgment result.

[0036] In this embodiment, obtaining a seatbelt mask based on each frame of driving images includes: recognizing a face in each frame of the driving image, and locating the shoulder based on the recognized face; determining the seatbelt area based on the located shoulder position; and segmenting the seatbelt within the seatbelt area to obtain the seatbelt mask. Face recognition facilitates face location, thereby determining the shoulder position based on the located face position, and further determining the seatbelt area based on the shoulder position, thus improving the positioning accuracy of the seatbelt area and facilitating subsequent seatbelt detection.

[0037] Furthermore, determining whether the two boundaries of the seatbelt are parallel based on the seatbelt mask includes: using the Hough transform based on the seatbelt mask to obtain the linear equations of the two boundaries; calculating the slopes corresponding to the two boundaries based on their linear equations, and comparing the deviation of the slopes to a first preset range. If the deviation is within the first preset range, then the two boundaries of the seatbelt are parallel. It should be noted that the first preset range can be determined according to actual design requirements; for example, the first preset range can be set to less than 5.

[0038] Specifically, based on the seat belt mask, the coordinate equations of the two boundaries of the seat belt are obtained using the Hough transform, including: performing edge detection on the seat belt mask to obtain the edge points of the seat belt; for each edge point, converting all straight lines passing through the edge point into points in polar coordinate space to form a curve corresponding to each edge point; obtaining multiple intersection points based on the curves corresponding to all edge points; selecting two intersection points according to the number of curves passing through the intersection points, and calculating the straight line equations in the corresponding rectangular coordinate space for each, to obtain the straight line equations of the two boundaries of the seat belt.

[0039] It should be noted that since the curves corresponding to each edge point intersect, and different curves may form different intersection points, multiple intersection points are obtained for all curves, and each intersection point is traversed by at least one straight line. Furthermore, when selecting two intersection points based on the number of curves passing through them, the two intersection points with the most straight lines can be selected for subsequent calculation of the straight line equations. This yields the straight line equations for the two boundaries of the seatbelt, making it easier to determine whether the two boundaries of the seatbelt are parallel based on the straight line equations.

[0040] In an alternative embodiment, before obtaining the linear equations of the two boundaries of the seatbelt using the Hough transform based on the seatbelt mask, the method further includes: subjecting the seatbelt mask to an corrosion-expansion treatment. This corrosion-expansion treatment of the seatbelt mask facilitates the separation of the seatbelt and maintains smooth seatbelt boundaries, thereby enhancing the seatbelt's characteristics.

[0041] Specifically, the seat belt mask is subjected to corrosion and expansion treatment, including: corrosion of the seat belt mask to eliminate seat belt boundary points, thereby reducing the size of the seat belt mask and facilitating the elimination of noise points smaller than structural elements; and expansion of the corroded seat belt mask to merge all background points in contact with the seat belt into the seat belt, thereby increasing the size of the seat belt mask and filling the voids in the seat belt mask.

[0042] It should be noted that the etching of the seatbelt mask includes: scanning each pixel of the seatbelt mask using a preset first structuring element, and performing an AND operation between each pixel in the first structuring element and the seatbelt mask pixels it covers. If all values ​​are 1, the corresponding seatbelt mask pixel is 1; otherwise, it is 0. The expansion of the etched seatbelt mask includes: scanning each pixel of the etched seatbelt mask using a preset second structuring element, and performing an AND operation between each pixel in the second structuring element and the seatbelt mask pixels it covers. If all values ​​are 0, the corresponding seatbelt mask pixel is 0; otherwise, it is 1.

[0043] It should be noted that the size of the first and second structural elements can both be 3×3, and the first and second structural elements can be selected according to the actual number of pixels in the seat belt; no further restrictions are imposed here.

[0044] Step S03: Segment the seat belt mask to obtain the first seat belt and the second seat belt corresponding to each frame of driving image; wherein, the first seat belt is the part of the seat belt that contacts the driver's body, and the second seat belt is the part of the seat belt that does not contact the driver's body.

[0045] It should be noted that when segmenting the seatbelt mask to obtain the first and second seatbelts corresponding to each frame of the driving image, the portion of the seatbelt in contact with the driver's body and the portion not in contact with the driver's body can be identified through image recognition. This identified portion is then segmented to obtain the first and second seatbelts. It should also be noted that during seatbelt mask recognition, it is also possible to determine whether the boundaries of the first and second seatbelts are connected.

[0046] Step S04: Determine whether the first seat belt and the second seat belt are connected, and obtain the connection determination result.

[0047] In this embodiment, determining whether the first seat belt and the second seat belt are continuous includes: identifying the seat belt mask to obtain the boundary connection result of the first seat belt and the second seat belt; if the boundary connection result shows that they are connected, then locating the middle region of the first seat belt and the second seat belt, and extracting the first gray value of the first seat belt and the second gray value of the second seat belt located in the middle region; comparing the deviation between the first gray value and the second gray value with a second preset range, and if it is within the second preset range, then the first seat belt and the second seat belt are continuous.

[0048] It should be noted that when locating the middle area of ​​the first seat belt and the second seat belt, a portion of the area where the first seat belt and the second seat belt are connected can be selected as the middle area based on preset rules. This makes it easier to determine whether the first seat belt and the second seat belt are connected based on the first gray value and the difference between the second gray value of the middle area and the boundary connection result of the first seat belt and the second seat belt.

[0049] In addition, the second preset range can be set according to the angle of the camera device that actually collects driving images relative to the driving position, the body size of different drivers and / or the position of the driving position relative to the steering wheel, etc. For example, the second preset range can be less than 10.

[0050] Step S05: Based on the parallel judgment results and the continuous judgment results, obtain the seat belt wearing judgment result for each frame of driving image.

[0051] In this embodiment, when the parallel judgment result shows that the two boundaries of the seat belt are parallel, and the continuity judgment result shows that the first seat belt and the second seat belt are continuous, the corresponding seat belt wearing judgment result shows that there is a valid seat belt in the corresponding driving image; otherwise, the corresponding seat belt wearing judgment result shows that there is no valid seat belt in the corresponding driving image.

[0052] It should be noted that, in order to facilitate the subsequent statistical analysis of seat belt wearing judgment results for all driving images, the seat belt wearing judgment result where a valid seat belt is present in the driving image can be recorded as 0, and the seat belt wearing judgment result where a valid seat belt is not present in the driving image can be recorded as 1.

[0053] Step S06: Based on the fact that the sum of the seat belt wearing judgment results of each frame of driving image is greater than the first preset threshold, generate and send alarm information.

[0054] In this embodiment, an alarm message is generated and sent based on the sum of the seat belt wearing judgment results of each frame of driving images being greater than a first preset threshold. This includes: determining an alarm message reminder mode based on the sum of the seat belt wearing judgment results of each frame of driving images and a preset alarm message reminder mode; wherein the alarm message includes at least one of a voice reminder mode, an image reminder mode, a steering wheel indicator light flashing mode, a buzzer reminder mode, and an alarm mode; generating an alarm message according to the determined alarm message reminder mode, and sending the alarm message according to the alarm message reminder mode.

[0055] It should be noted that by summing the seat belt wearing judgment results of each frame of driving images, it is possible to determine whether the driver is wearing a seat belt based on the seat belt wearing judgment results of multiple frames of images, avoiding the situation where the seat belt detection results of a single frame of driving images are inaccurate, thereby reducing the situation of false detection and missed detection.

[0056] In addition, when pre-setting the alarm information reminder mode, the corresponding alarm information reminder mode can be set according to the sum of the seat belt wearing judgment results of each frame of driving images, thereby reminding the driver not to not wear the seat belt for an extended period of time. Specifically, the settings can be configured according to actual reminder needs. For example, if the driver's unbelt-wearing time exceeds the nationally stipulated time, the alarm information reminder mode is set to an alarm mode, facilitating the generation and transmission of alarm information based on the alarm mode. Alternatively, if the driver's unbelt-wearing time does not exceed the nationally stipulated time but is greater than a first preset threshold, the specific alarm information reminder mode is set according to the duration of the unbelt-wearing time. If the driver's unbelt-wearing time is short, the alarm information reminder mode can be set to a voice reminder mode, an image reminder mode, a steering wheel indicator light flashing mode, or a buzzer reminder mode. If the driver's unbelt-wearing time is long, the alarm information reminder mode can be set to at least two of these modes. The duration of the driver's unbelt-wearing time can be determined based on the sum of the seat belt wearing judgment results of each frame of driving images.

[0057] In an optional embodiment, the method further includes:

[0058] S11, acquire driving images within a preset time period;

[0059] S12, based on each frame of driving images, obtain the seat belt, shoulder position and steering wheel area, detect whether the seat belt and shoulder position overlap, and detect whether the overlapping area of ​​the seat belt and steering wheel area is greater than a preset threshold, and obtain the reliability judgment result;

[0060] S13. Based on each frame of driving image, obtain the seat belt mask, and determine whether the two boundaries of the seat belt are parallel based on the seat belt mask, and obtain the parallel judgment result;

[0061] S14, segment the seat belt mask to obtain the first seat belt and the second seat belt corresponding to each frame of driving image, and determine whether the first seat belt and the second seat belt are continuous, and obtain the continuity judgment result;

[0062] S15. Based on the reliability judgment result, parallel judgment result and coherent judgment result, obtain the seat belt wearing judgment result for each frame of driving image;

[0063] S16, count the number of invalid seat belts in each frame of driving image, and send an alarm message if the number is greater than a first preset threshold.

[0064] It should be noted that if the seat belt overlaps with the shoulder area, and the overlap between the seat belt and the steering wheel area exceeds a preset threshold, the reliability judgment result is considered reliable. If the reliability judgment result is reliable, the parallel judgment result shows parallelism, and the coherence judgment result shows coherence, then a valid seat belt exists in the corresponding driving image, and the corresponding seat belt wearing judgment result shows that a seat belt is worn; otherwise, the corresponding seat belt wearing judgment result shows that a seat belt is not worn.

[0065] In an optional embodiment, the method further includes:

[0066] S21, acquire driving images within a preset time period;

[0067] S22, based on each frame of driving image, obtain the seat belt mask, and determine whether the two boundaries of the seat belt are parallel based on the seat belt mask;

[0068] S23, if parallel, then segment the seat belt mask to obtain the first seat belt and the second seat belt corresponding to each frame of driving image, and determine whether the first seat belt and the second seat belt are continuous.

[0069] S24, if the sequence is continuous, determine that there is a valid seat belt in the corresponding driving image; otherwise, determine that there is an invalid seat belt in the corresponding driving image.

[0070] S25, count the number of invalid seat belts in each frame of driving image, and send an alarm message if the number is greater than a first preset threshold.

[0071] In an optional embodiment, the method further includes:

[0072] S31, acquire driving images within a preset time period;

[0073] S32, based on each frame of driving image, obtain the seat belt, shoulder position and steering wheel area, detect whether the seat belt and shoulder position overlap, and detect whether the overlapping area of ​​the seat belt and steering wheel area is greater than a preset threshold.

[0074] S33, if the seat belt overlaps with the shoulder position and the overlapping area is greater than a preset threshold, then obtain the seat belt mask based on each frame of driving image, and determine whether the two boundaries of the seat belt are parallel based on the seat belt mask;

[0075] S34, if parallel, then segment the seat belt mask to obtain the first seat belt and the second seat belt corresponding to each frame of driving image, and determine whether the first seat belt and the second seat belt are continuous.

[0076] S35, if the sequence is continuous, determine that there is a valid seat belt in the corresponding driving image; otherwise, determine that there is an invalid seat belt in the corresponding driving image.

[0077] S36, count the number of invalid seat belts in each frame of driving image, and send an alarm message if the number is greater than a first preset threshold.

[0078] In summary, the embodiments of the present invention determine whether a valid seat belt exists in a corresponding driving image by judging the parallelism of the seat belt boundary within the driving image and the continuity of the first and second seat belts. This avoids situations where seat belt detection accuracy is poor due to obstruction by objects, effectively improving the accuracy of seat belt detection. Furthermore, by statistically analyzing the seat belt judgment results of each frame of the driving image, the inaccuracy of judgment results from a single frame of the driving image is avoided, further improving the accuracy of seat belt detection.

[0079] The seat belt detection device provided by the present invention is described below. The seat belt detection device described below can be referred to in correspondence with the seat belt detection method described above.

[0080] Figure 2 A schematic diagram of a seatbelt detection device is shown. The device includes:

[0081] Image acquisition module 21 acquires driving images within a preset time period;

[0082] The parallel judgment module 22 obtains the seat belt mask based on each frame of driving image, and judges whether the two boundaries of the seat belt are parallel based on the seat belt mask;

[0083] The segmentation module 23 segments the seat belt mask to obtain the first seat belt and the second seat belt corresponding to each frame of driving image; wherein, the first seat belt is the part of the seat belt that contacts the driver's body, and the second seat belt is the part of the seat belt that does not contact the driver's body;

[0084] The continuity judgment module 24 determines whether the first seat belt and the second seat belt are continuous.

[0085] The detection module 25 obtains the seat belt wearing judgment result for each frame of driving image based on the parallel judgment result of the two boundaries of the seat belt and the continuous judgment result of the first seat belt and the second seat belt.

[0086] The control module 26 generates and sends an alarm message based on the fact that the sum of the seat belt wearing judgment results of each frame of driving image is greater than a first preset threshold.

[0087] In this embodiment, the image acquisition module 21 includes a camera unit that acquires a video stream of the driving area; and an image extraction unit that extracts driving images within a preset time period based on the video stream. Specifically, the image extraction unit, based on the video stream, acquires driving images of a certain number of frames before and after the current frame driving image, for example, acquiring five frames before and after the current frame driving image, thereby avoiding the situation where subsequent judgment and detection results based on single-frame driving images are poor. The specific number of driving images acquired can be set according to actual detection needs and is not further limited here. In other embodiments, the image acquisition module 21 includes a single-frame image acquisition unit that acquires multiple consecutively captured driving images within a preset time period.

[0088] It should be noted that the driving images are taken from the driving area within the vehicle. The vehicle can be a car, ship, or airplane used to carry people or goods. The vehicle can be a private car or a commercial vehicle, such as a car-sharing service, ride-hailing service, taxi, bus, school bus, truck, coach, train, subway, and tram.

[0089] In one optional embodiment, a face-tracking camera can be used to acquire driving images of the driving area. The face-tracking camera is positioned in front of the driver's area to capture images of the driving area in the direction of the driver's face. It should be noted that the camera includes at least one of a high-resolution infrared camera, a DMS camera, an OMS camera, a video camera, a video recorder, and electronic devices with a camera. The electronic device can be a mobile terminal device such as a mobile terminal, computer, camera, or tablet computer.

[0090] In an optional embodiment, the device further includes: a reliability judgment module, which determines whether the seat belt is effective based on each frame of driving images. Specifically, the reliability judgment module includes: a face recognition unit, which recognizes a face for each frame of driving images and locates the shoulder based on the recognized face; an overlap detection unit, which detects whether the seat belt overlaps with the shoulder position based on the located shoulder position; a comparison unit, which recognizes the steering wheel area for each frame of driving images and detects whether the overlap area between the seat belt and the steering wheel area is greater than a preset threshold; and a control unit, which controls the parallel judgment module 22 to determine whether the two boundaries of the seat belt are parallel based on the seat belt mask if the seat belt overlaps with the shoulder position and the overlap area is greater than the preset threshold. It should be noted that the preset threshold can be determined according to the driver's body type, the size of the steering wheel, and the area of ​​the seat belt located at the steering wheel position and in contact with the driver's body, and is not further limited here.

[0091] Parallel judgment module 22 includes a mask acquisition submodule and a parallel judgment submodule, wherein:

[0092] The mask acquisition submodule includes: a face recognition unit, which recognizes faces in each frame of the driving image and locates the shoulders based on the recognized faces; a localization unit, which determines the seat belt area based on the located shoulder position; and a segmentation unit, which segments the seat belt within the seat belt area to obtain a seat belt mask. Face recognition facilitates face localization, thereby determining the shoulder position based on the located face position, and further determining the seat belt area based on the shoulder position, thus improving the localization accuracy of the seat belt area and facilitating subsequent seat belt detection.

[0093] The parallelism determination submodule includes: an equation acquisition unit, which, based on the seatbelt mask, uses Hough transform to obtain the linear equations of the two boundaries of the seatbelt; and a parallelism determination unit, which, based on the linear equations of the two boundaries, calculates the slopes corresponding to the two boundaries respectively, and compares the deviation of the slopes of the two boundaries with a first preset range. If the deviation is within the first preset range, then the two boundaries of the seatbelt are parallel. It should be noted that the first preset range can be determined according to actual design requirements; for example, the first preset range can be set to less than 5.

[0094] Furthermore, the equation acquisition unit includes: a detection subunit, which performs edge detection on the seat belt mask to obtain the edge points of the seat belt; a curve acquisition subunit, which, for each edge point, converts all straight lines passing through the edge point into points in polar coordinate space to form a curve corresponding to each edge point; an intersection point acquisition subunit, which obtains multiple intersection points based on the curves corresponding to all edge points; and an equation calculation subunit, which selects two intersection points according to the number of curves passing through the intersection points and calculates the straight line equations in the corresponding rectangular coordinate space for each intersection point to obtain the straight line equations of the two boundaries of the seat belt.

[0095] In an optional embodiment, the parallel determination module 22 further includes a corrosion expansion treatment submodule, which performs corrosion expansion treatment on the seat belt mask. By performing corrosion expansion treatment on the seat belt mask, the seat belt can be separated more easily and its boundaries can be kept smooth, thereby enhancing the seat belt characteristics.

[0096] Specifically, the corrosion and expansion processing submodule includes: a corrosion unit that erodes the seat belt mask to eliminate seat belt boundary points, thereby reducing the size of the seat belt mask and facilitating the elimination of noise points smaller than structural elements; and an expansion unit that expands the eroded seat belt mask to merge all background points in contact with the seat belt into the seat belt, thereby increasing the size of the seat belt mask and filling the voids in the seat belt mask.

[0097] When segmentation module 23 segments the seat belt mask to obtain the first and second seat belts corresponding to each frame of driving image, it can identify the portion of the seat belt in contact with the driver's body and the portion not in contact with the driver's body through image recognition. The identified seat belt portions are then segmented to obtain the first and second seat belts. It should be noted that during seat belt mask recognition, it can also determine whether the boundaries of the first and second seat belts are connected.

[0098] The continuity judgment module 24 includes: an identification unit, which identifies the seat belt mask and obtains the boundary connection result of the first seat belt and the second seat belt; a feature extraction unit, which, if the boundary connection result shows that they are connected, locates the middle area of ​​the first seat belt and the second seat belt, and extracts the first gray value of the first seat belt and the second gray value of the second seat belt located in the middle area; and a comparison unit, which compares the deviation between the first gray value and the second gray value with a second preset range, and if they are within the second preset range, then the first seat belt and the second seat belt are continuous.

[0099] The detection module 25 includes a detection unit that obtains a seatbelt wearing determination result for each frame of the driving image based on the parallelism determination result of the two boundaries of the seatbelt and the continuity determination result of the first and second seatbelts. It should be noted that if the parallelism determination result shows that the two boundaries of the seatbelt are parallel, and the continuity determination result shows that the first and second seatbelts are continuous, then the corresponding seatbelt wearing determination result indicates that a valid seatbelt exists in the corresponding driving image; otherwise, the corresponding seatbelt wearing determination result indicates that a valid seatbelt does not exist in the corresponding driving image.

[0100] To facilitate subsequent statistical analysis of seatbelt wearing judgment results for all driving images, the detection module 25 further includes a marking unit, which records the seatbelt wearing judgment result of a valid seatbelt in the driving image as 0, and records the seatbelt wearing judgment result of a non-valid seatbelt in the driving image as 1.

[0101] The control module 26 includes: a mode selection unit, which determines an alarm information reminder mode based on the sum of seat belt wearing judgment results of each frame of driving images and a preset alarm information reminder mode; wherein the alarm information includes at least one of voice reminder mode, image reminder mode, steering wheel indicator light flashing mode, buzzer reminder mode and alarm mode; and an information prompting unit, which generates alarm information according to the determined alarm information reminder mode and sends the alarm information according to the alarm information reminder mode.

[0102] In an optional embodiment, the device further includes a mode setting module, which is used to set a warning information reminder mode according to the sum of the seat belt wearing judgment results of each frame of driving images, thereby reminding the driver to wear the seat belt and pay attention to driving safety.

[0103] In an optional embodiment, the apparatus further includes:

[0104] The image acquisition module acquires driving images within a preset time period;

[0105] The reliability judgment module acquires the seat belt, shoulder position and steering wheel area based on each frame of driving image, detects whether the seat belt and shoulder position overlap, and detects whether the overlapping area of ​​the seat belt and steering wheel area is greater than a preset threshold, and obtains the reliability judgment result.

[0106] The parallel judgment module obtains the seat belt mask based on each frame of driving image, and judges whether the two boundaries of the seat belt are parallel based on the seat belt mask to obtain the parallel judgment result;

[0107] The coherence judgment module segments the seat belt mask to obtain the first and second seat belts corresponding to each frame of driving image, and judges whether the first and second seat belts are coherent to obtain the coherence judgment result.

[0108] The detection module obtains the seat belt wearing judgment result for each frame of driving image based on the reliability judgment result, parallel judgment result, and continuous judgment result;

[0109] The control module counts the number of invalid seat belts in each frame of the driving image and sends an alarm message if the number exceeds a first preset threshold.

[0110] It should be noted that if the seat belt overlaps with the shoulder area, and the overlap between the seat belt and the steering wheel area exceeds a preset threshold, the reliability judgment result is considered reliable. If the reliability judgment result is reliable, the parallel judgment result shows parallelism, and the coherence judgment result shows coherence, then a valid seat belt exists in the corresponding driving image, and the corresponding seat belt wearing judgment result shows that a seat belt is worn; otherwise, the corresponding seat belt wearing judgment result shows that a seat belt is not worn.

[0111] In an optional embodiment, the method further includes:

[0112] The image acquisition module acquires driving images within a preset time period;

[0113] The parallel judgment module obtains the seat belt mask based on each frame of driving image, and determines whether the two boundaries of the seat belt are parallel based on the seat belt mask;

[0114] The continuity judgment module, if parallel, segments the seat belt mask to obtain the first and second seat belts corresponding to each frame of driving image, and determines whether the first and second seat belts are continuous.

[0115] If the detection module detects a continuous sequence of images, it determines that a valid seatbelt exists within the corresponding driving image; otherwise, it determines that an invalid seatbelt exists within the corresponding driving image.

[0116] The control module counts the number of invalid seat belts in each frame of the driving image and sends an alarm message if the number exceeds a first preset threshold.

[0117] In an optional embodiment, the method further includes:

[0118] The image acquisition module acquires driving images within a preset time period;

[0119] The reliability judgment module acquires the seat belt, shoulder position and steering wheel area based on each frame of driving image, detects whether the seat belt and shoulder position overlap, and detects whether the overlapping area of ​​the seat belt and steering wheel area is greater than a preset threshold.

[0120] The parallel judgment module, if the seat belt overlaps with the shoulder position and the overlapping area is greater than a preset threshold, then obtains the seat belt mask based on each frame of driving image, and judges whether the two boundaries of the seat belt are parallel based on the seat belt mask;

[0121] If the safety belt is parallel, the safety belt mask is segmented to obtain the first and second safety belts corresponding to each frame of driving image, and it is determined whether the first and second safety belts are continuous.

[0122] If the detection module detects a continuous sequence of images, it determines that a valid seatbelt exists within the corresponding driving image; otherwise, it determines that an invalid seatbelt exists within the corresponding driving image.

[0123] The control module counts the number of invalid seat belts in each frame of the driving image and sends an alarm message if the number exceeds a first preset threshold.

[0124] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3 As shown, the electronic device may include a processor 31, a communication interface 32, a memory 33, and a communication bus 34. The processor 31, communication interface 32, and memory 33 communicate with each other via the communication bus 34. The processor 31 can call logical instructions in the memory 33 to execute a seatbelt detection method. This method includes: acquiring driving images within a preset time period; acquiring a seatbelt mask based on each frame of the driving images, and determining whether the two boundaries of the seatbelt are parallel based on the seatbelt mask to obtain a parallelism judgment result; segmenting the seatbelt mask to obtain a first seatbelt and a second seatbelt corresponding to each frame of the driving images; wherein the first seatbelt is the part of the seatbelt that contacts the driver's body, and the second seatbelt is the part of the seatbelt that does not contact the driver's body; determining whether the first seatbelt and the second seatbelt are continuous to obtain a continuity judgment result; obtaining a seatbelt wearing judgment result for each frame of the driving images based on the parallelism judgment result and the continuity judgment result; and generating and sending an alarm message if the sum of the seatbelt wearing judgment results for each frame of the driving images is greater than a first preset threshold.

[0125] Furthermore, the logical instructions in the aforementioned memory 33 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0126] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the seat belt detection method provided by the above methods. The method includes: acquiring driving images within a preset time period; acquiring a seat belt mask based on each frame of driving images, and determining whether the two boundaries of the seat belt are parallel based on the seat belt mask to obtain a parallelism judgment result; segmenting the seat belt mask to obtain a first seat belt and a second seat belt corresponding to each frame of driving images; wherein the first seat belt is the part of the seat belt that contacts the driver's body, and the second seat belt is the part of the seat belt that does not contact the driver's body; determining whether the first seat belt and the second seat belt are continuous to obtain a continuity judgment result; obtaining a seat belt wearing judgment result for each frame of driving images based on the parallelism judgment result and the continuity judgment result; and generating and sending an alarm message based on the sum of the seat belt wearing judgment results for each frame of driving images being greater than a first preset threshold.

[0127] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the seat belt detection method provided by the above methods. The method includes: acquiring driving images within a preset time period; acquiring a seat belt mask based on each frame of driving images, and determining whether the two boundaries of the seat belt are parallel based on the seat belt mask to obtain a parallelism judgment result; segmenting the seat belt mask to obtain a first seat belt and a second seat belt corresponding to each frame of driving images; wherein the first seat belt is the part of the seat belt in contact with the driver's body, and the second seat belt is the part of the seat belt not in contact with the driver's body; determining whether the first seat belt and the second seat belt are continuous to obtain a continuity judgment result; obtaining a seat belt wearing judgment result for each frame of driving images based on the parallelism judgment result and the continuity judgment result; and generating and sending an alarm message based on the sum of the seat belt wearing judgment results of each frame of driving images being greater than a first preset threshold.

[0128] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0129] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting seat belts, characterized in that, include: Acquire driving images within a preset time period; Based on each frame of driving image, a seat belt mask is obtained, and based on the seat belt mask, it is determined whether the two boundaries of the seat belt are parallel, and a parallelism determination result is obtained; The seat belt mask is segmented to obtain a first seat belt and a second seat belt corresponding to each frame of driving image; wherein, the first seat belt is the part of the seat belt that contacts the driver's body, and the second seat belt is the part of the seat belt that does not contact the driver's body; Determine whether the first seat belt and the second seat belt are continuous, and obtain a continuity determination result; Based on the parallel judgment results and the continuous judgment results, the seat belt wearing judgment result for each frame of driving image is obtained; If the sum of the seatbelt wearing judgment results of each frame of driving image is greater than a first preset threshold, an alarm message is generated and sent. The seatbelt mask is segmented to obtain the first and second seatbelts corresponding to each frame of the driving image, including: Image recognition is used to identify the portion of the seat belt that is in contact with the driver's body and the portion that is not in contact with the driver's body. The identified portion of the seat belt is then segmented to obtain the first seat belt and the second seat belt. Based on the parallel judgment results and the continuous judgment results, the seat belt wearing judgment result for each frame of driving image is obtained, including: If the parallel judgment result shows that the two boundaries of the seat belt are parallel, and the continuity judgment result shows that the first seat belt and the second seat belt are continuous, then the corresponding seat belt wearing judgment result shows that there is a valid seat belt in the corresponding driving image; otherwise, the corresponding seat belt wearing judgment result shows that there is no valid seat belt in the corresponding driving image.

2. The seat belt detection method according to claim 1, characterized in that, The step of determining whether the two boundaries of the seat belt are parallel based on the seat belt mask includes: Based on the seat belt mask, the equations of the straight lines of the two boundaries of the seat belt are obtained using the Hough transform. Based on the straight line equations of the two boundaries of the seat belt, the slopes corresponding to the two boundaries are calculated respectively, and the deviation of the slopes corresponding to the two boundaries is compared with a first preset range. If it is within the first preset range, then the two boundaries of the seat belt are parallel.

3. The seat belt detection method according to claim 2, characterized in that, Based on the seatbelt mask, the coordinate equations of the two boundaries of the seatbelt are obtained using the Hough transform, including: Edge detection is performed on the seat belt mask to obtain the edge points of the seat belt; For each edge point, all straight lines passing through the edge point are converted into points in polar coordinate space, forming a curve corresponding to each edge point; Based on the curves corresponding to all edge points, multiple intersection points are obtained; Based on the number of curves passing through the intersection points, two intersection points are selected, and the equations of the straight lines in their corresponding rectangular coordinate spaces are calculated to obtain the straight line equations of the two boundaries of the seat belt.

4. The seat belt detection method according to claim 2, characterized in that, Before obtaining the linear equations of the two boundaries of the seat belt using the Hough transform based on the seat belt mask, the process further includes: subjecting the seat belt mask to corrosion and expansion treatment.

5. The seat belt detection method according to claim 1, characterized in that, The determination of whether the first seat belt and the second seat belt are continuous includes: Identify the seatbelt mask to obtain the boundary connection result between the first seatbelt and the second seatbelt; If the boundary connection result shows that they are connected, then locate the middle area between the first seat belt and the second seat belt, and extract the first gray value of the first seat belt located in the middle area and the second gray value of the second seat belt located in the middle area. The deviation between the first grayscale value and the second grayscale value is compared with a second preset range. If it is within the second preset range, then the first seat belt and the second seat belt are continuous.

6. The seat belt detection method according to claim 1, characterized in that, After acquiring driving images within the preset time period, the process includes: For each frame of driving image, the face is recognized, and the shoulders are located based on the recognized face; Based on the location of the shoulder, the system detects whether the seat belt overlaps with the shoulder position. For each frame of driving image, the steering wheel area is identified, and it is detected whether the overlap between the seat belt and the steering wheel area is greater than a preset threshold. Based on the fact that the seat belt overlaps with the shoulder position and the overlapping area is greater than the preset threshold, it is determined whether the two boundaries of the seat belt are parallel based on the seat belt mask.

7. A seatbelt detection device, characterized in that, include: The image acquisition module acquires driving images within a preset time period; The parallel judgment module obtains a seat belt mask based on each frame of driving image, and determines whether the two boundaries of the seat belt are parallel based on the seat belt mask; The segmentation module segments the seat belt mask to obtain a first seat belt and a second seat belt corresponding to each frame of driving image; wherein, the first seat belt is the part of the seat belt that contacts the driver's body, and the second seat belt is the part of the seat belt that does not contact the driver's body; A continuity determination module determines whether the first seat belt and the second seat belt are continuous; The detection module obtains the seat belt wearing judgment result for each frame of driving image based on the parallel judgment result of the two boundaries of the seat belt and the coherent judgment result of the first seat belt and the second seat belt. The control module generates and sends an alarm message based on the fact that the sum of the seat belt wearing judgment results of each frame of driving image is greater than a first preset threshold. The segmentation module is used for: Image recognition is used to identify the portion of the seat belt that is in contact with the driver's body and the portion that is not in contact with the driver's body. The identified portion of the seat belt is then segmented to obtain the first seat belt and the second seat belt. The detection module is used for: If the parallel judgment result shows that the two boundaries of the seat belt are parallel, and the continuity judgment result shows that the first seat belt and the second seat belt are continuous, then the corresponding seat belt wearing judgment result shows that there is a valid seat belt in the corresponding driving image; otherwise, the corresponding seat belt wearing judgment result shows that there is no valid seat belt in the corresponding driving image.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the seat belt detection method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the seatbelt detection method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the seatbelt detection method as described in any one of claims 1 to 6.

Images