Safety-belt wearing detection method and apparatus, and electronic device and storage medium

Abstract

The present application relates to the technical field of vehicle driving. Disclosed are a safety-belt wearing detection method and apparatus, and an electronic device and a storage medium. The method comprises: acquiring an in-vehicle driving image of a driver and a front-seat passenger; checking the in-vehicle driving image, so as to determine a driver safety-belt area, a front-seat-passenger safety-belt area, driver head key points and front-seat-passenger head key points; on the basis of the driver safety-belt area and the front-seat-passenger safety-belt area, constructing a safety-belt fan-shaped area; and on the basis of the state of overlap between the driver head key points and the safety-belt fan-shaped area and the state of overlap between the front-seat-passenger head key points and the safety-belt fan-shaped area, determining a safety-belt wearing detection result for the driver and the front-seat passenger. The technical solution in the embodiments of the present invention realizes simultaneous detection of the actual safety-belt wearing condition of a driver and a front-seat passenger, thereby ensuring the safety of the driver and the front-seat passenger during vehicle travelling, and also improving the detection efficiency of safety-belt wearing detection.

Description

Seatbelt wearing detection methods, devices, electronic equipment and storage media Technical Field

[0001] This invention relates to the field of vehicle driving technology, and in particular to a method, device, electronic device, and storage medium for detecting seat belt wearing. Background Technology

[0002] With increased awareness of traffic safety, the use of seat belts has become an indispensable part of vehicle driving safety.

[0003] Existing vehicles are typically equipped with seatbelt reminder systems, which mainly rely on mechanical sensors to detect whether the seatbelt is inserted into the buckle. However, this method of seatbelt detection cannot guarantee that the driver is actually wearing the seatbelt, and safety during vehicle operation remains compromised; furthermore, the detection efficiency of existing seatbelt detection systems needs improvement.

[0004] Summary of the Invention

[0005] This invention provides a seat belt wearing detection method, device, electronic device, and storage medium, which realizes the simultaneous detection of the actual seat belt wearing status of the driver and front passenger, ensuring the safety of the driver and front passenger during vehicle operation, and also improving the detection efficiency of seat belt wearing detection.

[0006] According to one aspect of the present invention, a seatbelt wearing detection method is provided, the method comprising:

[0007] Acquire in-vehicle driving images of the driver and front passenger;

[0008] The in-vehicle driving image is detected to determine the driver's seat belt area, the passenger's seat belt area, the key head points of the driver and the passenger; wherein, the driver's seat belt area is the seat belt area that crosses the upper torso of the driver; the passenger's seat belt area is the seat belt area that crosses the upper torso of the passenger.

[0009] Construct a seatbelt sector area based on the driver's seatbelt area and the passenger's seatbelt area;

[0010] The seatbelt wearing detection results for the driver and passenger are determined based on the overlap between the key head points of the driver's head and the fan-shaped area of ​​the seatbelt, and the overlap between the key head points of the passenger's head and the fan-shaped area of ​​the seatbelt.

[0011] According to another aspect of the present invention, a seatbelt wearing detection device is provided, the device comprising:

[0012] The in-vehicle driving image acquisition module is used to acquire in-vehicle driving images of the driver and front passenger.

[0013] The seat belt area and key point detection module is used to detect the in-vehicle driving image and determine the driver's seat belt area, the passenger's seat belt area, the driver's head key point, and the passenger's head key point; wherein, the driver's seat belt area is the seat belt area that crosses the upper torso of the driver; the passenger's seat belt area is the seat belt area that crosses the upper torso of the passenger;

[0014] A seatbelt sector area construction module is used to construct a seatbelt sector area based on the driver's seatbelt area and the passenger's seatbelt area;

[0015] The seatbelt wearing detection module is used to determine the seatbelt wearing detection results of the driver and passenger based on the overlap between the key head point of the driver and the fan-shaped area of ​​the seatbelt, and the overlap between the key head point of the passenger and the fan-shaped area of ​​the seatbelt.

[0016] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0017] At least one processor; and

[0018] A memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the seat belt wearing detection method according to any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the seat belt wearing detection method according to any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the seat belt wearing detection method according to any embodiment of the present invention.

[0022] The technical solution of this invention detects in-vehicle driving images to determine the driver's seatbelt area, the passenger's seatbelt area, the driver's head key points, and the passenger's head key points. Based on the driver's and passenger's seatbelt areas, a seatbelt sector area is constructed. Considering the overlap between the driver's and passenger's head key points and the seatbelt sector area, and taking into account the geometric relationship between the seatbelt area and the head key points, the system can detect the seatbelt wearing status of both drivers and passengers. This improves the accuracy of seatbelt wearing detection and ensures the safety of vehicle driving. Furthermore, compared to detecting seatbelt wearing separately for each driver, simultaneously judging the seatbelt wearing status of both drivers and passengers further improves the detection efficiency.

[0023] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 is a flowchart of a seat belt wearing detection method according to Embodiment 1 of the present invention;

[0026] Figure 2 is a flowchart of a seat belt wearing detection method according to Embodiment 2 of the present invention;

[0027] Figure 3 is a flowchart of a seat belt wearing detection method according to Embodiment 2 of the present invention;

[0028] Figure 4 is a schematic diagram of seat belt area detection provided according to Embodiment 2 of the present invention;

[0029] Figure 5 is a schematic diagram of constructing a seat belt sector area according to Embodiment 2 of the present invention;

[0030] Figure 6 is a schematic diagram of the key eye points within the fan-shaped area of ​​the seat belt according to Embodiment 2 of the present invention;

[0031] Figure 7 is a schematic diagram of an incorrect seatbelt wearing posture according to Embodiment 2 of the present invention;

[0032] Figure 8 is a structural schematic diagram of a seat belt wearing detection device provided according to Embodiment 3 of the present invention;

[0033] Figure 9 is a schematic diagram of the structure of an electronic device that implements the seat belt wearing detection method of the present invention. Detailed Implementation

[0034] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0035] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0036] Example 1

[0037] Figure 1 is a flowchart of a seatbelt wearing detection method provided in Embodiment 1 of the present invention. This embodiment of the present invention is applicable to situations where the seatbelt wearing status of the driver and front passenger is jointly detected. The method can be executed by a seatbelt wearing detection device, which can be implemented in hardware and / or software. This seatbelt wearing detection device can be configured in an electronic device that carries the seatbelt wearing detection function, such as an in-vehicle terminal.

[0038] The seatbelt wearing detection method shown in Figure 1 includes:

[0039] S110: Obtain the in-vehicle driving images of the driver and passenger.

[0040] The driver and front passenger are included. The in-vehicle driving image can be an image of the driver and front passenger's interior during vehicle operation. Based on the in-vehicle driving image, the seatbelt wearing status of the driver and front passenger can be detected. Optionally, the in-vehicle driving image can include frontal or side images of the driver and front passenger. Preferably, the in-vehicle driving image is a frontal image of the driver and front passenger.

[0041] Specifically, an image acquisition device can be used to capture in-vehicle driving images of the driver and front passenger. For example, the image acquisition device can be a high-resolution camera, such as a DMS camera (Driver Monitoring System Camera). The installation location of the image acquisition device can include the A-pillar inside the vehicle, near the reading lights, or other forward locations where images of the upper torso and abdominal areas of the driver and front passenger can be detected. Optionally, considering different lighting conditions, the image acquisition device can be equipped with adaptive exposure adjustment to ensure clear in-vehicle driving images are obtained in different environments such as sunlight, shadows, or nighttime.

[0042] S120. Detect the in-vehicle driving image to determine the driver's seat belt area, the passenger's seat belt area, the driver's head key point, and the passenger's head key point.

[0043] The driver's seatbelt area can be the area across the driver's upper torso. The passenger's seatbelt area can be the area across the passenger's upper torso. The driver's seatbelt area can be used to identify the seatbelt worn by the driver's upper torso. The passenger's seatbelt area can be used to identify the seatbelt worn by the passenger's upper torso. By selecting the driver's and passenger's seatbelt areas diagonally across the upper torsos of both drivers for seatbelt wearing detection, and excluding the area across the abdomen of both drivers during the detection process, interference from the seatbelt wearing detection by this area can be avoided. The driver's head key points can be used to characterize the driver's head position. The driver's head key points can be used to identify the driver's head area. The passenger's head key points can be used to characterize the passenger's head position. The passenger's head key points can be used to identify the passenger's head area. The number of driver's head key points (or passenger's head key points) can be at least one.

[0044] In an optional embodiment of the present invention, the key points of the driver's head include the key points of the driver's eyes, the key points of the driver's nose, the key points of the driver's mouth, or the center point of the driver's head; the key points of the passenger's head include the key points of the passenger's eyes, the key points of the passenger's nose, the key points of the passenger's mouth, or the center point of the passenger's head.

[0045] The driver's eye keypoint can be used to identify the position of the driver's eyes. For example, the driver's eye keypoint can be the position of the driver's two eyeballs. The driver's nose keypoint can be used to identify the position of the driver's nose. For example, the driver's nose keypoint can be the position of the driver's nose tip. The driver's mouth keypoint can be used to identify the position of the driver's mouth. For example, the driver's mouth keypoint can be the center position of the driver's mouth. The driver's head center point can be used to identify the position of the driver's head. For example, the driver's head center point can be the geometric center position of the driver's face. The passenger's eye keypoint can be used to identify the position of the passenger's eyes. For example, the passenger's eye keypoint can be the position of the passenger's two eyeballs. The passenger's nose keypoint can be used to identify the position of the passenger's nose. For example, the passenger's nose keypoint can be the position of the passenger's nose tip. The passenger's mouth keypoint can be used to identify the position of the passenger's mouth. For example, the passenger's mouth keypoint can be the center position of the passenger's mouth. The front passenger head center point can be used to identify the front passenger's head position. For example, the front passenger head center point can be the geometric center of the front passenger's face.

[0046] This solution specifies the key points of the driver's head as the key points of the driver's eyes, nose, mouth, or the center point of the driver's head, and specifies the key points of the passenger's head as the key points of the passenger's eyes, nose, mouth, or the center point of the passenger's head. By using the typical key points of the head of the driver or passenger, this solution can both realize the detection of seat belt wearing for both the driver and passenger and improve the efficiency of the seat belt wearing detection.

[0047] Specifically, image recognition technology can be used to identify in-vehicle driving images and detect the driver's seat belt area, the passenger's seat belt area, the driver's head key points, and the passenger's head key points.

[0048] For example, a pre-trained seatbelt region detection model can be used to detect in-vehicle driving images and output the driver's seatbelt region and the passenger's seatbelt region. The input data for the seatbelt region detection model can be in-vehicle driving images, and the output can be the driver's seatbelt region and the passenger's seatbelt region. The driver's seatbelt region and the passenger's seatbelt region can be single-layer binary images, i.e., seatbelt region maps. In the image, pixels with a value of 0 represent non-seatbelt areas; pixels with a value of 1 represent seatbelt areas. The seatbelt region detection model can be trained using supervised training methods. The training samples for the seatbelt region detection model can include in-vehicle driving image samples, driver's seatbelt region samples, and passenger's seatbelt region samples. The in-vehicle driving image samples can be in-vehicle images of the driver and passenger at historical moments during vehicle driving. For example, the in-vehicle driving image samples can be frontal images of the driver and passenger. The historical driver can be the driver in the in-vehicle driving image samples. The historical passenger can be the passenger in the in-vehicle driving image samples. The driver's seatbelt region sample can be the pre-annotated driver's seatbelt area in the in-vehicle driving image sample. The passenger's seatbelt region sample can be the pre-annotated passenger's seatbelt area in the in-vehicle driving image sample. The seatbelt region detection model can be a semantic segmentation model. For example, the semantic segmentation model can be HRNET (a deep neural network model for image recognition tasks).

[0049] For example, a pre-trained head keypoint detection model can be used to detect in-vehicle driving images and output the head keypoints of the driver and passenger. The input data for the head keypoint detection model can be in-vehicle driving images, and the output can be the head keypoints of the driver and passenger. For instance, the driver's (or passenger's) head keypoints can be at least two layers of head keypoint heatmaps for the driver (or passenger), representing the probability map of the head keypoint positions predicted by the head keypoint detection model. The head keypoint detection model can be trained using supervised training. The training samples for the head keypoint detection model can include in-vehicle driving image samples, historical driver's head keypoint samples, and historical passenger's head keypoint samples. The in-vehicle driving image samples can be in-vehicle images of the driver and passenger at historical moments during vehicle driving. The historical driver's head keypoint samples can be the head key positions of the historical driver pre-annotated in the in-vehicle driving image samples. Historical co-driver head keypoint samples can be pre-annotated key head positions of historical co-driver users in in-vehicle driving image samples. For example, historical driver (or co-driver) head keypoint samples can be the coordinates of body key points of historical driver (or co-driver) users in in-vehicle driving images. The head keypoint detection model can be a convolutional neural network model or a deep learning model. For example, the head keypoint model can be HRNET (a deep neural network model for image recognition tasks) or Center Net (a center point detection model based on convolutional neural networks).

[0050] S130. Construct a seatbelt sector area based on the driver's seatbelt area and the passenger's seatbelt area.

[0051] The seatbelt sector can be used to characterize the overlap between the seatbelt that crosses the upper torso of the driver and the seatbelt that crosses the upper torso of the passenger. The seatbelt sector must encompass both the driver's and passenger's seatbelt areas while minimizing the overall size of the sector. This ensures accurate detection of seatbelt wearing status for both drivers and passengers simultaneously.

[0052] Specifically, a straight line can be fitted to the outer edge of the driver's seatbelt area to obtain the fitted driver's seatbelt outer edge line. A straight line can also be fitted to the outer edge of the passenger's seatbelt area to obtain the fitted passenger's seatbelt outer edge line. At least one sector-shaped region can be constructed using the driver's and passenger's outer edge lines. The sector-shaped region containing both the driver's and passenger's seatbelt areas can be identified as candidate sector-shaped regions. These candidate sector-shaped regions can be compared, and the one with the smallest area can be selected as the seatbelt sector-shaped region. The candidate sector-shaped regions are used to filter the seatbelt sector-shaped region. The candidate sector-shaped regions include both the driver's and passenger's seatbelt areas. There can be multiple candidate sector-shaped regions. The sector-shaped areas containing the driver's seat belt area and the passenger's seat belt area are identified as candidate sector-shaped areas. The seat belt sector-shaped area is then selected based on these candidate sector-shaped areas to ensure the comprehensiveness of the seat belt information contained within each sector-shaped area. The smallest candidate sector-shaped area is selected as the seat belt sector-shaped area to avoid interference from other areas outside the seat belt area for seat belt wearing detection. This ensures the accuracy of seat belt wearing detection for both driver and passenger.

[0053] S140. Based on the overlap between the key head point of the driver's seat and the fan-shaped area of ​​the seat belt, and the overlap between the key head point of the passenger's seat and the fan-shaped area of ​​the seat belt, determine the seat belt wearing test results for the driver and passenger.

[0054] The overlap between the driver's head landmark and the seatbelt sector area can be used to characterize whether the driver's head landmark is covered by the seatbelt sector area. Similarly, the overlap between the passenger's head landmark and the seatbelt sector area can be used to characterize whether the passenger's head landmark is covered by the seatbelt sector area. When both the driver and passenger are wearing seatbelts, the seatbelt sector area formed by the driver's and passenger's seatbelt areas can cover both the driver's and passenger's head landmarks. However, when either the driver or passenger is not wearing a seatbelt, the seatbelt sector area formed by the driver's and passenger's seatbelt areas may not completely cover both the driver's and passenger's head landmarks. Therefore, the seatbelt wearing status of both drivers and passengers can be simultaneously detected based on the overlap between the driver's and passenger's head landmarks and the seatbelt sector area.

[0055] Seatbelt wearing test results can be used to uniformly characterize the seatbelt wearing status of the driver and front passenger. Optionally, the seatbelt wearing test results can include "normal seatbelt wearing" and "abnormal seatbelt wearing." "Normal seatbelt wearing" can indicate that both the driver and front passenger are wearing their seatbelts correctly. For example, both drivers and front passenger are wearing seatbelts and / or their seatbelt wearing posture is correct. "Abnormal seatbelt wearing" can indicate that either the driver or front passenger is not wearing their seatbelt correctly. For example, either the driver or front passenger is not wearing a seatbelt or their seatbelt wearing posture is incorrect.

[0056] Specifically, the overlap between the driver's head landmark and the seatbelt's fan-shaped area can be detected. If both the driver's and passenger's head landmarks overlap with the seatbelt's fan-shaped area, it can be determined that both drivers and passengers are wearing their seatbelts correctly. If neither the driver's nor the passenger's head landmarks overlap with the seatbelt's fan-shaped area, it can be determined that both drivers and passengers are not wearing their seatbelts correctly.

[0057] In an optional embodiment of the present invention, before determining the seatbelt wearing detection results for both the driver and passenger based on the overlap between the driver's head key point and the seatbelt sector area, and the passenger's head key point and the seatbelt sector area, the method further includes: detecting the driver's seatbelt area and the passenger's seatbelt area; when both the driver's and passenger's seatbelt areas exist, determining the driver's seatbelt wearing status detection result based on the area of ​​the driver's seatbelt area; and determining the passenger's seatbelt wearing status detection result based on the area of ​​the passenger's seatbelt area; determining the seatbelt wearing detection results for both the driver and passenger based on the overlap between the driver's head key point and the seatbelt sector area, and the passenger's head key point and the seatbelt sector area, includes: when both the driver and passenger are wearing seatbelts, determining the seatbelt wearing posture detection results for both the driver and passenger based on the overlap between the driver's head key point and the seatbelt sector area, and the passenger's head key point and the seatbelt sector area.

[0058] Seatbelt wearing detection results can include seatbelt wearing status and seatbelt wearing posture. Seatbelt wearing status indicates whether the driver or front passenger is wearing a seatbelt. Seatbelt wearing status can be categorized as "wearing a seatbelt" or "not wearing a seatbelt." Seatbelt wearing posture indicates whether the driver and front passenger are wearing their seatbelts correctly. Seatbelt wearing posture can be categorized as "correct posture" or "incorrect posture." The presence of both driver and front passenger seatbelt areas can be understood as the detection of a seatbelt diagonally across the driver's and front passenger's torsos. The area of ​​the driver's seatbelt area can be defined as the area of ​​the seatbelt diagonally across the driver's torso in the in-vehicle driving image. The area of ​​the front passenger seatbelt area can also be defined as the area of ​​the seatbelt diagonally across the front passenger's torso in the in-vehicle driving image. Optionally, the area of ​​the driver's seatbelt area (or passenger's seatbelt area) may include the area occupied by the driver's seatbelt area (or passenger's seatbelt area) in the in-vehicle driving image, or the ratio between the area occupied by the driver's seatbelt area (or passenger's seatbelt area) in the in-vehicle driving image and the area of ​​the in-vehicle driving image. The area of ​​the driver's seatbelt area (or passenger's seatbelt area) can be used to further detect the area of ​​the diagonally crossed seatbelt on the upper torso of the driver (or passenger), thereby further determining whether the driver (or passenger) is wearing a seatbelt. Based on the detection that both the driver and passenger are wearing seatbelts, the seatbelt wearing posture of the driver and passenger can be jointly detected based on the overlap between the driver's head key points and the seatbelt fan-shaped area, and the overlap between the passenger's head key points and the seatbelt fan-shaped area.

[0059] When both the driver and passenger are wearing their seatbelts correctly, the fan-shaped area formed by their respective seatbelt regions can cover the critical head points of both drivers and passengers. However, if either driver or passenger is wearing their seatbelt incorrectly, the fan-shaped area cannot fully cover the critical head points of both drivers and passengers. Therefore, the seatbelt wearing posture of both drivers and passengers can be simultaneously detected based on the overlap between the driver's and passenger's critical head points and the fan-shaped area.

[0060] Specifically, the system can detect the driver's seatbelt area and the passenger's seatbelt area. When both areas exist, the area of ​​the driver's seatbelt area can be compared to a reference seatbelt area. If the area of ​​the driver's seatbelt area is greater than or equal to the reference area, the driver is confirmed to be wearing a seatbelt; if the area of ​​the driver's seatbelt area is smaller than the reference area, the driver is confirmed not to be wearing a seatbelt. Similarly, the area of ​​the passenger's seatbelt area can be compared to a reference area. If the area of ​​the passenger's seatbelt area is greater than or equal to the reference area, the passenger is confirmed to be wearing a seatbelt; if the area of ​​the passenger's seatbelt area is smaller than the reference area, the passenger is confirmed not to be wearing a seatbelt. When both the driver and passenger are wearing seatbelts, the overlap between the driver's head control point and the seatbelt sector area can be detected. When the key point of the driver's head overlaps with the sector area of ​​the seat belt, and the key point of the passenger's head overlaps with the sector area of ​​the seat belt, it can be determined that the seat belt wearing posture of both the driver and passenger is correct; when either the key point of the driver's head or the key point of the passenger's head does not overlap with the sector area of ​​the seat belt, it can be determined that the seat belt wearing posture of both the driver and passenger is incorrect.

[0061] Optionally, if the driver's seatbelt area is not present, it is determined that the driver is not wearing a seatbelt; if the front passenger seatbelt area is not present, it is determined that the front passenger is not wearing a seatbelt. If neither the driver's nor the front passenger seatbelt area is present, it is determined that neither the driver nor the front passenger is wearing a seatbelt.

[0062] This solution concretizes seatbelt wearing detection results into seatbelt wearing status and seatbelt wearing posture. Based on the presence of the driver's seatbelt area and the passenger's seatbelt area, it detects the seatbelt wearing status of both the driver and passenger. Based on the area of ​​the driver's seatbelt area and the passenger's seatbelt area, it further improves the accuracy of seatbelt wearing status detection. Based on the overlap between the driver's head key point and the seatbelt sector area, and the overlap between the passenger's head key point and the seatbelt sector area, it detects the seatbelt wearing posture of both the driver and passenger. Thus, it can achieve two-stage detection of seatbelt wearing status and posture for both the driver and passenger, further improving the efficiency and accuracy of seatbelt wearing detection for both drivers and passenger.

[0063] The technical solution of this invention detects in-vehicle driving images to determine the driver's seatbelt area, the passenger's seatbelt area, the driver's head key points, and the passenger's head key points. Based on the driver's and passenger's seatbelt areas, a seatbelt sector area is constructed. Considering the overlap between the driver's and passenger's head key points and the seatbelt sector area, and taking into account the geometric relationship between the seatbelt area and the head key points, the system can detect the seatbelt wearing status of both drivers and passengers. This improves the accuracy of seatbelt wearing detection and ensures the safety of vehicle driving. Furthermore, compared to detecting seatbelt wearing separately for each driver, simultaneously judging the seatbelt wearing status of both drivers and passengers further improves the detection efficiency.

[0064] In an optional embodiment of the present invention, after determining the seat belt wearing detection results of the driver and passenger based on the overlap state of the driver's head key point and the seat belt sector area, and the overlap state of the passenger's head key point and the seat belt sector area, the method further includes: determining seat belt wearing reminder information based on the seat belt wearing detection results of the driver and passenger, and feeding it back to the driver and passenger.

[0065] The seatbelt wearing reminder message is used to remind the driver and front passenger to adjust their seatbelt wearing status. Optionally, the seatbelt wearing reminder message may include seatbelt wearing status reminder and / or seatbelt wearing posture reminder. Optionally, the seatbelt wearing reminder message may include text information, voice information, image display information, or audio-visual information, etc. For example, the seatbelt wearing status reminder message may be a voice message saying "Please wear your seatbelt~"; the seatbelt wearing posture reminder message may be a voice message saying "Please adjust your seatbelt wearing posture~".

[0066] Optionally, when it is determined that the seat belt wearing of the driver and passenger is abnormal based on the overlap between the key head point of the driver's head and the fan-shaped area of ​​the seat belt, and the overlap between the key head point of the passenger's head and the fan-shaped area of ​​the seat belt, the seat belt wearing reminder information can be determined as seat belt wearing status reminder information and / or seat belt wearing posture reminder information.

[0067] Optionally, if it is determined that the seat belts of both the driver and passenger are being worn correctly based on the overlap between the key head points of the driver's head and the fan-shaped area of ​​the seat belt, and the overlap between the key head points of the passenger's head and the fan-shaped area of ​​the seat belt, then the seat belt wearing reminder information can be set to empty.

[0068] This solution determines the seatbelt wearing status of both the driver and passenger by analyzing the overlap between the driver's and passenger's head points and the seatbelt's fan-shaped area. Based on these results, a seatbelt wearing reminder is sent to both drivers and passenger, providing timely reminders about their seatbelt wearing status and further improving their safety during driving.

[0069] Example 2

[0070] Figure 2 is a flowchart of a seatbelt wearing detection method provided in Embodiment 2 of the present invention. Based on the above embodiments, this embodiment of the present invention specifies "constructing a seatbelt sector area based on the driver's seatbelt area and the passenger's seatbelt area" as "determining the outer edge tangent of the driver's seatbelt based on the driver's seatbelt area, and determining the outer edge tangent of the passenger's seatbelt based on the passenger's seatbelt area; determining the center of the seatbelt sector area based on the intersection of the outer edge tangents of the driver's seatbelt and the passenger's seatbelt; constructing the seatbelt sector area based on the outer edge tangents of the driver's seatbelt, the passenger's seatbelt, and the center of the seatbelt sector area." The introduction of the outer edge tangents of the driver's seatbelt and the passenger's seatbelt improves the efficiency of constructing the seatbelt sector area, thereby improving the efficiency of seatbelt wearing detection for both driver and passenger. It should be noted that parts not described in detail in this embodiment of the present invention can be found in the descriptions of other embodiments.

[0071] The seatbelt wearing detection method shown in Figure 2 includes:

[0072] S210: Obtain the in-vehicle driving images of the driver and passenger.

[0073] S220: Detect the in-vehicle driving image to determine the driver's seat belt area, the passenger's seat belt area, the driver's head key points, and the passenger's head key points.

[0074] The driver's seatbelt area is the area of ​​the seatbelt that crosses the upper torso of the driver; the passenger's seatbelt area is the area of ​​the seatbelt that crosses the upper torso of the passenger.

[0075] S230. Determine the tangent line of the outer edge of the driver's seat belt based on the driver's seat belt area, and determine the tangent line of the outer edge of the passenger's seat belt based on the passenger's seat belt area.

[0076] The fan-shaped area formed by the driver's seatbelt area and the passenger's seatbelt area is the associated area between these two areas. The area outside this fan-shaped area can be considered the unrelated area between the driver's and passenger's seatbelt areas, i.e., the area where seatbelt wearing detection cannot be performed based on the head-mounted key points. The tangent line of the outer edge of the driver's seatbelt can serve as the boundary between the driver's side seatbelt fan-shaped area and the unrelated area for seatbelt wearing detection. Similarly, the tangent line of the outer edge of the passenger's seatbelt can serve as the boundary between the passenger's side seatbelt fan-shaped area and the unrelated area for seatbelt wearing detection. Compared to fitting straight lines to the outer edges of the driver's and passenger's seatbelt areas, constructing the seatbelt fan-shaped area based on the tangent lines of the driver's and passenger's seatbelt outer edges is simpler and faster.

[0077] Specifically, a tangent can be drawn along the outer edge of the driver's seatbelt area to obtain at least one tangent line along the outer edge of the driver's seatbelt. Similarly, a tangent can be drawn along the outer edge of the passenger's seatbelt area to obtain at least one tangent line along the outer edge of the passenger's seatbelt.

[0078] In an optional embodiment of the present invention, determining the tangent line of the outer edge of the driver's seat belt based on the driver's seat belt area includes: determining the outermost edge tangent point of the driver's seat belt based on the driver's seat belt area; and determining the tangent line of the outer edge of the driver's seat belt based on the tangent line between the outermost edge tangent point of the driver's seat belt and the outer edge of the driver's seat belt area.

[0079] The outermost tangent point of the driver's seatbelt can be the outermost point of tangency on the outer edge of the driver's seatbelt area. The outermost tangent point of the driver's seatbelt area can be used to characterize the outermost boundary of the driver's seatbelt area.

[0080] Specifically, points along the outer edge of the driver's seatbelt area can be compared, and the outermost point can be determined as the tangent point of the outermost edge of the driver's seatbelt. A tangent line can then be drawn along this tangent point to the outer edge of the driver's seatbelt area to obtain the tangent line of the driver's seatbelt's outer edge.

[0081] This solution introduces the outermost edge tangent point of the driver's seat belt. Compared to determining the corresponding outer edge tangent line of the driver's seat belt based on all points on the outer edge of the driver's seat belt area, and then constructing the seat belt sector area, this solution can directly determine the outer edge tangent line of the driver's seat belt based on the outermost edge tangent point. This reduces the computational workload of determining the outer edge tangent line of the driver's seat belt and constructing the seat belt sector area, thereby improving the efficiency of determining the outer edge tangent line of the driver's seat belt.

[0082] Accordingly, based on the passenger seat belt area, the outer edge tangent of the passenger seat belt is determined, including: based on the passenger seat belt area, determining the outermost edge tangent point of the passenger seat belt; and based on the tangent between the outermost edge tangent point of the passenger seat belt and the outer edge of the passenger seat belt area, determining the outer edge tangent of the passenger seat belt.

[0083] The outermost tangent point of the passenger seatbelt can be the outermost point of tangency on the outer edge of the passenger seatbelt area. The outermost tangent point of the passenger seatbelt area can be used to characterize the outermost boundary of the passenger seatbelt area.

[0084] Specifically, points along the outer edge of the passenger seatbelt area can be compared, and the outermost point can be determined as the tangent point of the outermost edge of the passenger seatbelt. A tangent line can then be drawn along this tangent point to the outer edge of the passenger seatbelt area to obtain the tangent line of the outermost edge of the passenger seatbelt.

[0085] By introducing the outermost tangent point of the passenger seat belt, compared to determining the corresponding outer edge tangent line of the passenger seat belt based on all points on the outer edge of the passenger seat belt area, and then constructing the seat belt fan-shaped area, this solution can directly determine the outer edge tangent line of the passenger seat belt based on the outermost tangent point. This reduces the computational workload of determining the outer edge tangent line of the passenger seat belt and constructing the seat belt fan-shaped area, thus improving the efficiency of determining the outer edge tangent line of the passenger seat belt.

[0086] In an optional embodiment of the present invention, determining the outer edge tangent of the driver's seat belt based on the tangent between the outermost edge tangent point of the driver's seat belt and the outer edge of the driver's seat belt area includes: detecting at least two tangents between the outermost edge tangent point of the driver's seat belt and the outer edge of the driver's seat belt area to obtain at least two candidate driver's seat belt tangents; obtaining the outer edge tangent of the passenger's seat belt; detecting the intersection angle between each candidate driver's seat belt tangent and the outer edge tangent of the passenger's seat belt; and taking the candidate driver's seat belt tangent corresponding to the minimum intersection angle as the outer edge tangent of the driver's seat belt.

[0087] The number of tangents between the outermost edge of the driver's seatbelt and the outer edge of the driver's seatbelt area can be at least two. This can be understood as at least two outermost edge tangents, or, due to the shape of the outer edge of the driver's seatbelt area, one outermost edge tangent corresponds to at least two tangents. Alternate driver's seatbelt tangents can be used to filter the outer edge tangents of the driver's seatbelt. The intersection angle can be the angle between each alternate driver's seatbelt tangent and the outer edge tangent of the passenger's seatbelt. The intersection angle can be used to characterize the size of the fan-shaped area formed by the alternate driver's seatbelt tangents and the outer edge tangent of the passenger's seatbelt. This can be understood as follows: the larger the intersection angle, the larger the fan-shaped area formed by the alternate driver's seatbelt tangents and the outer edge tangent of the passenger's seatbelt; the smaller the intersection angle, the smaller the fan-shaped area formed by the alternate driver's seatbelt tangents and the outer edge tangent of the passenger's seatbelt. The driver's side alternative tangent line corresponding to the minimum intersection angle can be understood as the smallest driver's side alternative tangent line in the resulting fan-shaped area, which is also the driver's side alternative tangent line corresponding to the seat belt fan-shaped area.

[0088] Specifically, at least two tangent lines can be detected between the outermost edge of the driver's seatbelt and the outer edge of the driver's seatbelt area, resulting in at least two candidate driver's seatbelt tangent lines. The outer edge tangent line of the passenger's seatbelt can also be obtained. The intersection angle between each candidate driver's seatbelt tangent line and the outer edge tangent line of the passenger's seatbelt can be detected. The candidate driver's seatbelt tangent line corresponding to the minimum intersection angle can be used as the outer edge tangent line of the driver's seatbelt.

[0089] Optionally, at least two tangents between the outermost edge of the passenger seatbelt and the outer edge of the passenger seatbelt area can be detected to obtain at least two candidate passenger seatbelt tangents. The outer edge tangent of the driver's seatbelt can be obtained. The intersection angle between each candidate passenger seatbelt tangent and the outer edge tangent of the driver's seatbelt can be detected. The candidate passenger seatbelt tangent corresponding to the minimum intersection angle is taken as the outer edge tangent of the passenger seatbelt. The number of tangents between the outermost edge of the passenger seatbelt and the outer edge of the passenger seatbelt area can be at least two. This can be understood as the number of outermost edge tangents of the passenger seatbelt being at least two, or, due to the shape of the outer edge of the passenger seatbelt area, one outermost edge tangent of the passenger seatbelt corresponding to at least two tangents. The candidate passenger seatbelt tangents can be used to filter the outer edge tangents of the passenger seatbelt. The intersection angle can be the angle between the candidate passenger seatbelt tangent and the outer edge tangent of the driver's seatbelt. The intersection angle can be used to characterize the size of the fan-shaped area formed by the candidate passenger seatbelt tangent and the outer edge tangent of the driver's seatbelt. This can be understood as follows: the larger the intersection angle, the larger the fan-shaped area formed by the passenger-side alternative tangent line and the driver's seatbelt outer edge tangent line; the smaller the intersection angle, the smaller the fan-shaped area formed by the passenger-side alternative tangent line and the driver's seatbelt outer edge tangent line. The passenger-side alternative tangent line corresponding to the minimum intersection angle can be understood as the smallest passenger-side alternative tangent line forming the fan-shaped area, that is, the passenger-side alternative tangent line corresponding to the seatbelt fan-shaped area.

[0090] Optionally, at least two tangents between the outermost edge of the driver's seatbelt and the outer edge of the driver's seatbelt area can be detected to obtain at least two alternative driver's seatbelt tangents. At least two tangents between the outermost edge of the passenger's seatbelt and the outer edge of the passenger's seatbelt area can be detected to obtain at least two alternative passenger's seatbelt tangents. The intersection angle between each alternative driver's seatbelt tangent and each alternative passenger's seatbelt tangent can be detected. The alternative driver's seatbelt tangent and the alternative passenger's seatbelt tangent corresponding to the minimum intersection angle are taken as the outer edge tangents of the driver's seatbelt and the passenger's seatbelt. The intersection angle can be the angle between each alternative driver's seatbelt tangent and each alternative passenger's seatbelt tangent. The intersection angle can be used to characterize the size of the fan-shaped area formed by the alternative driver's seatbelt tangent and the alternative passenger's seatbelt tangent. It can be understood that the larger the intersection angle, the larger the fan-shaped area formed by the alternative driver's seatbelt tangent and the alternative passenger's seatbelt tangent; the smaller the intersection angle, the smaller the fan-shaped area formed by the alternative driver's seatbelt tangent and the alternative passenger's seatbelt tangent. The driver's alternative tangent and passenger alternative tangent corresponding to the minimum intersection angle can be understood as the smallest driver's alternative tangent and passenger alternative tangent in the resulting fan-shaped area, which is also the driver's alternative tangent and passenger alternative tangent corresponding to the seat belt fan-shaped area.

[0091] Optionally, at least two tangent lines between the outermost edge of the driver's seatbelt and the outer edge of the driver's seatbelt area can be detected to obtain at least two alternative driver's seatbelt tangent lines. At least two tangent lines between the outermost edge of the passenger's seatbelt and the outer edge of the passenger's seatbelt area can be detected to obtain at least two alternative passenger's seatbelt tangent lines. The intersection angle between each alternative driver's seatbelt tangent line and each alternative passenger's seatbelt tangent line can be detected, and the alternative driver's seatbelt tangent line and the alternative passenger's seatbelt tangent line corresponding to the minimum intersection angle can be used as the outer edge tangent lines of the driver's seatbelt and the passenger's seatbelt, respectively.

[0092] This solution proposes a method for detecting at least two tangent lines between the outermost edge of the driver's seatbelt and the outer edge of the driver's seatbelt area. It introduces the intersection angle between the alternative tangent line of the driver's seatbelt and the outer edge tangent line of the passenger's seatbelt. Based on the minimum intersection angle, the outer edge tangent line of the driver's seatbelt is determined, thereby improving the fault tolerance and applicability of the seatbelt wearing detection method.

[0093] S240. Determine the center of the fan-shaped area of ​​the seat belt based on the intersection of the tangent lines of the outer edges of the driver's seat belt and the passenger's seat belt.

[0094] Specifically, the center of the seatbelt sector can be determined based on the intersection of the outer edge tangents of the driver's seatbelt and the outer edge tangents of the passenger's seatbelt. Optionally, the center of each seatbelt sector, as well as the corresponding outer edge tangents of the driver's seatbelt and the passenger's seatbelt, can be determined based on the intersection of the outer edge tangents of each driver's seatbelt and each passenger's seatbelt.

[0095] S250. Construct a seatbelt sector based on the tangent line of the outer edge of the driver's seatbelt, the tangent line of the outer edge of the passenger's seatbelt, and the center of the sector area of ​​the seatbelt.

[0096] Specifically, it connects the outer edge tangent of the driver's seatbelt, the outer edge tangent of the passenger's seatbelt, and the center of the fan-shaped area of ​​the seatbelt. The radius of the fan-shaped area can be adjusted to detect whether the fan-shaped area corresponding to different radii includes both the driver's and passenger's seatbelt areas. When a fan-shaped area includes both the driver's and passenger's seatbelt areas, it is designated as a candidate fan-shaped area. The candidate fan-shaped areas can be compared, and the smallest candidate fan-shaped area is designated as the official seatbelt fan-shaped area.

[0097] In an optional embodiment of the present invention, a seatbelt sector is constructed based on the tangent of the outer edge of the driver's seatbelt, the tangent of the outer edge of the passenger's seatbelt, and the center of the sector area of ​​the seatbelt. This includes: calculating the distance between the tangent point of the outermost edge of the driver's seatbelt and the center of the sector area of ​​the seatbelt to obtain the driver's tangent point distance; calculating the distance between the tangent point of the outermost edge of the passenger's seatbelt and the center of the sector area of ​​the seatbelt to obtain the passenger's tangent point distance; calculating the sum of the driver's tangent point distance and the passenger's tangent point distance to obtain the radius of the sector area of ​​the seatbelt; and constructing the seatbelt sector area based on the radius and the center of the sector area of ​​the seatbelt.

[0098] The driver's side tangent distance can be considered the distance between the outermost edge of the driver's seatbelt and the center of the seatbelt's fan-shaped area. The passenger's side tangent distance can be considered the distance between the outermost edge of the passenger's seatbelt and the center of the seatbelt's fan-shaped area. Essentially, the seatbelt fan-shaped area is formed based on the spatial relationship between the driver's and passenger's seatbelt areas. Determining the radius of the seatbelt fan-shaped area based on the sum of the driver's and passenger's tangent distances can be understood as taking into account the geometric information of both areas, resulting in a more reliable radius.

[0099] Specifically, the distance between the outermost tangent point of the driver's seatbelt and the center of the fan-shaped area of ​​the seatbelt can be calculated to obtain the driver's tangent point distance. The distance between the outermost tangent point of the passenger's seatbelt and the center of the fan-shaped area of ​​the seatbelt can also be calculated to obtain the passenger's tangent point distance. The sum of the driver's and passenger's tangent point distances can then be calculated to obtain the radius of the fan-shaped area of ​​the seatbelt. Based on the radius and center of the seatbelt area, the fan-shaped area of ​​the seatbelt can be constructed.

[0100] This solution introduces the driver's side tangent distance and the passenger's side tangent distance. Based on the sum of the driver's side tangent distance and the passenger's side tangent distance, the radius of the seat belt sector is determined, and then the seat belt sector is constructed. This takes into account the geometric correlation between the driver's side seat belt sector and the passenger's side seat belt sector, improving the accuracy of seat belt wearing detection. At the same time, it simplifies the construction process of the seat belt sector, further improving the construction efficiency of the seat belt sector and thus improving the efficiency of seat belt wearing detection.

[0101] Optionally, the distances to the driver's side and passenger's side can be compared, and twice the larger value can be used as the radius of the seatbelt sector. The seatbelt sector can be constructed based on its radius and center. This avoids situations where the seatbelt sector is too small, leading to missed detections, and improves the comprehensiveness of seatbelt-based detection.

[0102] Optionally, the distances to the driver's side and the passenger's side can be compared, and twice the smaller value can be used as the radius of the seatbelt sector. The seatbelt sector can be constructed based on its radius and center. This reduces the size of the seatbelt sector and improves the detection accuracy based on seatbelt wearing detection.

[0103] S260. Based on the overlap between the key head point of the driver's seat and the fan-shaped area of ​​the seat belt, and the overlap between the key head point of the passenger's seat and the fan-shaped area of ​​the seat belt, determine the seat belt wearing test results for the driver and passenger.

[0104] The technical solution of this invention introduces the outer edge tangent of the driver's seat belt and the outer edge tangent of the passenger's seat belt. Compared with straight line fitting of the outer edge of the driver's seat belt area and the outer edge of the passenger's seat belt area, the method of constructing the seat belt fan-shaped area based on the outer edge tangent of the driver's seat belt and the outer edge tangent of the passenger's seat belt is simpler and faster, which can improve the construction efficiency of the seat belt fan-shaped area, and thus improve the efficiency of seat belt wearing detection for both driver and passenger.

[0105] Figure 3 is a flowchart of a seatbelt wearing detection method according to Embodiment 2 of the present invention. Based on the above embodiments, Figure 3 shows a preferred embodiment of the present invention. Referring to Figure 3, the seatbelt wearing detection method includes:

[0106] S310: Acquire real-time video stream inside the vehicle.

[0107] Real-time in-vehicle video stream refers to the in-vehicle driving video.

[0108] Specifically, a camera inside the vehicle can be used to acquire real-time video streams from the vehicle's interior. These real-time video streams can then be processed frame-by-frame to create in-vehicle driving images. For example, the in-vehicle camera can be a high-resolution camera, such as a DMS camera (Driver Monitoring System Camera). The image acquisition device can be installed at locations such as the A-pillar, reading lights, or other forward positions that can detect images of the upper torso and abdominal areas of the driver and front passenger. Optionally, considering different lighting conditions, the image acquisition device can be equipped with adaptive exposure adjustment to ensure clear in-vehicle driving images are obtained in various environments, including sunlight, shadows, and nighttime.

[0109] S320, inspect the driver's seat belt area and the passenger's seat belt area.

[0110] Specifically, a pre-trained seatbelt region detection model can be used to detect in-vehicle driving images and output the driver's seatbelt region and the passenger's seatbelt region. The seatbelt region detection model can be a semantic segmentation model. For example, the semantic segmentation model could be HRNET (a deep neural network model for image recognition tasks).

[0111] For example, Figure 4 is a schematic diagram of the input data and output results for seat belt area detection. As shown in Figure 4, the black area represents the segmented driver's seat belt area and passenger's seat belt area. This solution only uses the seat belt that crosses diagonally across the driver's chest, without considering the horizontal seat belt that obstructs the abdomen.

[0112] S330. Determine whether the driver's seat belt area and the passenger's seat belt area meet the position and area conditions. If yes, proceed to S340; otherwise, proceed to S370.

[0113] Specifically, the system can detect the existence of the driver's seatbelt area and the passenger's seatbelt area, and calculate their respective areas. If two independent driver's seatbelt areas and passenger's seatbelt areas are detected, and the areas of both areas are greater than or equal to a pre-set reference seatbelt area, then the position and area conditions are considered met. If two independent driver's seatbelt areas and passenger's seatbelt areas are not detected, or if either area is smaller than the pre-set reference seatbelt area, then the position and area conditions are not met, and it is determined that the driver and passenger are not wearing seatbelts. The reference seatbelt area can be a fixed value, such as 300 or 500, or a ratio, such as 1 / 30 of the entire in-vehicle driving image.

[0114] S340. Construct a seatbelt sector area.

[0115] Specifically, a fan-shaped area for seat belts can be constructed based on the driver's seat belt area and the passenger's seat belt area.

[0116] For example, Figure 5 is a schematic diagram of the principle of constructing the seat belt sector area. As shown in Figure 5, the following steps can be used to construct the seat belt sector area.

[0117] S3401. Determine the center of the circle.

[0118] Specifically, the outermost edge points of the two regions can be identified to obtain the outermost tangent points of the driver's seat belt and the passenger's seat belt. External tangent lines can be drawn through these points to their respective regions. If the tangent lines are not unique, the tangent line corresponding to the minimum relative angle between the two lines is selected as the outermost tangent lines of the driver's seat belt and the passenger's seat belt.

[0119] For example, as shown in Figure 5, the two green dots on the chest are the tangent points of the seat belt area (i.e., the tangent points of the outermost edges of the driver's seat belt and the passenger's seat belt); the two purple lines are the tangent lines of the outer edges of the driver's seat belt and the passenger's seat belt, and the black dot where the two lines intersect is the center.

[0120] S3403, Determine the radius.

[0121] For example, the radius of the seat belt sector can be determined using the following formula: R = L1 + L2;

[0122] In the formula, R is the radius of the seat belt sector; L1 is the distance from the driver's tangent point; and L2 is the distance from the passenger's tangent point.

[0123] S3403, Construct a fan-shaped area for seat belts.

[0124] Specifically, a circle can be drawn to construct the fan-shaped area of ​​the seat belt, based on the center and radius of the circle. As shown in Figure 5, this circle is tangent to two tangent lines to form a fan, as shown by the area formed by the purple and yellow line segments in the figure.

[0125] S350 identifies key eye points of the driver and passenger.

[0126] Specifically, a pre-trained head landmark detection model can be used to detect head landmarks in in-vehicle driving images and output the head landmarks of the driver and passenger. For example, the head landmark detection model can be Center Net (a center point detection model based on convolutional neural networks). The red dots in Figure 5 represent the eye landmarks of the driver and passenger.

[0127] S360: Determine whether the seat belt sector area contains the key points of the driver's eyes and the key points of the passenger's eyes. If yes, determine that the driver and passenger have worn seat belts and that the seat belt wearing posture is correct, and return to execute S310 for the next frame of in-vehicle driving image; if not, execute S370.

[0128] As the facial feature closest to the top of the head, the position of the eyes directly reflects the head's posture and movement. If the key eye point is located outside the seatbelt's fan-shaped area, it may indicate that the driver is not wearing the seatbelt correctly, and the seatbelt is failing to properly restrain the driver's body, thus failing to provide its protective function. In the event of a collision, due to the lack of proper restraint, the driver may be ejected from the vehicle, increasing the risk of injury. In a real-world project, a series of experiments simulated vehicle collisions. By detecting the head movement trajectory and key eye point of drivers under different seatbelt wearing postures, and comparing experimental data, the correlation between the location of the key eye point within the seatbelt's fan-shaped area and correct seatbelt wearing was verified.

[0129] For example, as shown in Figure 6, it can be determined whether all four key eye points are within the seat belt sector area. If they are, it proves that both the driver and passenger are wearing seat belts and in the correct posture; if not, it proves that at least one of the driver and passenger is wearing the seat belt incorrectly.

[0130] S370 sends a seatbelt reminder message to the driver and front passenger.

[0131] Seatbelt wearing reminders may include audible reminders or flashing light reminders.

[0132] Specifically, if it is detected that the driver and front passenger are not wearing seat belts or are wearing them incorrectly, a seat belt reminder message will be sent to the driver and front passenger to remind them to adjust their seat belt wearing posture in order to improve driving safety.

[0133] This solution simultaneously determines the seatbelt wearing posture of both the driver and front passenger. Compared to existing technologies, this solution does not determine the seatbelt wearing posture separately for the driver and front passenger. Instead, it establishes a connection between the two users, using a single method to detect the seatbelt wearing posture of both simultaneously, thus reducing the computational load of the seatbelt wearing detection process. Only when both the driver and front passenger are wearing their seatbelts correctly is the posture considered correct, thus ensuring higher safety standards. Furthermore, by judging the seatbelt wearing posture based on the overlap between key eye points and the seatbelt's fan-shaped area, the solution uses the eye points' position within the fan-shaped area as the criterion for seatbelt effectiveness. This establishes a geometric connection between the driver and front passenger, simplifying the seatbelt wearing detection process. Figure 7 illustrates an incorrect seatbelt wearing posture. As shown in Figure 7, in this common incorrect wearing posture, the seat belt does not cross the chest, causing the overall height of the seat belt to decrease compared to the correct wearing posture. Using the seat belt wearing detection method of this solution, the constructed seat belt fan-shaped area will sweep past the left and right of the chin, and the key point of the eyes will be outside the seat belt fan-shaped area, thus triggering an alarm, proving the effectiveness of the seat belt wearing detection method proposed in this solution.

[0134] Example 3

[0135] Figure 8 is a schematic diagram of a seatbelt wearing detection device provided in Embodiment 3 of the present invention. This embodiment of the present invention is applicable to situations where the seatbelt wearing status of the driver and front passenger is jointly detected. The device can execute a seatbelt wearing detection method and can be implemented in hardware and / or software. The device can be configured in an electronic device that carries the seatbelt wearing detection function, such as an in-vehicle terminal.

[0136] Referring to Figure 8, the seatbelt wearing detection device includes: an in-vehicle driving image acquisition module 810, a seatbelt area and key point detection module 820, a seatbelt fan-shaped area construction module 830, and a seatbelt wearing detection module 840. The in-vehicle driving image acquisition module 810 acquires in-vehicle driving images of the driver and front passenger. The seatbelt area and key point detection module 820 detects the in-vehicle driving images to determine the driver's seatbelt area, the front passenger seatbelt area, the driver's head key point, and the front passenger's head key point. The driver's seatbelt area is the area that crosses the driver's upper torso; the front passenger seatbelt area is the area that crosses the front passenger's upper torso. The seatbelt fan-shaped area construction module 830 constructs a seatbelt fan-shaped area based on the driver's and front passenger seatbelt areas. The seatbelt wearing detection module 840 determines the seatbelt wearing detection results for both the driver and front passenger based on the overlap between the driver's head key point and the seatbelt fan-shaped area, and the overlap between the front passenger's head key point and the seatbelt fan-shaped area.

[0137] The technical solution of this invention detects in-vehicle driving images to determine the driver's seatbelt area, the passenger's seatbelt area, the driver's head key points, and the passenger's head key points. Based on the driver's and passenger's seatbelt areas, a seatbelt sector area is constructed. Considering the overlap between the driver's and passenger's head key points and the seatbelt sector area, and taking into account the geometric relationship between the seatbelt area and the head key points, the system can detect the seatbelt wearing status of both drivers and passengers. This improves the accuracy of seatbelt wearing detection and ensures the safety of vehicle driving. Furthermore, compared to detecting seatbelt wearing separately for each driver, simultaneously judging the seatbelt wearing status of both drivers and passengers further improves the detection efficiency.

[0138] In an optional embodiment of the present invention, the seat belt sector area construction module 830 includes: a seat belt outer edge tangent determination unit, used to determine the outer edge tangent of the driver's seat belt based on the driver's seat belt area, and to determine the outer edge tangent of the passenger's seat belt based on the passenger's seat belt area; a center determination unit, used to determine the center of the seat belt sector area based on the intersection point between the outer edge tangent of the driver's seat belt and the outer edge tangent of the passenger's seat belt; and a seat belt sector area construction unit, used to construct the seat belt sector area based on the outer edge tangent of the driver's seat belt, the outer edge tangent of the passenger's seat belt, and the center of the seat belt sector area.

[0139] In an optional embodiment of the present invention, the seat belt outer edge tangent determination unit includes: a driver's seat belt outermost edge tangent point subunit, used to determine the driver's seat belt outermost edge tangent point based on the driver's seat belt area; and a driver's seat belt outer edge tangent determination subunit, used to determine the driver's seat belt outer edge tangent based on the tangent between the driver's seat belt outermost edge tangent point and the outer edge of the driver's seat belt area.

[0140] In an optional embodiment of the present invention, the driver's seat belt outer edge tangent determination subunit includes: a driver's seat belt candidate tangent determination slave unit, used to detect at least two tangents between the outermost edge tangent point of the driver's seat belt and the outer edge of the driver's seat belt area, to obtain at least two driver's seat belt candidate tangents; a passenger seat belt outer edge tangent acquisition slave unit, used to acquire the passenger seat belt outer edge tangent; an intersection angle detection slave unit, used to detect the intersection angle between each driver's seat belt candidate tangent and the passenger seat belt outer edge tangent; and a driver's seat belt outer edge tangent determination slave unit, used to take the driver's seat belt candidate tangent corresponding to the minimum intersection angle as the driver's seat belt outer edge tangent.

[0141] In an optional embodiment of the present invention, the seat belt sector area construction unit includes: a driver's side tangent distance determination subunit, used to calculate the distance between the outermost tangent point of the driver's side seat belt and the center of the seat belt sector area to obtain the driver's side tangent distance; a passenger side tangent distance determination subunit, used to calculate the distance between the outermost tangent point of the passenger side seat belt and the center of the seat belt sector area to obtain the passenger side tangent distance; a radius determination subunit, used to calculate the sum of the driver's side tangent distance and the passenger side tangent distance to obtain the radius of the seat belt sector area; and a seat belt sector area construction subunit, used to construct the seat belt sector area based on the radius and the center of the seat belt area.

[0142] In an optional embodiment of the present invention, the key points of the driver's head include the key points of the driver's eyes, the key points of the driver's nose, the key points of the driver's mouth, or the center point of the driver's head; the key points of the passenger's head include the key points of the passenger's eyes, the key points of the passenger's nose, the key points of the passenger's mouth, or the center point of the passenger's head.

[0143] In an optional embodiment of the present invention, the device further includes: a seatbelt area detection module, used to detect the driver's seatbelt area and the passenger's seatbelt area before determining the seatbelt wearing detection result of the driver and passenger users based on the overlap state of the driver's head key point and the seatbelt fan-shaped area, and the overlap state of the passenger's head key point and the seatbelt fan-shaped area; a seatbelt wearing status detection module, used to determine the seatbelt wearing status detection result of the driver user based on the area of ​​the driver's seatbelt area when both the driver's and passenger's seatbelt areas exist; and to determine the seatbelt wearing status detection result of the passenger user based on the area of ​​the passenger's seatbelt area; the seatbelt wearing detection module includes: a seatbelt wearing posture detection unit, used to determine the seatbelt wearing posture detection result of the driver and passenger users based on the overlap state of the driver's head key point and the seatbelt fan-shaped area, and the overlap state of the passenger's head key point and the seatbelt fan-shaped area when both the driver and passenger users are wearing seatbelts.

[0144] In an optional embodiment of the present invention, the device further includes: a seat belt wearing reminder module, which, after determining the seat belt wearing detection results of the driver and passenger based on the overlap state of the driver's head key point and the seat belt fan-shaped area, and the overlap state of the passenger's head key point and the seat belt fan-shaped area, determines seat belt wearing reminder information based on the seat belt wearing detection results of the driver and passenger and feeds it back to the driver and passenger.

[0145] The seat belt wearing detection device provided in this embodiment of the invention can execute the seat belt wearing detection method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0146] In the technical solutions of this invention, the acquisition, storage, and application of in-vehicle driving images of the driver and passenger are in compliance with relevant laws and regulations and do not violate public order and good morals.

[0147] Example 4

[0148] Figure 9 illustrates a schematic diagram of an electronic device 900 that can be used to implement embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0149] As shown in Figure 9, the electronic device 900 includes at least one processor 901 and a memory, such as a read-only memory (ROM) 902 or a random access memory (RAM) 903, communicatively connected to the processor 901. The memory stores computer programs executable by the processor. The processor 901 can perform various appropriate actions and processes based on the computer program stored in the ROM 902 or loaded into the RAM 903 from storage unit 908. The RAM 903 can also store various programs and data required for the operation of the electronic device 900. The processor 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0150] Multiple components in electronic device 900 are connected to I / O interface 905, including: input unit 906, such as keyboard, mouse, etc.; output unit 907, such as various types of displays, speakers, etc.; storage unit 908, such as disk, optical disk, etc.; and communication unit 909, such as network card, modem, wireless transceiver, etc. Communication unit 909 allows electronic device 900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0151] Processor 901 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 901 performs the various methods and processes described above, such as a seatbelt wearing detection method.

[0152] In some embodiments, the seatbelt wearing detection method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 908. In some embodiments, part or all of the computer program may be loaded into and / or installed on electronic device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by processor 901, one or more steps of the seatbelt wearing detection method described above may be performed. Alternatively, in other embodiments, processor 901 may be configured to perform the seatbelt wearing detection method by any other suitable means (e.g., by means of firmware).

[0153] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0154] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0155] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0156] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0157] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0158] A computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system. It addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability.

[0159] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0160] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A seat belt wearing detection method characterized by comprising: The method includes: Acquire in-vehicle driving images of the driver and front passenger; The in-vehicle driving image is detected to determine the driver's seat belt area, the passenger's seat belt area, the key head points of the driver and the passenger; wherein, the driver's seat belt area is the seat belt area that crosses the upper torso of the driver; the passenger's seat belt area is the seat belt area that crosses the upper torso of the passenger. Construct a seatbelt sector area based on the driver's seatbelt area and the passenger's seatbelt area; The seatbelt wearing detection results for the driver and passenger are determined based on the overlap between the key head points of the driver's head and the fan-shaped area of ​​the seatbelt, and the overlap between the key head points of the passenger's head and the fan-shaped area of ​​the seatbelt.

2. The method of claim 1, wherein, The step of constructing a seatbelt sector area based on the driver's seatbelt area and the passenger's seatbelt area includes: Based on the driver's seat belt area, determine the tangent line of the outer edge of the driver's seat belt, and based on the passenger's seat belt area, determine the tangent line of the outer edge of the passenger's seat belt. The center of the fan-shaped area of ​​the seat belt is determined based on the intersection point between the outer edge tangent of the driver's seat belt and the outer edge tangent of the passenger's seat belt. The seat belt sector area is constructed based on the tangent line of the outer edge of the driver's seat belt, the tangent line of the outer edge of the passenger's seat belt, and the center of the sector area of ​​the seat belt.

3. The method of claim 2, wherein, Determining the tangent line of the outer edge of the driver's seat belt based on the driver's seat belt area includes: Based on the driver's seatbelt area, determine the tangent point of the outermost edge of the driver's seatbelt; The outer edge tangent of the driver's seat belt is determined based on the tangent between the outermost edge tangent point of the driver's seat belt and the outer edge of the driver's seat belt area.

4. The method of claim 3, wherein, Determining the outer edge tangent of the driver's seatbelt based on the tangent between the outermost edge tangent point of the driver's seatbelt and the outer edge of the driver's seatbelt area includes: Detect at least two tangent lines between the outermost edge of the driver's seat belt and the outer edge of the driver's seat belt area to obtain at least two alternative driver's seat belt tangent lines; Obtain the tangent line of the outer edge of the passenger seat belt; Detect the intersection angle between each of the aforementioned driver's side alternative tangent lines and the outer edge tangent line of the passenger's side seat belt; The driver's side alternative tangent line corresponding to the minimum intersection angle is taken as the outer edge tangent line of the driver's side seat belt.

5. The method of claim 2, wherein, The step of constructing the seatbelt sector region based on the tangent line of the outer edge of the driver's seatbelt, the tangent line of the outer edge of the passenger's seatbelt, and the center of the sector region of the seatbelt includes: Calculate the distance between the outermost edge tangent point of the driver's seat belt and the center of the fan-shaped area of ​​the seat belt to obtain the driver's tangent point distance; Calculate the distance between the outermost edge tangent point of the passenger seat belt and the center of the fan-shaped area of ​​the seat belt to obtain the passenger seat tangent point distance; Calculate the sum of the driver's side tangent distance and the passenger's side tangent distance to obtain the radius of the seat belt sector area; The seat belt sector area is constructed based on the radius of the seat belt area and the center of the seat belt area.

6. The method of claim 1, wherein, The key points of the driver's head include the key points of the driver's eyes, the key points of the driver's nose, the key points of the driver's mouth, or the center point of the driver's head. The key points of the co-pilot's head include the key points of the co-pilot's eyes, nose, mouth, or the center point of the co-pilot's head.

7. The method of claim 1, wherein, Before determining the seatbelt wearing detection results for the driver and passenger based on the overlap between the driver's head key point and the seatbelt sector area, and the overlap between the passenger's head key point and the seatbelt sector area, the method further includes: The driver's seat belt area and the passenger's seat belt area were inspected; When both the driver's seatbelt area and the passenger's seatbelt area exist, the seatbelt wearing status detection result of the driver's seatbelt user is determined based on the area of ​​the driver's seatbelt area; and the seatbelt wearing status detection result of the passenger's seatbelt user is determined based on the area of ​​the passenger's seatbelt area. The determination of the seatbelt wearing detection results for the driver and passenger based on the overlap between the driver's head key point and the seatbelt fan-shaped area, and the overlap between the passenger's head key point and the seatbelt fan-shaped area, includes: When both the driver and passenger are wearing seat belts, the seat belt wearing posture detection results of the driver and passenger are determined based on the overlap between the key points of the driver's head and the fan-shaped area of ​​the seat belt, and the overlap between the key points of the passenger's head and the fan-shaped area of ​​the seat belt.

8. The method of claim 1, wherein, After determining the seatbelt wearing detection results for the driver and passenger based on the overlap between the driver's head key point and the seatbelt sector area, and the overlap between the passenger's head key point and the seatbelt sector area, the method further includes: Based on the seatbelt wearing detection results of the driver and front passenger, a seatbelt wearing reminder message is determined and fed back to the driver and front passenger.

9. A seat belt wearing detection device characterized by comprising: The device includes: The in-vehicle driving image acquisition module is used to acquire in-vehicle driving images of the driver and front passenger. The seat belt area and key point detection module is used to detect the in-vehicle driving image and determine the driver's seat belt area, the passenger's seat belt area, the driver's head key point, and the passenger's head key point; wherein, the driver's seat belt area is the seat belt area that crosses the upper torso of the driver; the passenger's seat belt area is the seat belt area that crosses the upper torso of the passenger; A seatbelt sector area construction module is used to construct a seatbelt sector area based on the driver's seatbelt area and the passenger's seatbelt area; The seatbelt wearing detection module is used to determine the seatbelt wearing detection results of the driver and passenger based on the overlap between the key head point of the driver and the fan-shaped area of ​​the seatbelt, and the overlap between the key head point of the passenger and the fan-shaped area of ​​the seatbelt.

10. An electronic device, comprising: The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the seatbelt wearing detection method according to any one of claims 1-8.

11. A computer readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the seatbelt wearing detection method according to any one of claims 1-8.

12. A computer program product, characterised in that, The computer program product includes a computer program that, when executed by a processor, implements the seatbelt wearing detection method according to any one of claims 1-8.

Images