Safety-belt wearing detection method and apparatus, and electronic device and storage medium

Abstract

The present application relates to the technical field of vehicle driving. Disclosed are a safety-belt wearing detection method and apparatus, and an electronic device and a storage medium. The method comprises: acquiring an in-vehicle driving image of a driver and a front-seat passenger; checking the in-vehicle driving image, so as to determine a driver safety-belt area, a front-seat-passenger safety-belt area, driver head key points and front-seat-passenger head key points; on the basis of the driver safety-belt area and the front-seat-passenger safety-belt area, constructing a safety-belt fan-shaped area; and on the basis of the state of overlap between the driver head key points and the safety-belt fan-shaped area and the state of overlap between the front-seat-passenger head key points and the safety-belt fan-shaped area, determining a safety-belt wearing detection result for the driver and the front-seat passenger. The technical solution in the embodiments of the present invention realizes simultaneous detection of the actual safety-belt wearing condition of a driver and a front-seat passenger, thereby ensuring the safety of the driver and the front-seat passenger during vehicle travelling, and also improving the detection efficiency of safety-belt wearing detection.

Description

Safety belt wearing detection method and device, electronic equipment and storage medium TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle driving, and in particular to a safety belt wearing detection method and device, electronic equipment and storage medium. BACKGROUND

[0002] With the enhancement of traffic safety awareness, the use of safety belts has become an indispensable part of vehicle driving safety.

[0003] Existing vehicles are usually equipped with a safety belt wearing reminder system, which mainly relies on mechanical sensors to detect whether the safety belt has been inserted into the lock buckle. However, this safety belt wearing detection method cannot ensure whether the driver actually wears the safety belt, and the safety during vehicle driving cannot be guaranteed; in addition, the detection efficiency of the existing safety belt wearing detection also needs to be improved.

[0004] SUMMARY

[0005] The present application provides a safety belt wearing detection method and device, electronic equipment and storage medium, which realizes the detection of the actual safety belt wearing of the main and vice drivers at the same time, ensures the safety of the main and vice drivers during vehicle driving, and also improves the detection efficiency of the safety belt wearing detection.

[0006] According to an aspect of the present application, a safety belt wearing detection method is provided, the method comprising:

[0007] obtaining an in-vehicle driving image of a main and vice driver;

[0008] detecting the in-vehicle driving image to determine a main driver safety belt region, a vice driver safety belt region, a main driver head key point and a vice driver head key point; wherein the main driver safety belt region is a safety belt region across the upper torso of the main driver; the vice driver safety belt region is a safety belt region across the upper torso of the vice driver;

[0009] constructing a safety belt sector region according to the main driver safety belt region and the vice driver safety belt region;

[0010] determining a safety belt wearing detection result of the main and vice drivers according to the overlap state of the main driver head key point and the safety belt sector region, and the overlap state of the vice driver head key point and the safety belt sector region.

[0011] According to another aspect of the present application, a safety belt wearing detection device is provided, the device comprising:

[0012] an in-vehicle driving image acquisition module for acquiring an in-vehicle driving image of a main and vice driver;

[0013] a seat belt area and key point detection module configured to detect the in-vehicle driving image to determine a main driver seat belt area, a co-driver seat belt area, a main driver head key point, and a co-driver head key point, wherein the main driver seat belt area is a seat belt area crossing an upper torso of a main driver user, and the co-driver seat belt area is a seat belt area crossing an upper torso of a co-driver user;

[0014] a seat belt sector area construction module configured to construct a seat belt sector area according to the main driver seat belt area and the co-driver seat belt area;

[0015] a seat belt wearing detection module configured to determine a seat belt wearing detection result of the main and co-driver users according to an overlapping state of the main driver head key point and the seat belt sector area, and an overlapping state of the co-driver head key point and the seat belt sector area.

[0016] According to another aspect of the present application, an electronic device is provided, which comprises:

[0017] at least one processor; and

[0018] a memory connected to the at least one processor in communication; wherein

[0019] the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the seat belt wearing detection method according to any one of the embodiments of the present application.

[0020] According to another aspect of the present application, a computer readable storage medium is provided, which stores computer instructions for enabling a processor to implement the seat belt wearing detection method according to any one of the embodiments of the present application when executed by the processor.

[0021] According to another aspect of the present application, a computer program product is provided, which comprises a computer program for implementing the seat belt wearing detection method according to any one of the embodiments of the present application when executed by a processor.

[0022] The technical scheme of the embodiment of the present application determines the main driver safety belt region, the co-driver safety belt region, the main driver head key point and the co-driver head key point by detecting the driving image in the vehicle, constructs a safety belt sector region according to the main driver safety belt region and the co-driver safety belt region, considers the geometric relationship between the safety belt region and the head key point based on the overlapping state of the main driver head key point and the safety belt sector region and the overlapping state of the co-driver head key point and the safety belt sector region, and can realize the detection of the safety belt wearing condition of the main and co-driver users, improve the accuracy of the safety belt wearing detection, ensure the safety of the vehicle driving process, and further improve the detection efficiency of the safety belt wearing detection by simultaneously judging the safety belt wearing condition of the main and co-driver users compared with respectively detecting the safety belt wearing of a single driving user.

[0023] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0025] Fig. 1 is a flowchart of a safety belt wearing detection method according to an embodiment of the present application;

[0026] Fig. 2 is a flowchart of a safety belt wearing detection method according to an embodiment of the present application;

[0027] Fig. 3 is a flowchart of a safety belt wearing detection method according to an embodiment of the present application;

[0028] Fig. 4 is a schematic diagram of safety belt region detection according to an embodiment of the present application;

[0029] Fig. 5 is a principle diagram of constructing a safety belt sector region according to an embodiment of the present application;

[0030] Fig. 6 is a schematic diagram of eye key points in a safety belt sector region according to an embodiment of the present application;

[0031] Fig. 7 is a schematic diagram when the safety belt wearing posture is wrong according to an embodiment of the present application;

[0032] Fig. 8 is a structural schematic diagram of a safety belt wearing detection device according to an embodiment of the present application;

[0033] Fig. 9 is a structural schematic diagram of an electronic device implementing a seat belt wearing detection method according to an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work should fall within the scope of protection of the present application.

[0035] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to include only those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to the process, method, product or device.

[0036] Embodiment One

[0037] Fig. 1 is a flowchart of a seat belt wearing detection method according to an embodiment of the present application. The embodiments of the present application can be applied to the case of jointly detecting the seat belt wearing situation of the main driver and the co-driver. The method can be executed by a seat belt wearing detection device, which can be implemented in the form of hardware and / or software. The seat belt wearing detection device can be configured in an electronic device that carries a seat belt wearing detection function, such as a vehicle terminal.

[0038] Referring to the seat belt wearing detection method shown in Fig. 1, the method comprises the following steps.

[0039] S110, obtaining a driving-in-vehicle image of the main driver and the co-driver.

[0040] The main and assistant driver users include a main driver user and an assistant driver user. The in-vehicle driving images can be in-vehicle images of the main driver user and the assistant driver user during vehicle driving. Based on the in-vehicle driving images, the safety belt wearing conditions of the main driver user and the assistant driver user can be detected. Optionally, the in-vehicle driving images can include front images or side images of the main driver user and the assistant driver user. Preferably, the in-vehicle driving images are front images of the main driver user and the assistant driver user.

[0041] Specifically, the in-vehicle driving images of the main driver user and the assistant driver user can be collected by an image collection device. Exemplarily, the image collection device can be a high-resolution camera, for example, a DMS camera (Driver Monitoring System Camera). The installation position of the image collection device can include an in-vehicle A-pillar, a reading lamp, or other front positions that can detect images of the upper torso and abdominal regions of the main driver and the assistant driver, etc. Optionally, considering different lighting conditions, the image collection device can be equipped with an adaptive exposure adjustment function to ensure that clear in-vehicle driving images can be obtained in different environments such as sunlight, shadow, or night.

[0042] S120, detecting the in-vehicle driving images to determine a main driver safety belt region, an assistant driver safety belt region, a main driver head key point, and an assistant driver head key point.

[0043] The main driver safety belt region can be a safety belt region across the upper torso of the main driver user. The assistant driver safety belt region can be a safety belt region across the upper torso of the assistant driver user. The main driver safety belt region can be used to identify the safety belt worn by the upper torso of the main driver user. The assistant driver safety belt region can be used to identify the safety belt worn by the upper torso of the assistant driver user. By selecting the main driver safety belt region and the assistant driver safety belt region obliquely across the upper torso of the main and assistant driver users for safety belt wearing detection of the main and assistant driver users, the safety belt region across the abdomen of the main and assistant driver users can be excluded in the safety belt wearing detection of the main and assistant driver users, so as to avoid the interference of the safety belt region across the abdomen of the main and assistant driver users on the safety belt wearing detection. The main driver head key point can be used to represent the head position of the main driver user. The main driver head key point can be used to identify the head region of the main driver user. The assistant driver head key point can be used to represent the head position of the assistant driver user. The assistant driver head key point can be used to identify the head region of the assistant driver user. The number of the main driver head key point (or the assistant driver head key point) can be at least one.

[0044] In an optional embodiment of the present application, the main driver head key point comprises a main driver eye key point, a main driver nose key point, a main driver mouth key point or a main driver head center point; and the co-driver head key point comprises a co-driver eye key point, a co-driver nose key point, a co-driver mouth key point or a co-driver head center point.

[0045] The main driver eye key point can be used to identify the eye position of the main driver user. For example, the main driver eye key point can be the eyeball position of the main driver's eyes. The main driver nose key point can be used to identify the nose position of the main driver user. For example, the main driver nose key point can be the nose tip position of the main driver user. The main driver mouth key point can be used to identify the mouth position of the main driver user. For example, the main driver mouth key point can be the center position of the main driver user's mouth. The main driver head center point can be used to identify the head position of the main driver user. For example, the main driver head center point can be the geometric center position of the main driver user's face. The co-driver eye key point can be used to identify the eye position of the co-driver user. For example, the co-driver eye key point can be the eyeball position of the co-driver's eyes. The co-driver nose key point can be used to identify the nose position of the co-driver user. For example, the co-driver nose key point can be the nose tip position of the co-driver user. The co-driver mouth key point can be used to identify the mouth position of the co-driver user. For example, the co-driver mouth key point can be the center position of the co-driver user's mouth. The co-driver head center point can be used to identify the head position of the co-driver user. For example, the co-driver head center point can be the geometric center position of the co-driver user's face.

[0046] The present scheme can realize the safety belt wearing detection of the main and co-driver users and improve the efficiency of the safety belt wearing detection of the main and co-driver users by specifying the main driver head key point as the main driver eye key point, the main driver nose key point, the main driver mouth key point or the main driver head center point, and specifying the co-driver head key point as the co-driver eye key point, the co-driver nose key point, the co-driver mouth key point or the co-driver head center point.

[0047] Specifically, the image recognition technology can be used to perform image recognition on the in-vehicle driving image to detect the main driver safety belt region, the co-driver safety belt region, the main driver head key point and the co-driver head key point.

[0048] Exemplarily, a pre-trained safety belt region detection model can be adopted to detect the in-vehicle driving image and output the main driver safety belt region and the co-driver safety belt region. The input data of the safety belt region detection model can be the in-vehicle driving image, and the output result can be the main driver safety belt region and the co-driver safety belt region. The main driver safety belt region and the co-driver safety belt region can be a one-layer binary image, i.e., a safety belt region image. In the figure, the point with a pixel value of 0 represents a non-safety belt region, and the point with a pixel value of 1 represents a safety belt region. The training method of the safety belt region detection model can be a supervised model training method. The training samples of the safety belt region detection model can include in-vehicle driving image samples, main driver safety belt region samples, and co-driver safety belt region samples. The in-vehicle driving image samples can be in-vehicle images of historical main driver users and historical co-driver users in the vehicle driving process at historical time. For example, the in-vehicle driving image samples can be front image samples of the historical main driver users and the historical co-driver users. The historical main driver user can be the main driver user in the in-vehicle driving image sample. The historical co-driver user can be the co-driver user in the in-vehicle driving image sample. The main driver safety belt region sample can be a main driver safety belt region pre-labeled in the in-vehicle driving image sample. The co-driver safety belt region sample can be a co-driver safety belt region pre-labeled in the in-vehicle driving image sample. The safety belt region detection model can be a semantic segmentation model. For example, the semantic segmentation model can be an HRNET (a deep neural network model for image recognition tasks).

[0049] Exemplarily, a pre-trained head key point detection model can be adopted to detect the in-vehicle driving image and output the head key point of the main driver and the head key point of the co-driver. The input data of the head key point detection model can be the in-vehicle driving image, and the output result can be the head key point of the main driver and the head key point of the co-driver. For example, the head key point of the main driver (or the head key point of the co-driver) can be at least two layers of head key point heat maps of the main driver (or the co-driver), which are used to represent the probability map of the head key point position estimated by the head key point detection model. The training method of the head key point detection model can be a supervised model training method. The training samples of the head key point detection model can include in-vehicle driving image samples, historical main driver head key point samples and historical co-driver head key point samples. The in-vehicle driving image samples can be in-vehicle images of the historical main driver and the historical co-driver during the driving process of the vehicle at the historical time. The historical main driver head key point sample can be the head key position of the historical main driver pre-labeled in the in-vehicle driving image sample. The historical co-driver head key point sample can be the head key position of the historical co-driver pre-labeled in the in-vehicle driving image sample. For example, the historical main driver head key point sample (or the historical co-driver head key point sample) can be the body key point coordinates of the historical main driver (or the historical co-driver) in the in-vehicle driving image. The head key point detection model can be a convolutional neural network model or a deep learning model. For example, the head key point model can be HRNET (a deep neural network model for image recognition tasks) or Center Net (a center point detection model based on a convolutional neural network).

[0050] In S130, a seat belt sector region is constructed according to the main driver seat belt region and the co-driver seat belt region.

[0051] The seat belt sector region can be used to represent the intersection between the seat belt obliquely across the upper torso of the main driver and the seat belt obliquely across the upper torso of the co-driver. For the seat belt sector region, the seat belt sector region should not only contain the main driver seat belt region and the co-driver seat belt region, but also ensure that the seat belt sector region is minimized. In this way, the accuracy of simultaneously detecting the seat belt wearing conditions of the main driver and the co-driver can be ensured.

[0052] Specifically, the outer edge of the main driver safety belt region can be linearly fitted to obtain a fitted main driver outer edge line. The outer edge of the co-driver safety belt region can be linearly fitted to obtain a fitted co-driver outer edge line. The main driver outer edge line and the co-driver outer edge line can be used to construct at least one sector region. The sector region containing the main driver safety belt region and the co-driver safety belt region can be determined as a candidate sector region. The candidate sector regions can be compared, and the candidate sector region with the smallest area can be determined as the safety belt sector region. The candidate sector region is used to screen the safety belt sector region. The candidate sector region contains the main driver safety belt region and the co-driver safety belt region. The number of candidate sector regions can be multiple. Determining the sector region containing the main driver safety belt region and the co-driver safety belt region as the candidate sector region, and screening the safety belt sector region based on the candidate sector region, can ensure the comprehensiveness of the safety belt information contained in the safety belt sector region; determining the candidate sector region with the smallest area as the safety belt sector region can avoid the interference of other regions outside the safety belt region on the safety belt wearing detection, thereby ensuring the accuracy of the safety belt wearing detection of the main and co-driver users.

[0053] S140, determining the safety belt wearing detection result of the main and co-driver users according to the overlap state of the main driver head key point and the safety belt sector region, and the overlap state of the co-driver head key point and the safety belt sector region.

[0054] The overlap state of the main driver head key point and the safety belt sector region can be used to represent whether the head key point of the main driver user is covered by the safety belt sector region. The overlap state of the co-driver head key point and the safety belt sector region can be used to represent whether the head key point of the co-driver user is covered by the safety belt sector region. When the main driver user and the co-driver user both normally wear safety belts, the safety belt sector region formed by the main driver safety belt region and the co-driver safety belt region can cover the main driver head key point and the co-driver head key point. However, when any one of the main driver user and the co-driver user does not normally wear a safety belt, the safety belt sector region formed by the main driver safety belt region and the co-driver safety belt region is difficult to cover the main driver head key point and the co-driver head key point. Therefore, the safety belt wearing situation of the main and co-driver users can be simultaneously detected based on the overlap state of the main driver head key point and the safety belt sector region, and the overlap state of the co-driver head key point and the safety belt sector region.

[0055] The safety belt wearing detection result can be used to uniformly represent the safety belt wearing conditions of the driver and the front passenger. Optionally, the safety belt wearing detection result can include safety belt wearing normal and safety belt wearing abnormal. The safety belt wearing normal can be used to represent that both the driver and the front passenger normally wear the safety belt. For example, both the driver and the front passenger have worn the safety belt and / or the safety belt wearing posture is correct. The safety belt wearing abnormal can be used to represent that the driver and the front passenger do not normally wear the safety belt. For example, either the driver or the front passenger does not wear the safety belt or the safety belt wearing posture is incorrect.

[0056] Specifically, the overlap state of the driver head key point and the safety belt sector area can be detected. When the driver head key point and the safety belt sector area overlap and the front passenger head key point and the safety belt sector area overlap, it can be determined that the safety belt wearing of the driver and the front passenger is normal. When any one of the driver head key point and the front passenger head key point does not overlap with the safety belt sector area, it can be determined that the safety belt wearing of the driver and the front passenger is abnormal.

[0057] In an optional embodiment of the present application, before determining the safety belt wearing detection result of the driver and the front passenger according to the overlap state of the driver head key point and the safety belt sector area and the overlap state of the front passenger head key point and the safety belt sector area, the method further comprises: detecting the driver safety belt area and the front passenger safety belt area; when both the driver safety belt area and the front passenger safety belt area exist, determining the safety belt wearing state detection result of the driver according to the area of the driver safety belt area, and determining the safety belt wearing state detection result of the front passenger according to the area of the front passenger safety belt area; and determining the safety belt wearing detection result of the driver and the front passenger according to the overlap state of the driver head key point and the safety belt sector area and the overlap state of the front passenger head key point and the safety belt sector area, comprising: when the driver has worn the safety belt and the front passenger has worn the safety belt, determining the safety belt wearing posture detection result of the driver and the front passenger according to the overlap state of the driver head key point and the safety belt sector area and the overlap state of the front passenger head key point and the safety belt sector area.

[0058] The seat belt wearing detection result can include a seat belt wearing state and a seat belt wearing posture. The seat belt wearing state can be used to represent whether the primary driver user or the co-driver user wears a seat belt. The seat belt wearing state can include a seat belt worn and a seat belt not worn. The seat belt wearing posture can be used to represent whether the seat belt wearing postures of the primary driver user and the co-driver user are correct. The seat belt wearing posture can include a correct wearing posture and an incorrect wearing posture. The primary driver seat belt region and the co-driver seat belt region both exist, which can be understood as that the seat belt obliquely across the upper torso of the primary driver user is detected, and the seat belt obliquely across the upper torso of the co-driver user is detected. The area of the primary driver seat belt region can be the area of the seat belt region obliquely across the upper torso of the primary driver user in the in-vehicle driving image. The area of the co-driver seat belt region is the area of the seat belt region obliquely across the upper torso of the co-driver user in the in-vehicle driving image. Optionally, the area of the primary driver seat belt region (or the co-driver seat belt region) can include the occupied area of the primary driver seat belt region (or the co-driver seat belt region) in the in-vehicle driving image or the ratio between the occupied area of the primary driver seat belt region (or the co-driver seat belt region) in the in-vehicle driving image and the area of the in-vehicle driving image. The area of the primary driver seat belt region (or the co-driver seat belt region) can be used to further detect the area of the seat belt obliquely across the upper torso of the primary driver user (or the co-driver user), so as to further determine whether the primary driver user (or the co-driver user) wears a seat belt. On the basis of detecting that the primary driver user has worn a seat belt and the co-driver user has worn a seat belt, the seat belt wearing postures of the primary driver user and the co-driver user can be jointly detected based on the overlap state of the primary driver head key point and the seat belt sector region and the overlap state of the co-driver head key point and the seat belt sector region.

[0059] When the seat belt wearing posture of the primary driver user is correct and the seat belt wearing posture of the co-driver user is correct, the seat belt sector region constituted by the primary driver seat belt region and the co-driver seat belt region can cover the primary driver head key point and the co-driver head key point. However, when the seat belt wearing posture of any one of the primary driver user and the co-driver user is incorrect, the seat belt sector region constituted by the primary driver seat belt region and the co-driver seat belt region is difficult to cover the primary driver head key point and the co-driver head key point completely. Therefore, the seat belt wearing postures of the primary driver user and the co-driver user can be simultaneously detected based on the overlap state of the primary driver head key point and the seat belt sector region and the overlap state of the co-driver head key point and the seat belt sector region.

[0060] Specifically, the driver seat safety belt area and the front passenger seat safety belt area can be detected. When both the driver seat safety belt area and the front passenger seat safety belt area exist, the area of the driver seat safety belt area and the reference safety belt area can be compared. When the area of the driver seat safety belt area is greater than or equal to the reference safety belt area, it is determined that the driver user has worn the safety belt. When the area of the driver seat safety belt area is less than the reference safety belt area, it is determined that the driver user has not worn the safety belt. The area of the front passenger seat safety belt area and the reference safety belt area can be compared. When the area of the front passenger seat safety belt area is greater than or equal to the reference safety belt area, it is determined that the front passenger user has worn the safety belt. When the area of the front passenger seat safety belt area is less than the reference safety belt area, it is determined that the front passenger user has not worn the safety belt. When the driver user has worn the safety belt and the front passenger user has worn the safety belt, the overlap state of the driver head key point and the safety belt sector area can be detected. When the head key point of the driver user and the head key point of the front passenger user overlap with the safety belt sector area, it is determined that the safety belt wearing posture of the driver and front passenger users is correct. When any one of the head key point of the driver user and the head key point of the front passenger user does not overlap with the safety belt sector area, it is determined that the safety belt wearing posture of the driver and front passenger users is incorrect.

[0061] Optionally, when the driver seat safety belt area does not exist, it is determined that the driver user has not worn the safety belt. When the front passenger seat safety belt area does not exist, it is determined that the front passenger user has not worn the safety belt. When both the driver seat safety belt area and the front passenger seat safety belt area do not exist, it is determined that the driver user and the front passenger user have not worn the safety belt.

[0062] The safety belt wearing detection result is specifically the safety belt wearing state and the safety belt wearing posture. Based on whether the driver seat safety belt area and the front passenger seat safety belt area exist, the safety belt wearing state of the driver user and the front passenger user is detected. Based on the area of the driver seat safety belt area and the area of the front passenger seat safety belt area, the accuracy of the safety belt wearing state detection of the driver user and the front passenger user is further improved. Based on the overlap state of the driver head key point and the safety belt sector area, and the overlap state of the front passenger head key point and the safety belt sector area, the safety belt wearing posture of the driver and front passenger users is detected. Thus, the two-stage detection of the safety belt wearing state and the safety belt wearing posture of the driver and front passenger users is realized, and the efficiency and accuracy of the safety belt wearing detection of the driver and front passenger users are further improved.

[0063] The technical scheme of the embodiment of the present application determines the main driver safety belt region, the co-driver safety belt region, the main driver head key point and the co-driver head key point by detecting the driving image in the vehicle, constructs a safety belt sector region according to the main driver safety belt region and the co-driver safety belt region, considers the geometric relationship between the safety belt region and the head key point based on the overlapping state of the main driver head key point and the safety belt sector region and the overlapping state of the co-driver head key point and the safety belt sector region, and can realize the detection of the safety belt wearing condition of the main and co-driver users, improve the accuracy of the safety belt wearing detection, ensure the safety of the vehicle driving process, and further improve the detection efficiency of the safety belt wearing detection by simultaneously judging the safety belt wearing condition of the main and co-driver users compared with respectively detecting the safety belt wearing of a single driving user.

[0064] In an optional embodiment of the present application, after determining the safety belt wearing detection result of the main and co-driver users according to the overlapping state of the main driver head key point and the safety belt sector region and the overlapping state of the co-driver head key point and the safety belt sector region, the method further comprises: determining the safety belt wearing reminding information according to the safety belt wearing detection result of the main and co-driver users, and feeding back the main and co-driver users.

[0065] The safety belt wearing reminding information is used to remind the main and co-driver users to adjust the safety belt wearing condition. Optionally, the safety belt wearing reminding information can include safety belt wearing state reminding information and / or safety belt wearing posture reminding information. Optionally, the safety belt wearing reminding information can include text information, voice information, image display information or sound and light information, etc. Exemplarily, the safety belt wearing state reminding information can be voice information of “please wear a safety belt ~”; the safety belt wearing posture reminding information can be voice information of “please adjust the safety belt wearing posture oh ~”.

[0066] Optionally, when determining the safety belt wearing abnormality of the main and co-driver users according to the overlapping state of the main driver head key point and the safety belt sector region and the overlapping state of the co-driver head key point and the safety belt sector region, the safety belt wearing reminding information can be determined as the safety belt wearing state reminding information and / or the safety belt wearing posture reminding information.

[0067] Optionally, when determining the safety belt wearing normality of the main and co-driver users according to the overlapping state of the main driver head key point and the safety belt sector region and the overlapping state of the co-driver head key point and the safety belt sector region, the safety belt wearing reminding information can be determined as empty.

[0068] The scheme determines the safety belt wearing reminding information according to the safety belt wearing detection results of the main driver and the copilot, and feeds back the main driver and the copilot, based on the safety belt wearing reminding information, realizes the timely reminding of the safety belt wearing situation of the main driver and the copilot, and further improves the safety of the main driver and the copilot in the vehicle driving process.

[0069] Embodiment two

[0070] Fig. 2 is a flow chart of a safety belt wearing detection method provided by the embodiment two of the present application. The embodiment of the present application further specifies the "constructing the safety belt sector region according to the main driver safety belt region and the copilot safety belt region" as "determining the main driver safety belt outer edge tangent according to the main driver safety belt region, and determining the copilot safety belt outer edge tangent according to the copilot safety belt region; determining the center of the safety belt sector region according to the intersection between the main driver safety belt outer edge tangent and the copilot safety belt outer edge tangent; constructing the safety belt sector region according to the main driver safety belt outer edge tangent, the copilot safety belt outer edge tangent and the center of the safety belt sector region", introduces the main driver safety belt outer edge tangent and the copilot safety belt outer edge tangent, improves the construction efficiency of the safety belt sector region, and further improves the efficiency of the safety belt wearing detection of the main driver and the copilot. It should be noted that the parts not described in detail in the embodiment of the present application can refer to the description of other embodiments.

[0071] Referring to the safety belt wearing detection method shown in Fig. 2, the method comprises the following steps:

[0072] S210, obtaining the in-vehicle driving image of the main driver and the copilot.

[0073] S220, detecting the in-vehicle driving image to determine the main driver safety belt region, the copilot safety belt region, the main driver head key point and the copilot head key point.

[0074] The main driver safety belt region is a safety belt region across the upper torso of the main driver; the copilot safety belt region is a safety belt region across the upper torso of the copilot.

[0075] S230, determining the main driver safety belt outer edge tangent according to the main driver safety belt region, and determining the copilot safety belt outer edge tangent according to the copilot safety belt region.

[0076] The safety belt fan-shaped area formed by the main driver safety belt area and the co-driver safety belt area is the associated area between the main driver safety belt area and the co-driver safety belt area; outside the safety belt fan-shaped area, it can be understood that the main driver safety belt area and the co-driver safety belt area are irrelevant areas, that is, areas that cannot be combined with the head key point for safety belt wearing detection. The main driver safety belt outer edge tangent line can be used as a boundary line between the main driver safety belt area and the irrelevant area of the safety belt wearing detection. The co-driver safety belt outer edge tangent line can be used as a boundary line between the co-driver safety belt area and the irrelevant area of the safety belt wearing detection. Compared with the linear fitting of the outer edge of the main driver safety belt area and the linear fitting of the outer edge of the co-driver safety belt area, the safety belt fan-shaped area is constructed based on the main driver safety belt outer edge tangent line and the co-driver safety belt outer edge tangent line, which is simpler and faster.

[0077] Specifically, a tangent line can be drawn on the outer edge of the main driver safety belt area to obtain at least one main driver safety belt outer edge tangent line. And a tangent line can be drawn on the outer edge of the co-driver safety belt area to obtain at least one co-driver safety belt outer edge tangent line.

[0078] In an optional embodiment of the present application, the main driver safety belt outer edge tangent line is determined according to the main driver safety belt area, comprising: determining the main driver safety belt outermost edge tangent point according to the main driver safety belt area; determining the main driver safety belt outer edge tangent line according to the tangent line between the main driver safety belt outermost edge tangent point and the outer edge of the main driver safety belt area.

[0079] The main driver safety belt outermost edge tangent point can be the outermost tangent point on the outer edge of the main driver safety belt area. The main driver safety belt area outermost edge tangent point can be used to represent the outermost boundary of the main driver safety belt area.

[0080] Specifically, the points on the outer edge of the main driver safety belt area can be compared, and the outermost point can be determined as the main driver safety belt outermost edge tangent point. A tangent line can be drawn on the outer edge of the main driver safety belt area along the main driver safety belt outermost edge tangent point to obtain the main driver safety belt outer edge tangent line.

[0081] The present scheme introduces the main driver safety belt outermost edge tangent point. Compared with determining the main driver safety belt outer edge tangent line according to all points on the outer edge of the main driver safety belt area and then constructing the safety belt fan-shaped area, the present scheme can directly determine the main driver safety belt outer edge tangent line according to the main driver safety belt outermost edge tangent point, which can reduce the calculation amount of the determination process of the main driver safety belt outer edge tangent line and the construction process of the safety belt fan-shaped area, and improve the determination efficiency of the main driver safety belt outer edge tangent line.

[0082] Correspondingly, determining the tangent line of the outer edge of the co-driver safety belt according to the co-driver safety belt region comprises: determining the outermost edge tangent point of the co-driver safety belt according to the co-driver safety belt region; and determining the tangent line of the outer edge of the co-driver safety belt according to the tangent line between the outermost edge tangent point of the co-driver safety belt and the outer edge of the co-driver safety belt region.

[0083] The outermost edge tangent point of the co-driver safety belt can be the outermost tangent point on the outer edge of the co-driver safety belt region. The outermost edge tangent point of the co-driver safety belt region can be used to represent the outermost boundary of the co-driver safety belt region.

[0084] Specifically, the points on the outer edge of the co-driver safety belt region can be compared, and the outermost point is determined as the outermost edge tangent point of the co-driver safety belt. A tangent line can be drawn along the outermost edge tangent point of the co-driver safety belt to the outer edge of the co-driver safety belt region to obtain the tangent line of the outer edge of the co-driver safety belt.

[0085] By introducing the outermost edge tangent point of the co-driver safety belt, compared with determining the tangent line of the outer edge of the co-driver safety belt according to all points on the outer edge of the co-driver safety belt region, and then constructing the safety belt sector region, the present scheme can directly determine the tangent line of the outer edge of the co-driver safety belt according to the outermost edge tangent point of the co-driver safety belt, which can reduce the calculation amount of the determination process of the tangent line of the outer edge of the co-driver safety belt and the construction process of the safety belt sector region, and improve the determination efficiency of the tangent line of the outer edge of the co-driver safety belt.

[0086] In an optional embodiment of the present application, determining the tangent line of the outer edge of the main driver safety belt according to the tangent line between the outermost edge tangent point of the main driver safety belt and the outer edge of the main driver safety belt region comprises: detecting at least two tangent lines between the outermost edge tangent point of the main driver safety belt and the outer edge of the main driver safety belt region to obtain at least two main driver candidate tangent lines; obtaining the tangent line of the outer edge of the co-driver safety belt; detecting the intersection angle between each main driver candidate tangent line and the tangent line of the outer edge of the co-driver safety belt; and taking the main driver candidate tangent line corresponding to the minimum intersection angle as the tangent line of the outer edge of the main driver safety belt.

[0087] The number of tangent lines between the outermost edge intersection point of the driver's seat safety belt and the outer edge of the driver's seat safety belt region can be at least two. It can be understood that the number of outermost edge intersection points of the driver's seat safety belt is at least two, or due to the shape of the outer edge of the driver's seat safety belt region, one outermost edge intersection point of the driver's seat safety belt corresponds to at least two tangent lines. The driver's seat alternative tangent line can be used to screen the outer edge tangent line of the driver's seat safety belt. The intersection angle can be the angle between each driver's seat alternative tangent line and the outer edge tangent line of the co-driver's seat safety belt. The intersection angle can be used to represent the size of the sector region formed by the driver's seat alternative tangent line and the outer edge tangent line of the co-driver's seat safety belt. It can be understood that the larger the intersection angle, the larger the sector region formed by the driver's seat alternative tangent line and the outer edge tangent line of the co-driver's seat safety belt; the smaller the intersection angle, the smaller the sector region formed by the driver's seat alternative tangent line and the outer edge tangent line of the co-driver's seat safety belt. The driver's seat alternative tangent line corresponding to the minimum intersection angle can be understood as the smallest driver's seat alternative tangent line formed by the sector region, that is, the driver's seat alternative tangent line corresponding to the safety belt sector region.

[0088] Specifically, at least two tangent lines between the outermost edge intersection point of the driver's seat safety belt and the outer edge of the driver's seat safety belt region can be detected to obtain at least two driver's seat alternative tangent lines. The outer edge tangent line of the co-driver's seat safety belt can be obtained. The intersection angle between each driver's seat alternative tangent line and the outer edge tangent line of the co-driver's seat safety belt can be detected. The driver's seat alternative tangent line corresponding to the minimum intersection angle can be taken as the outer edge tangent line of the driver's seat safety belt.

[0089] Optionally, at least two tangent lines between the outermost edge tangent point of the front passenger safety belt and the outer edge of the front passenger safety belt region can be detected to obtain at least two front passenger candidate tangent lines. The outer edge tangent line of the driver safety belt can be obtained. The intersection angle between each front passenger candidate tangent line and the outer edge tangent line of the driver safety belt can be detected. The front passenger candidate tangent line corresponding to the minimum intersection angle is taken as the outer edge tangent line of the front passenger safety belt. The number of tangent lines between the outermost edge tangent point of the front passenger safety belt and the outer edge of the front passenger safety belt region can be at least two. It can be understood that the number of outermost edge tangent points of the front passenger safety belt is at least two, or due to the shape of the outer edge of the front passenger safety belt region, one outermost edge tangent point of the front passenger safety belt corresponds to at least two tangent lines. The front passenger candidate tangent line can be used to screen the outer edge tangent line of the front passenger safety belt. The intersection angle can be the angle between the front passenger candidate tangent line and the outer edge tangent line of the driver safety belt. The intersection angle can be used to represent the size of the sector region formed by the front passenger candidate tangent line and the outer edge tangent line of the driver safety belt. It can be understood that the larger the intersection angle, the larger the sector region formed by the front passenger candidate tangent line and the outer edge tangent line of the driver safety belt; the smaller the intersection angle, the smaller the sector region formed by the front passenger candidate tangent line and the outer edge tangent line of the driver safety belt. The front passenger candidate tangent line corresponding to the minimum intersection angle can be understood as the smallest front passenger candidate tangent line of the sector region formed, that is, the front passenger candidate tangent line corresponding to the safety belt sector region.

[0090] Optionally, at least two tangent lines between the outermost edge tangent point of the driver safety belt and the outer edge of the driver safety belt region can be detected to obtain at least two driver candidate tangent lines. At least two tangent lines between the outermost edge tangent point of the front passenger safety belt and the outer edge of the front passenger safety belt region can be detected to obtain at least two front passenger candidate tangent lines. The intersection angle between each driver candidate tangent line and each front passenger candidate tangent line can be detected. The driver candidate tangent line and the front passenger candidate tangent line corresponding to the minimum intersection angle are taken as the outer edge tangent line of the driver safety belt and the outer edge tangent line of the front passenger safety belt. The intersection angle can be the angle between each driver candidate tangent line and each front passenger candidate tangent line. The intersection angle can be used to represent the size of the sector region formed by the driver candidate tangent line and the front passenger candidate tangent line. It can be understood that the larger the intersection angle, the larger the sector region formed by the driver candidate tangent line and the front passenger candidate tangent line; the smaller the intersection angle, the smaller the sector region formed by the driver candidate tangent line and the front passenger candidate tangent line. The driver candidate tangent line and the front passenger candidate tangent line corresponding to the minimum intersection angle can be understood as the smallest driver candidate tangent line and front passenger candidate tangent line of the sector region formed, that is, the driver candidate tangent line and the front passenger candidate tangent line corresponding to the safety belt sector region.

[0091] Optionally, at least two tangent lines between the outermost edge tangent point of the main driver safety belt and the outer edge of the main driver safety belt region can be detected to obtain at least two main driver candidate tangent lines. At least two tangent lines between the outermost edge tangent point of the co-driver safety belt and the outer edge of the co-driver safety belt region can be detected to obtain at least two co-driver candidate tangent lines. The intersection angle between each main driver candidate tangent line and each co-driver candidate tangent line can be detected, and the main driver candidate tangent line and the co-driver candidate tangent line corresponding to the minimum intersection angle can be determined as the main driver safety belt outer edge tangent line and the co-driver safety belt outer edge tangent line.

[0092] The present scheme provides a case where there are at least two tangent lines between the outermost edge tangent point of the main driver safety belt and the outer edge of the main driver safety belt region, introduces the intersection angle between the main driver candidate tangent line and the co-driver safety belt outer edge tangent line, and determines the main driver safety belt outer edge tangent line based on the minimum intersection angle, thereby improving the fault tolerance and applicability of the safety belt wearing detection method.

[0093] S240, determining the center of the safety belt sector region according to the intersection point between the main driver safety belt outer edge tangent line and the co-driver safety belt outer edge tangent line.

[0094] Specifically, the center of the safety belt sector region can be determined based on the intersection point between the main driver safety belt outer edge tangent line and the co-driver safety belt outer edge tangent line. Optionally, the center of each safety belt sector region and the main driver safety belt outer edge tangent line and the co-driver safety belt outer edge tangent line corresponding to the center of each safety belt sector region can be determined based on the intersection point between each main driver safety belt outer edge tangent line and each co-driver safety belt outer edge tangent line.

[0095] S250, constructing the safety belt sector region according to the main driver safety belt outer edge tangent line, the co-driver safety belt outer edge tangent line, and the center of the safety belt sector region.

[0096] Specifically, the main driver safety belt outer edge tangent line, the co-driver safety belt outer edge tangent line, and the center of the safety belt sector region can be connected. The radius of the safety belt sector region can be adjusted, and it can be detected whether the sector region corresponding to the radius of the different safety belt sector region contains the main driver safety belt region and the co-driver safety belt. When the sector region contains the main driver safety belt region and the co-driver safety belt region, the sector region is determined as a candidate sector region. Each candidate sector region can be compared, and the candidate sector region with the smallest area is determined as the safety belt sector region.

[0097] In an optional embodiment of the present application, the safety belt sector region is constructed according to the tangent of the outer edge of the driver's safety belt, the tangent of the outer edge of the co-driver's safety belt, and the center of the safety belt sector region, including: calculating the distance between the outermost edge tangent point of the driver's safety belt and the center of the safety belt sector region to obtain the driver's tangent point distance; calculating the distance between the outermost edge tangent point of the co-driver's safety belt and the center of the safety belt sector region to obtain the co-driver's tangent point distance; calculating the sum of the driver's tangent point distance and the co-driver's tangent point distance to obtain the radius of the safety belt sector region; and constructing the safety belt sector region according to the radius of the safety belt region and the center of the safety belt region.

[0098] The driver's tangent point distance can be the distance between the outermost edge tangent point of the driver's safety belt and the center of the safety belt sector region. The co-driver's tangent point distance can be the distance between the outermost edge tangent point of the co-driver's safety belt and the center of the safety belt sector region. In essence, the safety belt sector region is formed based on the spatial correlation of the driver's safety belt region and the co-driver's safety belt region. Determining the radius of the safety belt sector region based on the sum of the driver's tangent point distance and the co-driver's tangent point distance can be understood as comprehensively considering the geometric information of the driver's safety belt region and the geometric information of the co-driver's safety belt region, and the obtained radius of the safety belt sector region is more referential.

[0099] Specifically, the distance between the outermost edge tangent point of the driver's safety belt and the center of the safety belt sector region can be calculated to obtain the driver's tangent point distance. The distance between the outermost edge tangent point of the co-driver's safety belt and the center of the safety belt sector region can be calculated to obtain the co-driver's tangent point distance. The sum of the driver's tangent point distance and the co-driver's tangent point distance can be calculated to obtain the radius of the safety belt sector region. The safety belt sector region is constructed according to the radius of the safety belt region and the center of the safety belt region.

[0100] The present scheme introduces the driver's tangent point distance and the co-driver's tangent point distance, determines the radius of the safety belt sector region based on the sum of the driver's tangent point distance and the co-driver's tangent point distance, and then constructs the safety belt sector region, taking into account the geometric correlation of the driver's safety belt region and the co-driver's safety belt region, thereby improving the accuracy of safety belt wearing detection. At the same time, the construction process of the safety belt sector region is simplified, further improving the construction efficiency of the safety belt sector region and the efficiency of safety belt wearing detection.

[0101] Optionally, the driver's tangent point distance and the co-driver's tangent point distance can be compared, and twice the larger value is determined as the radius of the safety belt sector region. The safety belt sector region can be constructed according to the radius of the safety belt region and the center of the safety belt region. In this way, the situation of missed detection caused by the safety belt sector region being too small can be avoided, and the comprehensiveness of safety belt wearing detection is improved.

[0102] Optionally, the main driver cut point distance and the co-driver cut point distance are compared, and twice the smaller value is determined as the radius of the safety belt sector region. The safety belt sector region can be constructed according to the radius of the safety belt region and the center of the safety belt region. In this way, the safety belt sector region can be reduced, and the detection accuracy based on safety belt wearing detection can be improved.

[0103] S260, according to the overlap state of the main driver head key point and the safety belt sector region, and the overlap state of the co-driver head key point and the safety belt sector region, determine the safety belt wearing detection result of the main and co-driver users.

[0104] The technical scheme of the embodiment of the application introduces the main driver safety belt outer edge tangent and the co-driver safety belt outer edge tangent. Compared with the linear fitting of the outer edge of the main driver safety belt region and the linear fitting of the outer edge of the co-driver safety belt region, the way of constructing the safety belt sector region based on the main driver safety belt outer edge tangent and the co-driver safety belt outer edge tangent is simpler and faster, which can improve the construction efficiency of the safety belt sector region, and further improve the efficiency of the safety belt wearing detection of the main and co-driver users.

[0105] Fig. 3 is a flowchart of a safety belt wearing detection method according to an embodiment of the application. On the basis of the above-mentioned embodiment, Fig. 3 is a preferred embodiment of the application. Referring to the safety belt wearing detection method shown in Fig. 3, it comprises:

[0106] S310, acquiring a real-time video stream in the vehicle.

[0107] The real-time video stream in the vehicle is the in-vehicle driving video.

[0108] Specifically, the real-time video stream in the vehicle monitored by the camera in the vehicle can be acquired in real time. The real-time video stream in the vehicle can be processed frame by frame into in-vehicle driving images. Exemplarily, the camera in the vehicle can be a high-resolution camera, for example, a DMS camera (Driver Monitoring System Camera). The installation position of the image acquisition device can include the A-pillar in the vehicle, the reading light, or other front positions that can detect images of the upper torso and abdominal regions of the main driver and the co-driver, etc. Optionally, considering different lighting conditions, the image acquisition device can be equipped with an adaptive exposure adjustment function to ensure that clear in-vehicle driving images can be obtained in different environments such as daylight, shadow, or night.

[0109] S320, detecting a main driver safety belt region and a co-driver safety belt region.

[0110] Specifically, a pre-trained safety belt area detection model can be used to detect the in-vehicle driving image, and output the main driver safety belt area and the co-driver safety belt area. The safety belt area detection model can be a semantic segmentation model. For example, the semantic segmentation model can be an HRNET (a deep neural network model for image recognition tasks).

[0111] For example, Fig. 4 is a schematic diagram of safety belt area detection input data and output results. As shown in Fig. 4, the black area is the segmented main driver safety belt area and co-driver safety belt area. The present scheme only uses the safety belt that diagonally crosses the chest of the driving user, and does not need to consider the horizontal safety belt that blocks the abdomen.

[0112] S330, judge whether the main driver safety belt area and the co-driver safety belt area meet the position and area conditions, if yes, execute S340; if no, execute S370.

[0113] Specifically, whether the main driver safety belt area and the co-driver safety belt area exist can be detected, and the area of the main driver safety belt area and the area of the co-driver safety belt area can be calculated. If two independent main driver safety belt areas and co-driver safety belt areas are detected, and the areas of the two regions (i.e. the main driver safety belt area and the co-driver safety belt area) are both greater than or equal to a pre-set reference safety belt area, it is considered that the position and area conditions are met. If two independent main driver safety belt areas and co-driver safety belt areas are not detected, or the area of any of the two regions is less than the pre-set reference safety belt area, it is considered that the position and area conditions are not met, and it is determined that the main and co-driver users do not wear safety belts. The reference safety belt area can be a fixed value, such as 300 or 500, or a certain ratio, such as 1 / 30 of the entire in-vehicle driving image.

[0114] S340, construct a safety belt sector area.

[0115] Specifically, the safety belt sector area can be constructed according to the main driver safety belt area and the co-driver safety belt area.

[0116] For example, Fig. 5 is a schematic diagram of constructing a safety belt sector area. As shown in Fig. 5, the following steps can be used to construct the safety belt sector area.

[0117] S3401, determine the center of the circle.

[0118] Specifically, the outermost edge points of the two regions can be identified to obtain the outermost edge tangent point of the driver's seat safety belt and the outermost edge tangent point of the front passenger's seat safety belt. A tangent line of the region can be drawn through the point. If the tangent line is not unique, the tangent line corresponding to the minimum value between the relative angles formed by the two lines is taken as the tangent line of the outer edge of the driver's seat safety belt and the tangent line of the outer edge of the front passenger's seat safety belt.

[0119] For example, as shown in FIG. 5, the two green dots in the chest are the safety belt region tangent points (i.e., the outermost edge tangent point of the driver's seat safety belt and the outermost edge tangent point of the front passenger's seat safety belt); the two purple lines are the tangent lines of the outer edges of the driver's seat safety belt and the front passenger's seat safety belt, and the black dot at which the two lines intersect is the center of the circle.

[0120] S3403, determine the radius.

[0121] For example, the following formula can be used to determine the radius of the safety belt sector region: R = L1 + L2.

[0122] In the formula, R is the radius of the safety belt sector region, L1 is the distance of the driver's seat tangent point, and L2 is the distance of the front passenger's seat tangent point.

[0123] S3403, construct a safety belt sector region.

[0124] Specifically, the safety belt sector region can be constructed according to the center and radius of the safety belt sector region. As shown in FIG. 5, the circle is tangent to the two tangent lines to form a sector, such as the region formed by the purple line segment and the yellow line segment.

[0125] S350, identify the driver's eye key points and the front passenger's eye key points.

[0126] Specifically, a pre-trained head key point detection model can be used to detect the in-vehicle driving image to output the driver's head key points and the front passenger's head key points. For example, the head key point detection model can be Center Net (a center point detection model based on a convolutional neural network). The red points shown in FIG. 5 are the driver's eye key points and the front passenger's eye key points.

[0127] S360, determine whether the safety belt sector region contains the driver's eye key points and the front passenger's eye key points. If yes, it is determined that the driver and the front passenger have worn the safety belt and the safety belt wearing posture is correct, and the next frame of in-vehicle driving image is returned to S310; if no, S370 is performed.

[0128] The eye, as the part closest to the top of the head among the five human organs, its position can directly reflect the posture and movement state of the head. If the eye key points are located outside the safety belt fan-shaped area, it may mean that the driving user does not wear the safety belt correctly, and the safety belt cannot correctly constrain the body of the driving user, and cannot play its protective role. In the event of a collision, due to the lack of correct restraint, the driving user may be thrown out of the vehicle, increasing the risk of injury. In actual projects, through a series of experiments, the situation of vehicle collision is simulated, through the detection of the head movement trajectory and eye key points of the driving user under different safety belt wearing postures, and the comparison of experimental data, the correlation between the eye key points in the safety belt fan-shaped area and the correct wearing of the safety belt is verified.

[0129] For example, as shown in FIG. 6, it can be judged whether the four eye key points are all in the safety belt fan-shaped area. If yes, it proves that the main driving user and the co-pilot user both wear the safety belt and the wearing posture is correct; if no, it proves that at least one of the main driving user and the co-pilot user has an incorrect safety belt wearing posture.

[0130] S370, issuing a safety belt wearing reminding information to the main and co-pilot users.

[0131] The safety belt wearing reminding information can include sound reminding information or flash light reminding information, etc.

[0132] Specifically, if it is detected that the main and co-pilot users do not wear the safety belt or have an incorrect safety belt wearing posture, a safety belt wearing reminding information is issued to the main and co-pilot users to remind the main and co-pilot users to adjust the safety belt wearing situation, so as to improve the driving safety.

[0133] The scheme simultaneously realizes the safety belt wearing posture judgment of the main and vice drivers, compared with the prior art, the scheme is not to respectively judge the safety belt wearing posture of the main driver or the vice driver, but establishes a link between the main driver and the vice driver, and simultaneously realizes the detection of the safety belt wearing posture of the two people of the main driver and the vice driver, thereby reducing the calculation amount of the safety belt wearing detection process; only when the main driver and the vice driver both wear correctly, it is determined that the safety belt wearing posture of the main and vice drivers is correct, and therefore, the safety requirement is higher; at the same time, the safety belt wearing posture of the main and vice drivers is judged based on the overlap state of the eye key point and the safety belt fan-shaped area, the eye key point in the safety belt fan-shaped area is taken as the judgment basis of whether the safety belt can play a role, the geometric relationship between the main driver and the vice driver is established in the function application, thereby simplifying the judgment process of the safety belt wearing detection. Fig. 7 is a schematic diagram of the safety belt wearing posture error. As shown in Fig. 7, in this common error wearing posture, since the safety belt does not pass in front of the chest, the overall height of the safety belt will be lower compared with the correct wearing posture, using the safety belt wearing detection method of the scheme, the safety belt fan-shaped area constructed will sweep from the left and right of the chin, the eye key point is outside the safety belt fan-shaped area, thereby triggering the alarm, proving the effectiveness of the safety belt wearing detection method proposed in the scheme.

[0134] Embodiment three

[0135] Fig. 8 is a structural schematic diagram of a safety belt wearing detection device provided by the embodiment three of the present application. The embodiment of the present application can be applicable to the case of jointly detecting the safety belt wearing situation of the main and vice drivers, the device can execute the safety belt wearing detection method, the device can be realized in the form of hardware and / or software, and the device can be configured in an electronic device carrying the safety belt wearing detection function, such as a vehicle-mounted terminal.

[0136] Referring to the safety belt wearing detection device shown in FIG. 8, it comprises: a driver-in-vehicle image acquisition module 810, a safety belt area and key point detection module 820, a safety belt sector area construction module 830, and a safety belt wearing detection module 840. Among them, the driver-in-vehicle image acquisition module 810 is used to acquire the driver-in-vehicle image of the main and vice drivers; the safety belt area and key point detection module 820 is used to detect the driver-in-vehicle image to determine the main driver safety belt area, the vice driver safety belt area, the main driver head key point and the vice driver head key point; wherein the main driver safety belt area is the safety belt area across the upper torso of the main driver user; the vice driver safety belt area is the safety belt area across the upper torso of the vice driver user; the safety belt sector area construction module 830 is used to construct the safety belt sector area according to the main driver safety belt area and the vice driver safety belt area; the safety belt wearing detection module 840 is used to determine the safety belt wearing detection result of the main and vice drivers according to the overlapping state of the main driver head key point and the safety belt sector area, and the overlapping state of the vice driver head key point and the safety belt sector area.

[0137] The technical scheme of the embodiment of the application determines the main driver safety belt area, the vice driver safety belt area, the main driver head key point and the vice driver head key point by detecting the driver-in-vehicle image, constructs the safety belt sector area according to the main driver safety belt area and the vice driver safety belt area, and considers the geometric relationship between the safety belt area and the head key point based on the overlapping state of the main driver head key point and the safety belt sector area, and the overlapping state of the vice driver head key point and the safety belt sector area, so that the safety belt wearing situation of the main and vice drivers can be detected, the accuracy of the safety belt wearing detection is improved, the safety of the vehicle driving process is ensured, and compared with the safety belt wearing detection of the single driver, the detection efficiency of the safety belt wearing detection can be further improved by simultaneously judging the safety belt wearing situation of the main and vice drivers.

[0138] In an optional embodiment of the application, the safety belt sector area construction module 830 comprises: a safety belt outer edge tangent line determination unit, which is used to determine the main driver safety belt outer edge tangent line according to the main driver safety belt area, and determine the vice driver safety belt outer edge tangent line according to the vice driver safety belt area; a center determination unit, which is used to determine the center of the safety belt sector area according to the intersection between the main driver safety belt outer edge tangent line and the vice driver safety belt outer edge tangent line; and a safety belt sector area construction unit, which is used to construct the safety belt sector area according to the main driver safety belt outer edge tangent line, the vice driver safety belt outer edge tangent line and the center of the safety belt sector area.

[0139] In an optional embodiment of the present application, the safety belt outer edge tangent line determination unit comprises: a main driver safety belt outermost edge tangent point subunit configured to determine a main driver safety belt outermost edge tangent point according to a main driver safety belt region; and a main driver safety belt outer edge tangent line determination subunit configured to determine a main driver safety belt outer edge tangent line according to a tangent line between the main driver safety belt outermost edge tangent point and an outer edge of the main driver safety belt region.

[0140] In an optional embodiment of the present application, the main driver safety belt outer edge tangent line determination subunit comprises: a main driver alternative tangent line determination subunit configured to detect at least two tangent lines between the main driver safety belt outermost edge tangent point and the outer edge of the main driver safety belt region, to obtain at least two main driver alternative tangent lines; a co-driver safety belt outer edge tangent line acquisition subunit configured to acquire a co-driver safety belt outer edge tangent line; an intersection angle detection subunit configured to detect an intersection angle between each main driver alternative tangent line and the co-driver safety belt outer edge tangent line; and a main driver safety belt outer edge tangent line determination subunit configured to take the main driver alternative tangent line corresponding to the minimum intersection angle as the main driver safety belt outer edge tangent line.

[0141] In an optional embodiment of the present application, the safety belt sector region construction unit comprises: a main driver tangent point distance determination subunit configured to calculate a distance between the main driver safety belt outermost edge tangent point and a center of the safety belt sector region, to obtain a main driver tangent point distance; a co-driver tangent point distance determination subunit configured to calculate a distance between the co-driver safety belt outermost edge tangent point and the center of the safety belt sector region, to obtain a co-driver tangent point distance; a radius determination subunit configured to calculate a sum of the main driver tangent point distance and the co-driver tangent point distance, to obtain a radius of the safety belt sector region; and a safety belt sector region construction subunit configured to construct the safety belt sector region according to the radius of the safety belt region and the center of the safety belt region.

[0142] In an optional embodiment of the present application, the main driver head key point comprises a main driver eye key point, a main driver nose key point, a main driver mouth key point or a main driver head center point; and the co-driver head key point comprises a co-driver eye key point, a co-driver nose key point, a co-driver mouth key point or a co-driver head center point.

[0143] In an optional embodiment of the present application, the device further comprises: a seat belt area detection module, configured to detect the driver seat belt area and the front passenger seat belt area before determining the seat belt wearing detection result of the driver and the front passenger according to the overlap state of the driver head key point and the seat belt sector area and the overlap state of the front passenger head key point and the seat belt sector area; a seat belt wearing state detection module, configured to determine the seat belt wearing state detection result of the driver according to the area of the driver seat belt area when the driver seat belt area and the front passenger seat belt area both exist, and determine the seat belt wearing state detection result of the front passenger according to the area of the front passenger seat belt area; and a seat belt wearing detection module, comprising: a seat belt wearing posture detection unit, configured to determine the seat belt wearing posture detection result of the driver and the front passenger according to the overlap state of the driver head key point and the seat belt sector area and the overlap state of the front passenger head key point and the seat belt sector area when the driver has worn the seat belt and the front passenger has worn the seat belt.

[0144] In an optional embodiment of the present application, the device further comprises: a wearing reminding module, configured to determine the seat belt wearing reminding information according to the seat belt wearing detection result of the driver and the front passenger after determining the seat belt wearing detection result of the driver and the front passenger according to the overlap state of the driver head key point and the seat belt sector area and the overlap state of the front passenger head key point and the seat belt sector area, and feed back the driver and the front passenger.

[0145] The seat belt wearing detection device provided by the embodiments of the present application can execute the seat belt wearing detection method provided by any of the embodiments of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0146] In the technical solutions of the embodiments of the present application, the acquisition, storage and application of the in-vehicle driving images of the driver and the front passenger and the like all conform to the relevant legal regulations and do not violate public order and good customs.

[0147] Embodiment Four

[0148] FIG. 9 shows a diagram of the structure of an electronic device 900 that can be used to implement embodiments of the present application. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smartphones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present application described and / or claimed in this document.

[0149] As shown in FIG. 9, the electronic device 900 includes at least one processor 901, and a memory, such as a read-only memory (ROM) 902, a random access memory (RAM) 903, and the like, which is communicatively connected to the at least one processor 901, wherein the memory stores a computer program that can be executed by the at least one processor. The processor 901 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 902 or loaded into the random access memory (RAM) 903 from the storage unit 908. In the RAM 903, various programs and data required for the operation of the electronic device 900 can also be stored. The processor 901, the ROM 902, and the RAM 903 are connected to each other through a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0150] A plurality of components in the electronic device 900 are connected to the I / O interface 905, including an input unit 906, such as a keyboard, a mouse, and the like, an output unit 907, such as various types of displays, a speaker, and the like, a storage unit 908, such as a magnetic disk, an optical disk, and the like, and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 909 allows the electronic device 900 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0151] The processor 901 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, and the like. The processor 901 performs various methods and processes described above, such as the seat belt wearing detection method.

[0152] In some embodiments, the seat belt wearing detection method can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the processor 901, one or more steps of the seat belt wearing detection method described above can be performed. Alternatively, in other embodiments, the processor 901 can be configured to perform the seat belt wearing detection method by any other appropriate means, such as by means of firmware.

[0153] The various embodiments of the systems and techniques described above can be implemented in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0154] Computer programs used to implement the processes of the application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer program

[0155] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store computer programs for use by or in connection with an instruction execution system, apparatus, or device. Computer-readable storage media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0156] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0157] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), blockchain network, and the Internet.

[0158] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. A server can be a cloud server, also known as cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS (Virtual Private Server).

[0159] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the scope of the present disclosure. For example, the steps recited in the present disclosure can be executed in parallel, executed in sequence, or executed in a different order, as long as the desired results of the present disclosure are achieved, and the present disclosure is not limited herein.

[0160] The above detailed description does not limit the scope of the application. Various modifications, combinations, sub-combinations and alternatives can be made to the detailed embodiment within the scope of the application. Any modification, equivalent replacement and improvement made without departing from the spirit and principle of the application shall fall within the scope of the application.

Claims

1. A seat belt wearing detection method characterized by comprising: The method comprises: acquiring an in-vehicle driving image of a main driver and a co-driver; detecting the in-vehicle driving image to determine a main driver seat belt region, a co-driver seat belt region, a main driver head key point and a co-driver head key point; wherein the main driver seat belt region is a seat belt region across the upper torso of the main driver; the co-driver seat belt region is a seat belt region across the upper torso of the co-driver; constructing a seat belt sector region according to the main driver seat belt region and the co-driver seat belt region; determining a seat belt wearing detection result of the main driver and the co-driver according to an overlapping state of the main driver head key point and the seat belt sector region, and an overlapping state of the co-driver head key point and the seat belt sector region.

2. The method of claim 1, wherein, The method of constructing a seat belt sector region according to the main driver seat belt region and the co-driver seat belt region comprises: determining a main driver seat belt outer edge tangent according to the main driver seat belt region, and determining a co-driver seat belt outer edge tangent according to the co-driver seat belt region; determining a center of the seat belt sector region according to an intersection between the main driver seat belt outer edge tangent and the co-driver seat belt outer edge tangent; constructing the seat belt sector region according to the main driver seat belt outer edge tangent, the co-driver seat belt outer edge tangent and the center of the seat belt sector region.

3. The method of claim 2, wherein, The method of determining a main driver seat belt outer edge tangent according to the main driver seat belt region comprises: determining a main driver seat belt outermost edge tangent point according to the main driver seat belt region; determining a main driver seat belt outer edge tangent according to a tangent line between the main driver seat belt outermost edge tangent point and an outer edge of the main driver seat belt region.

4. The method of claim 3, wherein, The method of determining a main driver seat belt outer edge tangent according to a tangent line between the main driver seat belt outermost edge tangent point and an outer edge of the main driver seat belt region comprises: detecting at least two tangent lines between the main driver seat belt outermost edge tangent point and the outer edge of the main driver seat belt region to obtain at least two main driver candidate tangents; acquiring a co-driver seat belt outer edge tangent; detecting an intersection angle between each of the main driver candidate tangents and the co-driver seat belt outer edge tangent; taking the main driver candidate tangent corresponding to the minimum intersection angle as the main driver seat belt outer edge tangent.

5. The method of claim 2, wherein, The method of constructing a seat belt sector region according to the main driver seat belt outer edge tangent, the co-driver seat belt outer edge tangent and the center of the seat belt sector region comprises: calculating a distance between the main driver seat belt outermost edge tangent point and the center of the seat belt sector region to obtain a main driver tangent point distance; calculating a distance between the co-driver seat belt outermost edge tangent point and the center of the seat belt sector region to obtain a co-driver tangent point distance; calculating a sum of the main driver tangent point distance and the co-driver tangent point distance to obtain a radius of the seat belt sector region; constructing the seat belt sector region according to the radius of the seat belt region and the center of the seat belt region.

6. The method of claim 1, wherein, The main driver head key point comprises a main driver eye key point, a main driver nose key point, a main driver mouth key point or a main driver head center point. The co-driver head key point comprises a co-driver eye key point, a co-driver nose key point, a co-driver mouth key point or a co-driver head center point.

7. The method of claim 1, wherein, Before the step of determining the seat belt wearing detection result of the main and co-driver users according to the overlap state of the main driver head key point and the seat belt sector area and the overlap state of the co-driver head key point and the seat belt sector area, the method further comprises: detecting the main driver seat belt area and the co-driver seat belt area; when the main driver seat belt area and the co-driver seat belt area both exist, determining the seat belt wearing state detection result of the main driver user according to the area of the main driver seat belt area, and determining the seat belt wearing state detection result of the co-driver user according to the area of the co-driver seat belt area; the step of determining the seat belt wearing detection result of the main and co-driver users according to the overlap state of the main driver head key point and the seat belt sector area and the overlap state of the co-driver head key point and the seat belt sector area comprises: when the main driver user has worn the seat belt and the co-driver user has worn the seat belt, determining the seat belt wearing posture detection result of the main and co-driver users according to the overlap state of the main driver head key point and the seat belt sector area and the overlap state of the co-driver head key point and the seat belt sector area.

8. The method of claim 1, wherein, After the step of determining the seat belt wearing detection result of the main and co-driver users according to the overlap state of the main driver head key point and the seat belt sector area and the overlap state of the co-driver head key point and the seat belt sector area, the method further comprises: determining the seat belt wearing reminding information according to the seat belt wearing detection result of the main and co-driver users, and feeding back the main and co-driver users.

9. A seat belt wearing detection device characterized by comprising: The device comprises: an in-vehicle driving image acquisition module configured to acquire in-vehicle driving images of main and co-driver users; a seat belt area and key point detection module configured to detect the in-vehicle driving images to determine a main driver seat belt area, a co-driver seat belt area, a main driver head key point and a co-driver head key point, wherein the main driver seat belt area is a seat belt area crossing an upper torso of the main driver user, and the co-driver seat belt area is a seat belt area crossing an upper torso of the co-driver user; a seat belt sector area construction module configured to construct a seat belt sector area according to the main driver seat belt area and the co-driver seat belt area; a seat belt wearing detection module configured to determine a seat belt wearing detection result of the main and co-driver users according to an overlap state of the main driver head key point and the seat belt sector area and an overlap state of the co-driver head key point and the seat belt sector area.

10. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory connected with the at least one processor in communication; wherein The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the safety belt wearing detection method in any one of claims 1-8.

11. A computer readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for enabling the processor to implement the safety belt wearing detection method in any one of claims 1-8 when executed.

12. A computer program product, characterised in that, The computer program product comprises a computer program which, when executed by the processor, implements the safety belt wearing detection method according to any one of claims 1-8.

Images