Seat belt wearing detection method and apparatus, electronic device, and storage medium

Abstract

The present invention relates to the technical field of vehicle driving. Disclosed are a seat belt wearing detection method and apparatus, an electronic device, and a storage medium. The method comprises: acquiring an in-vehicle image of a vehicle occupant captured during driving and a seat belt pixel point range in real time; detecting the in-vehicle image captured during driving, and determining a body key line segment of the vehicle occupant and a seat belt area; detecting the number of seat belt pixel points where the body key line segment of the vehicle occupant overlaps with the seat belt area; and determining a seat belt wearing detection result of the vehicle occupant on the basis of the number of the seat belt pixel points and the seat belt pixel point range. The technical solution in embodiments of the present invention achieves detection of an actual seat belt wearing condition of the vehicle occupant, ensuring the safety of the vehicle occupant during vehicle running.

Description

Safety belt wearing detection method and device, electronic equipment and storage medium TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle driving, and in particular to a safety belt wearing detection method and device, electronic equipment and storage medium. BACKGROUND

[0002] In order to ensure the safety of the vehicle user, the safety belt is particularly important during vehicle driving.

[0003] The traditional safety belt wearing detection method mainly relies on mechanical sensors. Through the mechanical sensors, it can be detected whether the safety belt has been inserted into the lock buckle. However, this safety belt wearing detection method cannot ensure that the vehicle user actually wears the safety belt, and the safety of the vehicle user during vehicle driving cannot be ensured.

[0004] SUMMARY

[0005] The present application provides a safety belt wearing detection method and device, electronic equipment and storage medium, which realizes the detection of the actual safety belt wearing situation of the vehicle user and ensures the safety of the vehicle user during vehicle driving.

[0006] According to an aspect of the present application, a safety belt wearing detection method is provided, which comprises:

[0007] real-time acquisition of an in-vehicle driving image of a vehicle user;

[0008] detection of the in-vehicle driving image to determine the body key line segment of the vehicle user and the safety belt area;

[0009] detection of the number of safety belt pixel points overlapped by the body key line segment of the vehicle user and the safety belt area;

[0010] determination of the safety belt wearing detection result of the vehicle user according to the number of safety belt pixel points.

[0011] According to another aspect of the present application, a safety belt wearing detection device is provided, which comprises:

[0012] an in-vehicle driving image acquisition module for real-time acquisition of an in-vehicle driving image of a vehicle user;

[0013] a body key line segment detection module for detecting the in-vehicle driving image to determine the body key line segment of the vehicle user and the safety belt area;

[0014] a safety belt pixel point detection module for detecting the number of safety belt pixel points overlapped by the body key line segment of the vehicle user and the safety belt area;

[0015] The safety belt wearing detection result determination module is configured to determine the safety belt wearing detection result of the vehicle user according to the number of the safety belt pixel points.

[0016] According to another aspect of the present application, an electronic device is provided, which comprises:

[0017] at least one processor; and

[0018] a memory connected with the at least one processor in communication; wherein,

[0019] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the safety belt wearing detection method according to any one of the embodiments of the present application.

[0020] According to another aspect of the present application, a computer readable storage medium is provided, which stores computer instructions for enabling a processor to perform the safety belt wearing detection method according to any one of the embodiments of the present application when executed by the processor.

[0021] According to another aspect of the present application, a computer program product is provided, which comprises a computer program for enabling a processor to perform the safety belt wearing detection method according to any one of the embodiments of the present application when executed by the processor.

[0022] The technical solution of the embodiments of the present application improves the real-time performance of the safety belt wearing detection by acquiring the in-vehicle driving image of the vehicle user in real time. The number of the safety belt pixel points, which are overlapped by the body key line segment of the vehicle user and the safety belt region, is detected, and the safety belt wearing detection result of the vehicle user is determined based on the number of the safety belt pixel points. The overlapping condition of the body of the vehicle user and the safety belt region is considered, the actual safety belt wearing condition of the vehicle user is detected, the accuracy of the safety belt wearing detection of the vehicle user is improved, and the safety of the vehicle user during the driving of the vehicle is ensured. In addition, the efficiency of the safety belt wearing detection of the vehicle user is improved by using the body key line segment of the vehicle user instead of the whole body of the vehicle user.

[0023] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to make the technical solutions in the embodiments of the present application clearer, the accompanying drawings needed in the embodiments will be briefly introduced. Obviously, the accompanying drawings in the following description only need to be some embodiments of the present application, and all other embodiments obtained by those of ordinary skill in the art without any creative work based on the embodiments of the present application should also belong to the protection scope of the present application.

[0025] Fig. 1 is a flow chart of a safety belt wearing detection method according to an embodiment of the present application;

[0026] Fig. 2 is a flow chart of a safety belt wearing detection method according to an embodiment of the present application;

[0027] Fig. 3 is a flow chart of a safety belt wearing detection method according to an embodiment of the present application;

[0028] Fig. 4 is a structural schematic diagram of a safety belt detection model according to an embodiment of the present application;

[0029] Fig. 5 is a schematic diagram of the overlap between a body key line segment and a safety belt area according to an embodiment of the present application;

[0030] Fig. 6 is a schematic diagram of a safety belt segmentation area according to an embodiment of the present application;

[0031] Fig. 7 is a structural schematic diagram of a safety belt wearing detection device according to an embodiment of the present application;

[0032] Fig. 8 is a structural schematic diagram of an electronic device implementing the safety belt wearing detection method according to an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make the technical solutions in the embodiments of the present application clearer, the accompanying drawings needed in the embodiments will be briefly introduced. Obviously, the accompanying drawings in the following description only need to be some embodiments of the present application, and all other embodiments obtained by those of ordinary skill in the art without any creative work based on the embodiments of the present application should also belong to the protection scope of the present application.

[0034] It is to be understood that the terminology "first", "second" and the like used throughout this specification and the annexed drawings is merely intended to distinguish between similar objects and not necessarily for describing a specific sequential or chronological order. It is to be understood that the use of the terms "including", "comprising" or "having" and variations thereof throughout this specification and the annexed drawings is intended to encompass the presence of one or more of the stated features, objects, steps, components, elements or principles, but not preclude the presence or addition of one or more other features, objects, steps, components, elements, principles or groups thereof. Furthermore, as used throughout this specification and the annexed drawings, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or". That is, unless specified otherwise, or clear from the context, the phrase "X employs A or B" is intended to mean that the

[0035] Embodiment One

[0036] FIG. 1 is a flow chart of a method for detecting seat belt wearing according to an embodiment of the present application. The embodiment of the present application can be applied to the case of detecting seat belt wearing of a vehicle user. The method can be executed by a seat belt wearing detection device, which can be implemented in the form of hardware and / or software. The seat belt wearing detection device can be configured in an electronic device that carries a seat belt wearing detection function, such as a vehicle terminal.

[0037] Referring to the method for detecting seat belt wearing shown in FIG. 1, the method comprises the following steps.

[0038] S110, acquiring a driving-in-vehicle image of a vehicle user in real time.

[0039] The vehicle user can be a party who is using a vehicle during driving of the vehicle. For example, the vehicle user can include a driving user and a riding user. The driving-in-vehicle image can be an image of the vehicle user in the vehicle during driving of the vehicle. Based on the driving-in-vehicle image, the seat belt wearing of the vehicle user can be detected. Optionally, the driving-in-vehicle image can include a front image or a side image of the vehicle user. Preferably, the driving-in-vehicle image is a front image of the vehicle user.

[0040] Specifically, the driving-in-vehicle image of the vehicle user can be acquired in real time by an image acquisition device in the vehicle.

[0041] Exemplarily, the image collection device in the vehicle can be a camera in the vehicle, for example, a DMS (Driver Monitoring System Camera). The installation position of the camera in the vehicle can include an A-pillar in the vehicle or a reading light, etc. The in-vehicle driving video of the vehicle user can be collected in real time by the camera in the vehicle, for example, a front in-vehicle driving video. The in-vehicle driving video can be processed frame by frame into in-vehicle driving images. Optionally, the in-vehicle driving images can also be subjected to interference suppression operation, and the in-vehicle driving images are updated. Exemplarily, the interference suppression operation can include filtering, denoising or downsampling, etc. The filtering is used to suppress the noise of the in-vehicle driving images, for example, mean filtering, median filtering or Gaussian filtering, etc., which can effectively reduce the noise and interference in the in-vehicle driving images. The denoising can be realized by local thresholding or wavelet transform, etc., which can effectively remove the salt and pepper noise or Gaussian noise, etc. The downsampling can suppress the interference by reducing the number of sampling points of the image.

[0042] S120, detecting the in-vehicle driving image to determine the body key line segment and the safety belt area of the vehicle user.

[0043] The body key line segment can be a body line segment that overlaps with the safety belt area. The body key line segment can be used to identify the body of the vehicle user, and can also be used as a reference line segment for safety belt wearing detection. Optionally, the number of body key line segments can be at least one. The number of body key line segments can be set and adjusted according to the data processing capability of the device. The safety belt area can be used to identify the safety belt. Compared with taking the entire body area of the vehicle user as a reference for safety belt wearing detection, taking the body key line segment as a reference for safety belt wearing detection is simpler and faster.

[0044] Specifically, the image recognition technology can be used to perform image recognition on the in-vehicle driving image to detect the body key line segment and the safety belt area of the vehicle user.

[0045] In an optional embodiment of the present application, detecting the in-vehicle driving image to determine the body key line segment and the safety belt area of the vehicle user comprises: inputting the in-vehicle driving image into a pre-trained safety belt detection model to output at least two body key points and a safety belt area of the vehicle user; connecting each body key point to obtain at least one body key line segment.

[0046] The input data of the safety belt detection model can be an in-vehicle driving image, and the output result can be at least two body key points of a vehicle user and a safety belt region. The at least two body key points of the vehicle user can be at least two layers of body key point heat maps of the vehicle user, which are used to represent a probability map of the body key point position estimated by the safety belt detection model. The safety belt region can be a binary image, that is, a safety belt region map. In the figure, the point with a pixel value of 0 represents a non-safety belt region, and the point with a pixel value of 1 represents a safety belt region. The safety belt detection model can be a convolutional neural network model. The training samples of the safety belt detection model can include in-vehicle driving image samples, historical body key point samples of a vehicle user, and safety belt region samples. The in-vehicle driving image sample can be an in-vehicle image of the vehicle user at a historical moment during vehicle driving. The in-vehicle driving image sample can be a front image sample of the vehicle user. The historical vehicle user can be the vehicle user in the in-vehicle driving image sample. The historical body key point sample of the vehicle user can be a pre-labeled body key position of the historical vehicle user in the in-vehicle driving image sample. The number of the historical body key point sample of the vehicle user can be at least two. The historical body key point sample of the vehicle user can be a body key point coordinate of the historical vehicle user in the in-vehicle driving image. The safety belt region sample can be a pre-labeled safety belt region in the in-vehicle driving image sample. The training method of the safety belt detection model can be a supervised model training method.

[0047] In an optional embodiment of the present application, the in-vehicle driving image is input into the pre-trained safety belt detection model to output at least two body key points of a vehicle user and a safety belt region, including: performing feature extraction on the in-vehicle driving image of the vehicle user through a feature extraction layer to obtain driving image semantic features in at least one dimension; performing feature fusion on the driving image semantic features in each dimension through a feature fusion layer to obtain a driving image feature map; performing detection on the driving image feature map through a first detection head layer to obtain at least two body key points of the vehicle user; and performing detection on the driving image feature map through a second detection head layer to obtain the safety belt region.

[0048] The model structure of the safety belt detection model includes a feature extraction layer, a feature fusion layer, a first detection head layer, and a second detection head layer. The feature extraction layer can be used for feature extraction on the in-vehicle driving image. For example, the feature extraction layer can use a Res Net (Residual Network) as a backbone network. For example, ResNet-50. The feature fusion layer can be used for feature fusion on the driving semantic image features. For example, the feature fusion layer can use a feature pyramid structure (FPN). The first detection head layer is used for detecting body key points. The second detection head layer is used for detecting a safety belt region. For example, the first detection head layer (or the second detection head layer) can include two layers of 3x3 convolution and one layer of 1x1 convolution. The driving image semantic features can be image semantic features of the in-vehicle driving image. The driving image feature map can be a feature fusion result of the driving image semantic features in each dimension.

[0049] Specifically, the in-vehicle driving image of the vehicle user can be subjected to feature extraction by the feature extraction layer to obtain driving image semantic features in at least one dimension. The driving image semantic features in each dimension can be subjected to feature fusion by the feature fusion layer to obtain a driving image feature map. At least two body key points of the vehicle user can be obtained by detecting the driving image feature map by the first detection head layer. The safety belt region can be obtained by detecting the driving image feature map by the second detection head layer.

[0050] The model structure of the safety belt detection model is specified as the feature extraction layer, the feature fusion layer, the first detection head layer, and the second detection head layer. The feature extraction layer is used for feature extraction on the in-vehicle driving image to obtain the driving image semantic features. The feature fusion layer is used for feature fusion on the driving image semantic features to obtain the driving image feature map. The first detection head layer is used for detecting the body key points. The second detection head layer is used for detecting the safety belt region. Based on the same feature extraction layer and feature fusion layer, the dual detection head structure is used to detect different dimensions of the same in-vehicle driving image to obtain the body key points and the safety belt region of the vehicle user. The efficiency, accuracy, and matching of the body key points and the safety belt region detection are further improved, thereby improving the efficiency and accuracy of the safety belt wearing detection.

[0051] The body key points can be used to represent the key positions of the body of the vehicle user. The body key points can be used to identify the body regions of the vehicle user. The number of body key points can be at least two. Correspondingly, the number of body key lines can be at least one.

[0052] In an optional embodiment of the present application, the body key points include a neck key point, two shoulder key points, and a safety belt lock buckle opposite thigh key point.

[0053] The neck key point is used to identify the neck position of the vehicle user. For example, the neck key point can be the center point of the neck. The double shoulder key point can be used to identify the double shoulder position of the vehicle user. For example, the double shoulder key point can be the outer top point of the double shoulder. The thigh key point opposite the seat belt buckle can be used to identify the thigh position of the vehicle user opposite the seat belt buckle. For example, the thigh key point opposite the seat belt buckle can be the outer top point of the thigh opposite the seat belt buckle.

[0054] The scheme can generate the body key line segment of the vehicle user based on the four body key points by specifically defining the body key points as the neck key point, the double shoulder key point, and the thigh key point opposite the seat belt buckle, so as to realize the seat belt wearing detection of the vehicle user and improve the efficiency of the seat belt wearing detection.

[0055] Specifically, the in-vehicle driving image can be input into the pre-trained seat belt detection model to output at least two body key points and a seat belt region of the vehicle user. The body key points can be sequentially connected, and a connection line overlapping the seat belt region is selected to obtain at least one body key line segment. For example, the sequential connection mode of the body key points can include sequentially connecting the body key points in a clockwise direction or a counterclockwise direction, or connecting the body key points two by two.

[0056] The scheme introduces the seat belt detection model, realizes the double detection of the body key points and the seat belt region of the vehicle user based on a single seat belt detection model, and further improves the efficiency and accuracy of the determination of the body key line segment and the seat belt region of the vehicle user, thereby improving the efficiency and accuracy of the seat belt wearing detection.

[0057] S130, detecting the number of seat belt pixel points overlapping the body key line segment and the seat belt region of the vehicle user.

[0058] The seat belt pixel points overlapping the body key line segment and the seat belt region of the vehicle user can be used to represent the overlapping condition of the seat belt and the body. Compared with the seat belt wearing detection using only the seat belt region, the seat belt wearing detection using the number of seat belt pixel points overlapping the body key line segment and the seat belt region not only considers the seat belt itself, but also considers the overlapping condition of the seat belt and the body, so that the seat belt wearing condition of the vehicle user can be more accurately detected.

[0059] Specifically, the number of seat belt pixel points overlapping the body key line segment and the seat belt region of the vehicle user can be detected.

[0060] S140, determining the seat belt wearing detection result of the vehicle user according to the number of seat belt pixel points.

[0061] The safety belt wearing detection result can be used to represent the safety belt wearing situation of the vehicle user. Optionally, the safety belt wearing detection result can include normal safety belt wearing and abnormal safety belt wearing. The normal safety belt wearing can be used to represent that the vehicle user wears the safety belt normally. For example, the vehicle user has worn the safety belt and / or the safety belt wearing posture of the vehicle user is correct. The abnormal safety belt wearing can be used to represent that the vehicle user does not wear the safety belt normally. For example, the vehicle user does not wear the safety belt or the safety belt wearing posture of the vehicle user is incorrect.

[0062] Specifically, the number of reference safety belt pixel points when the vehicle user wears the safety belt normally can be acquired, which is determined by a technician in advance. Optionally, the technician can determine the number of reference safety belt pixel points for multiple times, select the minimum value or the average value of the number of reference safety belt pixel points determined for multiple times, and update the number of reference safety belt pixel points. The number of safety belt pixel points and the number of reference safety belt pixel points can be compared. When the number of safety belt pixel points is greater than or equal to the number of reference safety belt pixel points, it is determined that the safety belt detection result of the vehicle user is normal safety belt wearing. When the number of safety belt pixel points is less than the number of reference safety belt pixel points, it is determined that the safety belt detection result of the vehicle user is abnormal safety belt wearing.

[0063] The technical scheme of the embodiment of the application improves the real-time performance of the safety belt wearing detection by acquiring the in-vehicle driving image of the vehicle user in real time. The safety belt wearing detection result of the vehicle user is determined based on the number of safety belt pixel points by detecting the number of safety belt pixel points overlapped by the body key line segment of the vehicle user and the safety belt region. The overlapping situation of the body of the vehicle user and the safety belt region is considered, the actual safety belt wearing situation of the vehicle user can be detected, the accuracy of the safety belt wearing detection of the vehicle user is improved, and the safety of the vehicle user during vehicle driving is ensured. In addition, the safety belt wearing detection efficiency of the vehicle user can be improved by using the body key line segment of the vehicle user instead of the whole body of the vehicle user.

[0064] In an optional embodiment of the application, after the safety belt wearing detection result of the vehicle user is determined based on the number of safety belt pixel points, the method further includes: determining safety belt wearing reminding information according to the safety belt wearing detection result of the vehicle user, and feeding back the vehicle user.

[0065] The safety belt wearing reminding information is used for reminding the vehicle user to adjust the safety belt wearing condition. Optionally, the safety belt wearing reminding information can comprise safety belt wearing state reminding information and / or safety belt wearing posture reminding information. Optionally, the safety belt wearing reminding information can comprise text information, voice information, image display information or sound and light information, etc. Exemplarily, the safety belt wearing state reminding information can be voice information of "please wear safety belt ~"; the safety belt wearing posture reminding information can be voice information of "please adjust the safety belt wearing posture oh ~".

[0066] Optionally, when the safety belt wearing state of the vehicle user is determined as not wearing safety belt according to the number of safety belt pixel points, the safety belt wearing reminding information can be determined as safety belt wearing state reminding information.

[0067] Optionally, when the safety belt wearing state of the vehicle user is determined as wearing safety belt and the safety belt wearing posture of the vehicle user is determined as incorrect wearing posture according to the number of safety belt pixel points, the safety belt wearing reminding information can be determined as safety belt wearing posture reminding information.

[0068] Optionally, when the safety belt wearing state of the vehicle user is determined as wearing safety belt and the safety belt wearing posture of the vehicle user is determined as correct wearing posture according to the number of safety belt pixel points, the safety belt wearing reminding information can be determined as null.

[0069] According to the above, the safety belt wearing reminding information is determined according to the safety belt wearing detection result of the vehicle user after the safety belt wearing detection result of the vehicle user is determined according to the number of safety belt pixel points, and the safety belt wearing reminding information is fed back to the vehicle user, so that the safety belt wearing condition of the vehicle user is timely reminded based on the safety belt wearing reminding information, and the safety of the vehicle user in the vehicle driving process is further improved.

[0070] Embodiment two

[0071] Fig. 2 is a flow chart of a safety belt wearing detection method provided by the embodiment two of the present application. The embodiment of the present application further specifies "determining the safety belt wearing detection result of the vehicle user according to the number of safety belt pixel points" as "acquiring the safety belt pixel point range; comparing the number of safety belt pixel points with the safety belt pixel point range; when the number of safety belt pixel points is out of the safety belt pixel point range, determining the safety belt wearing state of the vehicle user as not wearing safety belt; when the number of safety belt pixel points is in the safety belt pixel point range, determining the safety belt wearing state of the vehicle user as wearing safety belt", and specifies the safety belt wearing detection result as the safety belt wearing state, so as to further improve the detection efficiency of the safety belt wearing detection. It should be noted that the parts not described in detail in the embodiment of the present application can be referred to the description of other embodiments.

[0072] Referring to the safety belt wearing detection method shown in FIG. 2, the method comprises the following steps.

[0073] S210, acquiring in real time a driving-in-vehicle image of a vehicle user.

[0074] S220, detecting the driving-in-vehicle image to determine a body key line segment and a safety belt region of the vehicle user.

[0075] S230, detecting the number of safety belt pixel points overlapped by the body key line segment and the safety belt region of the vehicle user.

[0076] S240, acquiring a safety belt pixel point range.

[0077] The safety belt pixel point range can be used to judge the safety belt wearing detection condition of the vehicle user. The safety belt pixel point range can be the number range of the safety belt pixel points overlapped by the body key line segment and the safety belt region of the vehicle user in the case that the vehicle user wears the safety belt normally. The normal wearing of the safety belt can include that the safety belt has been worn.

[0078] Specifically, the number of reference safety belt pixel points measured by the technician multiple times can be acquired, and the safety belt pixel point range can be determined according to the maximum value and the minimum value of the number of reference safety belt pixel points measured multiple times.

[0079] In an optional embodiment of the present application, the acquiring of the safety belt pixel point range comprises: acquiring body shape information of the vehicle user; and determining the safety belt pixel point range according to the body shape information of the vehicle user.

[0080] The body shape information of the vehicle user can be used to represent the body contour of the vehicle user. The body shape information of the vehicle user is different, and the corresponding body key line segment and the safety belt region are different, and correspondingly, the safety belt overlap region is also different, and the number of safety belt pixel points overlapped by the body key line segment and the safety belt region of the vehicle user is also different.

[0081] The body shape information of the vehicle user and the safety belt pixel point range have a corresponding relationship. Optionally, the number of reference safety belt pixel points corresponding to the body shape information of different vehicle users measured by the technician multiple times can be acquired in advance, and the safety belt pixel point range corresponding to the body shape information of different vehicle users can be determined according to the maximum value and the minimum value of the number of reference safety belt pixel points corresponding to the body shape information of different vehicle users measured multiple times. The corresponding relationship between the body shape information of the vehicle user and the safety belt pixel point range can be stored in a database in advance.

[0082] Optionally, the body size information of the vehicle user can be obtained in the case that the vehicle user is authorized. The in-vehicle driving image of the vehicle user can be detected to determine the body size information of the vehicle user. For example, a pre-trained image recognition model can be used to detect the in-vehicle driving image of the vehicle user to determine the body size information of the vehicle user.

[0083] Specifically, the corresponding safety belt pixel point range can be queried based on the correspondence between the body size information of the vehicle user and the safety belt pixel point range according to the body size information of the vehicle user.

[0084] The scheme takes into account the differentiation of the body size of the vehicle user when obtaining the safety belt pixel point range, further improves the accuracy of the safety belt pixel point range, and thus improves the accuracy of the safety belt wearing detection of the vehicle user based on the safety belt pixel point range.

[0085] S250, compare the number of safety belt pixel points and the safety belt pixel point range.

[0086] Specifically, the number of safety belt pixel points and the safety belt pixel point range can be compared to determine whether the number of safety belt pixel points is included in the safety belt pixel point range.

[0087] S260, when the number of safety belt pixel points is outside the safety belt pixel point range, determine that the safety belt wearing state of the vehicle user is not wearing a safety belt.

[0088] The safety belt wearing detection result can be a safety belt wearing state. The safety belt wearing state can be used to represent whether the vehicle user wears a safety belt. The safety belt wearing state can include wearing a safety belt and not wearing a safety belt. When the number of safety belt pixel points is outside the safety belt pixel point range, it can be understood that the number of safety belt pixel points is less than the minimum value of the safety belt pixel point range or the number of safety belt pixel points is greater than the maximum value of the safety belt pixel point range. At this time, the number of safety belt pixel points is not included in the safety belt pixel point range, that is, it can be represented that the safety belt wearing state of the vehicle user is not wearing a safety belt.

[0089] Specifically, when the number of safety belt pixel points is outside the safety belt pixel point range, the safety belt wearing state of the vehicle user can be determined as not wearing a safety belt.

[0090] S270, when the number of safety belt pixel points is within the safety belt pixel point range, determine that the safety belt wearing state of the vehicle user is wearing a safety belt.

[0091] The number of safety belt pixels is within the safety belt pixel range. It can be understood that the number of safety belt pixels is greater than or equal to the minimum value of the safety belt pixel range and less than or equal to the maximum value of the safety belt pixel range. At this time, the number of safety belt pixels is within the safety belt pixel range, that is, the safety belt wearing state of the vehicle user can be represented as wearing a safety belt.

[0092] Specifically, when the number of safety belt pixels is within the safety belt pixel range, the safety belt wearing state of the vehicle user can be determined as wearing a safety belt.

[0093] In an optional embodiment of the present application, when the number of safety belt pixels is within the safety belt pixel range, the safety belt wearing state of the vehicle user is determined as wearing a safety belt, comprising: when the number of first safety belt pixels or the number of second safety belt pixels is within the safety belt pixel range, and the number of the other safety belt pixels is outside the safety belt pixel range, the safety belt wearing state of the vehicle user is determined as wearing a safety belt; detecting whether there is a safety belt shielding event according to the in-vehicle driving image; when it is detected that there is a safety belt shielding event, the safety belt wearing posture of the vehicle user is determined as correct; when it is detected that there is no safety belt shielding event, the safety belt wearing posture of the vehicle user is determined as incorrect.

[0094] The body key line segment can include a first body key line segment and a second body key line segment. The first body key line segment and the second body key line segment are both body key line segments that overlap the safety belt region. The safety belt pixel can include a first safety belt pixel and a second safety belt pixel. The first safety belt pixel can be a safety belt pixel where the first body key line segment of the vehicle user overlaps the safety belt region. The second safety belt pixel can be a safety belt pixel where the second body key line segment of the vehicle user overlaps the safety belt region.

[0095] The safety belt wearing detection result can include a safety belt wearing state and a safety belt wearing posture. The safety belt occlusion event can be used to represent a situation in which the safety belt is occluded. It can be understood that due to reasons such as the in-vehicle environment (for example, light reasons) or the dressing of the vehicle user, the safety belt can be in a situation of occlusion. At this time, even if the vehicle user has worn the safety belt and the wearing posture is correct, it is also difficult to correctly detect the safety belt wearing situation based on the in-vehicle driving image, especially the safety belt wearing posture. At this time, when the number of first safety belt pixel points or the number of second safety belt pixel points is within the safety belt pixel point range, and the number of another safety belt pixel point is outside the safety belt pixel point range, it can be determined that the safety belt wearing state of the vehicle user is that the safety belt is worn, and the safety belt wearing posture of the vehicle user is that the wearing posture is correct. Therefore, the influence of the safety belt occlusion event on the detection of the safety belt wearing posture of the vehicle user is avoided, and the accuracy of the safety belt wearing detection under the occlusion situation can be improved.

[0096] Specifically, when the number of first safety belt pixel points or the number of second safety belt pixel points is within the safety belt pixel point range, and the number of another safety belt pixel point is outside the safety belt pixel point range, it is determined that the safety belt wearing state of the vehicle user is that the safety belt is worn. The in-vehicle driving image can be detected, for example, the in-vehicle environmental factors or the dressing of the vehicle user are detected to determine whether there is a safety belt occlusion event. When it is detected that there is a safety belt occlusion event, it is determined that the safety belt wearing posture of the vehicle user is that the wearing posture is correct. When it is detected that there is no safety belt occlusion event, it is determined that the safety belt wearing posture of the vehicle user is that the wearing posture is incorrect.

[0097] The present scheme introduces the safety belt wearing posture detection of the vehicle user on the basis of the safety belt wearing state detection of the vehicle user, considers the influence of the safety belt occlusion event when the number of first safety belt pixel points or the number of second safety belt pixel points is within the safety belt pixel point range, and the number of another safety belt pixel point is outside the safety belt pixel point range, and further improves the accuracy of the safety belt wearing posture detection of the vehicle user.

[0098] Optionally, when the number of first safety belt pixel points and the number of second safety belt pixel points are both within the safety belt pixel point range, it can be determined that the safety belt wearing state of the vehicle user is that the safety belt is worn, and the safety belt wearing posture of the vehicle user is that the wearing posture is correct.

[0099] Optionally, when the number of first safety belt pixel points and the number of second safety belt pixel points are both outside the safety belt pixel point range, it can be determined that the safety belt wearing state of the vehicle user is that the safety belt is not worn.

[0100] The technical scheme of the embodiment of the present application introduces a safety belt pixel point range, and by specifically taking the safety belt wearing detection result as a safety belt wearing state, the safety belt wearing state of the user of the vehicle is detected based on the comparison between the number of safety belt pixel points and the safety belt pixel point range, and the detection efficiency of the safety belt wearing detection state is further improved.

[0101] Fig. 3 is a flow chart of a safety belt wearing detection method according to the second embodiment of the present application. Fig. 3 is a preferred embodiment of the present application based on the above-mentioned embodiment. Referring to the safety belt wearing detection method shown in Fig. 4, the method comprises:

[0102] S310, acquiring an in-vehicle real-time video stream.

[0103] The in-vehicle real-time video stream is an in-vehicle driving video.

[0104] Specifically, the in-vehicle real-time video stream monitored by the in-vehicle camera can be acquired in real time. The in-vehicle real-time video stream can be processed frame by frame into in-vehicle driving images, and after interference suppression operations such as filtering, the in-vehicle driving images are updated as input images for the next step. The in-vehicle camera can be installed at the front row position such as the A-pillar or reading light in the vehicle. The in-vehicle real-time video stream and the in-vehicle driving image should ensure that the front view of the driving user can be captured in each posture. Exemplarily, the in-vehicle camera can be a DMS camera. The detection of the in-vehicle driving video or the in-vehicle driving image based on the DMS camera takes into account the diversity of different light conditions and driving users, and has strong environmental adaptability, and can work stably in daytime, night or other complex light conditions.

[0105] S320, detecting the body key points and the safety belt region of the driving user based on a safety belt detection model.

[0106] The safety belt detection model can be a convolutional neural network model. The model structure of the safety belt detection model can include a feature extraction layer, a feature fusion layer, a first detection head layer, and a second detection head layer. For example, FIG. 4 is a structural diagram of the safety belt detection model. As shown in FIG. 4, the feature extraction layer can use Res Net (Residual Network) as the backbone network. For example, ResNet-50. The feature fusion layer can use a feature pyramid structure (FPN). By using the feature pyramid structure to fuse the semantic features of the driving image at different levels, the driving image feature map can be obtained, which can improve the recognition accuracy of the safety belt detection model for the body key points of the driving user and the safety belt. After obtaining the driving image feature map, a double detection head structure can be used, that is, the first detection head layer and the second detection head layer are used, and two detection branches are used to obtain the body key points of the driving user and the safety belt area. The first detection head layer and the second detection head layer can include two layers of 3x3 convolution and one layer of 1x1 convolution. For example, the output result of the key point branch (i.e., the first detection head layer) can be four layers of body key point heat maps, which respectively represent the probability maps of the four key points of the safety belt detection model, including the neck, the shoulders, and the opposite thigh (i.e., the right thigh) of the safety belt buckle. The output result of the safety belt segmentation branch (i.e., the second detection head layer) can be one layer of binary image (i.e., safety belt area map). The pixel value of 0 in the image indicates a non-safety belt area; and the pixel value of 1 in the image indicates a safety belt area.

[0107] When training the safety belt detection model, the output results of the two branches (i.e., the key point branch and the safety belt segmentation branch) can be compared with the pre-labeled true values respectively, and the loss function can be calculated. The network parameters of the initial model are updated iteratively through back propagation, the optimized model is trained, and the final safety belt detection model is obtained. The in-vehicle driving image can be input into the safety belt detection model, and the ability to output four body key point coordinates of the driving user and the safety belt area map can be obtained.

[0108] Specifically, the in-vehicle driving image can be input into the safety belt detection model, and the body key points of the driving user and the safety belt area can be output.

[0109] S330, calculating the body key line segment.

[0110] Specifically, based on the four body key point coordinates of the driving user output by the safety belt detection model, the neck key point and the right shoulder key point can be connected to obtain a first body key line segment. The left shoulder key point and the right thigh key point can be connected to obtain a second body key line segment.

[0111] For example, the following formula can be used to represent the first body key line segment: y1=ax1+b.

[0112] In the formula, y1 is the longitudinal coordinate of the first body key line segment; x1 is the transverse coordinate of the first body key line segment; a and b are linear coefficients of the first body key line segment.

[0113] The second body key line segment can be expressed by the following formula: y2=cx2+d.

[0114] In the formula, y2 is the longitudinal coordinate of the second body key line segment; x2 is the transverse coordinate of the second body key line segment; c and d are linear coefficients of the second body key line segment.

[0115] S340, detecting the number of safety belt pixels of the body key line segment of the driving user that overlap the safety belt region.

[0116] Specifically, on the basis of the safety belt region map, the first body key line segment and the second body key line segment are traversed pixel by pixel respectively, and the number of first safety belt pixels of the first body key line segment that overlap the safety belt region and the number of second safety belt pixels of the second body key line segment that overlap the safety belt region are counted respectively.

[0117] For example, FIG. 5 is a schematic diagram of the body key line segment overlapping the safety belt region. As shown in FIG. 5, the first body key line segment is L1, and the second body key line segment is L2. The first body key line segment L1 and the second body key line segment L2 overlap the safety belt region. FIG. 6 is a schematic diagram of a safety belt segmentation region. As shown in FIG. 6, the safety belt segmentation region is obtained by segmenting the safety belt region by the first body key line segment L1 and the second body key line segment L2. The number of first safety belt pixels of the first body key line segment that overlap the safety belt region and the number of second safety belt pixels of the second body key line segment that overlap the safety belt region can be counted. The number of first safety belt pixels can be represented as Count_1, and the number of second safety belt pixels can be represented as Count_2.

[0118] S350, determining whether the number of safety belt pixels is within the safety belt pixel range. If yes, performing S310 on the next frame of in-vehicle driving image; if no, performing S360.

[0119] Specifically, the pixel range of the seat belt in the in-vehicle driving image in the case of normal wearing of the seat belt (i.e., the seat belt is worn and the wearing posture is correct) can be calculated first to obtain a seat belt pixel range, denoted as [M, N]. The number of first seat belt pixels Count_1 and the number of second seat belt pixels Count_2 can be compared with the seat belt pixel range [M, N] respectively. If both Count_1 and Count_2 are within [M, N], it indicates that the current driving user has worn the seat belt and the wearing posture is correct. If neither Count_1 nor Count_2 is within [M, N], it indicates that the driving user has not worn the seat belt or the wearing posture is incorrect. If only one of Count_1 and Count_2 is within [M, N], it can be adjusted freely according to the offline situation. For example, the seat belt blocking event can be detected. If there is no seat belt blocking event, it is determined that the wearing posture is incorrect. If there is a seat belt blocking event, it is determined that the wearing posture is correct. The reason is that due to the blocking of objects such as down jackets or hats, it is easy to appear that the driving user has worn the seat belt, but it cannot be observed in the in-vehicle driving image. By detecting the seat belt blocking event, the situation of false detection of the seat belt wearing posture can be avoided.

[0120] S360, when it is detected that the driving user has not worn the seat belt or the wearing posture is incorrect, sending a seat belt wearing reminder information to the driving user.

[0121] The seat belt wearing reminder information can include sound reminder information or flash light reminder information, etc.

[0122] Specifically, when it is detected that the driving user has not worn the seat belt or the wearing posture is incorrect, the vehicle alarm can be triggered to send the seat belt wearing reminder information to the driving user to ensure safe driving.

[0123] The scheme connects the body key points of the driving user, generates the body key line segments of the driving user, and detects the number of safety belt pixel points overlapping the safety belt area on the body key line segments, innovatively evaluates whether the safety belt is correctly diagonal across the driving user's body, judges the wearing state of the safety belt, designs a double-branch detection model, accurately identifies the positions of the body key points of the driving user by using the key point branch, performs semantic segmentation of the safety belt by using the safety belt segmentation branch, can accurately judge the position of the safety belt relative to the driving user's body, and thus effectively judges whether the driving user correctly wears the safety belt; at the same time, through accurate safety belt wearing detection, the driving user can be ensured to be protected to the maximum extent when encountering an emergency, the harm in traffic accidents is reduced, and the safety of the driving process is improved; through the real-time monitoring and feedback mechanism, the driving user can be timely reminded to adjust the wearing state and posture of the safety belt, further increasing the safety in the driving process; in addition to improving the safety of riding, the driving experience is also enhanced, so that the driver and the passenger can feel the care and protection brought by technology; in summary, the scheme not only improves the accuracy and reliability of the traditional safety belt detection technology, but also provides a technical basis for integrating more advanced safety features into the intelligent driving system, and promotes the progress of intelligent automobile technology.

[0124] Embodiment three

[0125] FIG. 7 is a structural schematic diagram of a safety belt wearing detection device provided by an embodiment of the present application. The embodiment of the present application can be applied to the case of detecting the safety belt wearing situation of the vehicle user, the device can execute the safety belt wearing detection method, the device can be realized in the form of hardware and / or software, and the device can be configured in an electronic device carrying the safety belt wearing detection function, such as a vehicle terminal.

[0126] Referring to the safety belt wearing detection device shown in FIG. 7, it includes: an in-vehicle driving image acquisition module 710, a body key line segment detection module 720, a safety belt pixel point detection module 730, and a safety belt wearing detection result determination module 740. The in-vehicle driving image acquisition module 710 is used to acquire the in-vehicle driving image of the vehicle user in real time; the body key line segment detection module 720 is used to detect the in-vehicle driving image, determine the body key line segments and the safety belt area of the vehicle user; the safety belt pixel point detection module 730 is used to detect the number of safety belt pixel points overlapping the safety belt area of the vehicle user; and the safety belt wearing detection result determination module 740 is used to determine the safety belt wearing detection result of the vehicle user according to the number of safety belt pixel points.

[0127] The technical scheme of the embodiment of the present application improves the real-time performance of the safety belt wearing detection by acquiring the in-vehicle driving image of the vehicle user in real time; the safety belt wearing detection result of the vehicle user is determined based on the number of the safety belt pixel points by detecting the number of the safety belt pixel points overlapped by the body key line segment of the vehicle user and the safety belt region, the overlap of the body of the vehicle user and the safety belt region is considered, the detection of the actual safety belt wearing condition of the vehicle user can be realized, the accuracy of the safety belt wearing detection of the vehicle user is improved, and the safety of the vehicle user during the driving of the vehicle is ensured; in addition, the efficiency of the safety belt wearing detection of the vehicle user can be improved by using the body key line segment of the vehicle user instead of the whole body of the vehicle user.

[0128] In an optional embodiment of the present application, the safety belt wearing detection result determination module 740 comprises: a safety belt pixel point range acquisition unit, configured to acquire the safety belt pixel point range; a safety belt pixel point number comparison unit, configured to compare the number of the safety belt pixel points with the safety belt pixel point range; a first safety belt wearing state determination unit, configured to determine that the safety belt wearing state of the vehicle user is not wearing a safety belt when the number of the safety belt pixel points is outside the safety belt pixel point range; and a second safety belt wearing state determination unit, configured to determine that the safety belt wearing state of the vehicle user is wearing a safety belt when the number of the safety belt pixel points is within the safety belt pixel point range.

[0129] In an optional embodiment of the present application, the second safety belt wearing state determination unit comprises: a second safety belt wearing state determination subunit, configured to determine that the safety belt wearing state of the vehicle user is wearing a safety belt when the number of the first safety belt pixel points or the number of the second safety belt pixel points is within the safety belt pixel point range and the number of the other safety belt pixel points is outside the safety belt pixel point range; wherein the body key line segment comprises a first body key line segment and a second body key line segment; the first body key line segment and the second body key line segment are the body key line segments overlapped with the safety belt region; the safety belt pixel points comprise first safety belt pixel points and second safety belt pixel points; wherein the first safety belt pixel points are the safety belt pixel points overlapped by the first body key line segment of the vehicle user and the safety belt region; the second safety belt pixel points are the safety belt pixel points overlapped by the second body key line segment of the vehicle user and the safety belt region; a safety belt shielding event detection subunit, configured to detect whether there is a safety belt shielding event according to the in-vehicle driving image; a first safety belt wearing posture determination subunit, configured to determine that the safety belt wearing state of the vehicle user is wearing a safety belt and the safety belt wearing posture of the vehicle user is correct when it is detected that there is a safety belt shielding event; and a second safety belt wearing posture determination subunit, configured to determine that the safety belt wearing posture of the vehicle user is incorrect when it is detected that there is no safety belt shielding event.

[0130] In an optional embodiment of the present application, the seat belt pixel range acquisition unit comprises: a body type information acquisition subunit, configured to acquire the body type information of the vehicle user; and a seat belt pixel range acquisition subunit, configured to determine the seat belt pixel range according to the body type information of the vehicle user.

[0131] In an optional embodiment of the present application, the body key line segment detection module 720 comprises: a body key point detection unit, configured to input the in-vehicle driving image into a pre-trained seat belt detection model, and output at least two body key points and a seat belt region of the vehicle user; wherein the training sample of the seat belt detection model comprises an in-vehicle driving image sample, a body key point sample of the in-vehicle driving image sample, and a seat belt region sample of the in-vehicle driving image sample; and a body key line segment generation unit, configured to connect each body key point to obtain at least one body key line segment.

[0132] In an optional embodiment of the present application, the model structure of the seat belt detection model comprises a feature extraction layer, a feature fusion layer, a first detection head layer, and a second detection head layer; the body key point detection unit comprises: a feature extraction layer, configured to perform feature extraction on the in-vehicle driving image of the vehicle user to obtain driving image semantic features in at least one dimension; a feature fusion layer, configured to perform feature fusion on the driving image semantic features in each dimension to obtain a driving image feature map; a first detection head layer, configured to detect the driving image feature map to obtain at least two body key points of the vehicle user; and a second detection head layer, configured to detect the driving image feature map to obtain a seat belt region.

[0133] In an optional embodiment of the present application, the body key points comprise a neck key point, shoulder key points, and a thigh key point opposite to the seat belt buckle.

[0134] In an optional embodiment of the present application, the device further comprises: a seat belt wearing reminding information generation module, configured to, after determining the seat belt wearing detection result of the vehicle user according to the number of seat belt pixels, determine seat belt wearing reminding information according to the seat belt wearing detection result of the vehicle user, and feed back to the vehicle user.

[0135] The seat belt wearing detection device provided in the embodiments of the present application can execute the seat belt wearing detection method provided in any of the embodiments of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0136] In the technical solutions of the embodiments of the present application, the acquisition, storage, and application of the in-vehicle driving image of the vehicle user, the seat belt pixel range, and the body type information of the vehicle user, etc. all conform to the relevant legal regulations and do not violate public order and good customs.

[0137] Example Four

[0138] FIG. 8 illustrates a structural diagram of an electronic device 800 that can be used to implement embodiments of the present application. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application described and / or claimed in this document.

[0139] As shown in FIG. 8, the electronic device 800 includes at least one processor 801, and memory, such as read-only memory (ROM) 802, random access memory (RAM) 803, etc., that is communicatively connected to the at least one processor 801, where the memory stores computer programs that are executable by the at least one processor. The processor 801 can perform various appropriate actions and processes according to the computer programs stored in the read-only memory (ROM) 802 or loaded into the random access memory (RAM) 803 from the storage unit 808. In the RAM 803, various programs and data required for the operation of the electronic device 800 can also be stored. The processor 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0140] Various components in the electronic device 800 are connected to the I / O interface 805, including an input unit 806, such as a keyboard, a mouse, etc., an output unit 807, such as various types of displays, a speaker, etc., a storage unit 808, such as a magnetic disk, an optical disk, etc., and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0141] The processor 801 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the processor 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 801 performs various methods and processes described above, such as the seat belt wearing detection method.

[0142] In some embodiments, the seat belt wearing detection method can be implemented as a computer program tangibly embodied in a computer readable storage medium, e.g., storage unit 808. In some embodiments, parts or all of the computer program can be loaded and / or installed onto electronic device 800 via, e.g., ROM 802 and / or communication unit 809. When the computer program is loaded onto RAM 803 and executed by processor 801, one or more steps of the above-described seat belt wearing detection method can be performed. Alternatively, in other embodiments, processor 801 can be configured to perform the seat belt wearing detection method by other means, e.g., with the aid of firmware.

[0143] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0144] Computer programs used to implement the methods of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed by the processor, implements the functions / acts specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as part of a standalone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0145] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store computer programs for use by or in connection with an instruction execution system, apparatus, or device. Computer-readable storage media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0146] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0147] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), blockchain network, and the Internet.

[0148] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, and solves the defects of large management difficulty and weak business scalability in traditional physical host and VPS (Virtual Private Server).

[0149] It should be understood that the various forms of flow shown above can be reordered, added to, or have steps deleted. For example, the steps described in the present application can be performed in parallel, in series, or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, and the present application is not limited herein.

[0150] The above detailed description does not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A seat belt wearing detection method characterized by comprising: The method comprises: acquiring a driving-in-vehicle image of a vehicle user in real time; detecting the driving-in-vehicle image to determine a body key segment and a seat belt area of the vehicle user; detecting a number of seat belt pixel points where the body key segment of the vehicle user overlaps the seat belt area; determining a seat belt wearing detection result of the vehicle user according to the number of seat belt pixel points.

2. The method of claim 1, wherein, The determining of the seat belt wearing detection result of the vehicle user according to the number of seat belt pixel points comprises: acquiring a seat belt pixel point range; comparing the number of seat belt pixel points with the seat belt pixel point range; when the number of seat belt pixel points is outside the seat belt pixel point range, determining that a seat belt wearing state of the vehicle user is not wearing a seat belt; when the number of seat belt pixel points is within the seat belt pixel point range, determining that the seat belt wearing state of the vehicle user is wearing a seat belt.

3. The method of claim 2, wherein, The determining of the seat belt wearing state of the vehicle user when the number of seat belt pixel points is within the seat belt pixel point range comprises: when a first number of seat belt pixel points or a second number of seat belt pixel points is within the seat belt pixel point range and a number of other seat belt pixel points is outside the seat belt pixel point range, determining that the seat belt wearing state of the vehicle user is wearing a seat belt; wherein the body key segment comprises a first body key segment and a second body key segment; the first body key segment and the second body key segment are body key segments that overlap the seat belt area; the seat belt pixel points comprise the first seat belt pixel points and the second seat belt pixel points; wherein the the first seat belt pixel points are seat belt pixel points where the first body key segment of the vehicle user overlaps the seat belt area; and the second seat belt pixel points are seat belt pixel points where the second body key segment of the vehicle user overlaps the seat belt area; detecting, according to the driving-in-vehicle image, whether there is a seat belt occlusion event; when it is detected that there is the seat belt occlusion event, determining that a seat belt wearing state of the vehicle user is wearing a seat belt and a seat belt wearing posture of the vehicle user is correct; when it is detected that there is no seat belt occlusion event, determining that the seat belt wearing posture of the vehicle user is incorrect.

4. The method of claim 2, wherein, The acquiring of the seat belt pixel point range comprises: acquiring body type information of the vehicle user; determining a seat belt pixel point range according to the body type information of the vehicle user.

5. The method of claim 1, wherein, The detecting of the driving-in-vehicle image to determine the body key segment and the seat belt area of the vehicle user comprises: inputting the driving-in-vehicle image into a pre-trained seat belt detection model to output at least two body key points and a seat belt area of the vehicle user; wherein training samples of the seat belt detection model comprise a driving-in-vehicle image sample, at least two body key point samples and a seat belt area sample of a historical vehicle user; Connecting each of the body key points to obtain at least one body key line segment.

6. The method of claim 5, wherein, The model structure of the safety belt detection model comprises a feature extraction layer, a feature fusion layer, a first detection head layer, and a second detection head layer. The inputting of the in-vehicle driving image of the vehicle user into the pre-trained safety belt detection model and the outputting of at least two body key points and a safety belt region of the vehicle user comprise: The feature extraction layer is used to extract features of the in-vehicle driving image of the vehicle user to obtain driving image semantic features in at least one dimension; The feature fusion layer is used to fuse the driving image semantic features in each dimension, to obtain a driving image feature map; The first detection head layer is used to detect the driving image feature map to obtain at least two body key points of the vehicle user; The second detection head layer is used to detect the driving image feature map to obtain a safety belt region.

7. The method of claim 5, wherein, The body key points comprise a neck key point, shoulder key points, and opposite thigh key points of a safety belt buckle.

8. The method of claim 1, wherein, After the determination of the safety belt wearing detection result of the vehicle user according to the number of safety belt pixel points, the method further comprises: According to the safety belt wearing detection result of the vehicle user, safety belt wearing reminding information is determined, and the vehicle user is fed back.

9. A seat belt wearing detection device characterized by comprising: The device comprises: an in-vehicle driving image acquisition module configured to acquire an in-vehicle driving image of a vehicle user in real time; a body key line segment detection module configured to detect the in-vehicle driving image to determine a body key line segment and a safety belt region of the vehicle user; a safety belt pixel point detection module configured to detect the number of safety belt pixel points overlapped by the body key line segment and the safety belt region of the vehicle user; a safety belt wearing detection result determination module configured to determine a safety belt wearing detection result of the vehicle user according to the number of safety belt pixel points.

10. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the safety belt wearing detection method in any one of claims 1-8.

11. A computer readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for enabling the processor to execute the safety belt wearing detection method in any one of claims 1-8 when executed by the processor.

12. A computer program product, characterised in that, The computer program product comprises a computer program, and the computer program, when executed by the processor, implements the safety belt wearing detection method according to any one of claims 1-8.

Images