Safety belt detection method, device, equipment and storage medium

Abstract

Embodiments of the present application disclose a safety belt detection method, device and equipment and a storage medium. The method comprises: obtaining a to-be-detected image of a current vehicle, detecting a human body region of the to-be-detected image to obtain a target human body image; obtaining a key point heat map corresponding to at least one key point in the target human body image based on a pre-trained safety belt key point heat detection model according to the target human body image; and determining that a person in the current vehicle is a safety belt wearing person if the key point heat map meets a preset safety belt detection rule. The embodiments of the present application improve the accuracy of safety belt detection.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a seat belt detection method, apparatus, device, and storage medium. Background Technology

[0002] As an important measure to ensure safe driving and reduce the death and injury rate in traffic accidents, traffic management departments strictly require drivers and passengers to wear seat belts while driving. Therefore, intelligent seat belt detection technology is gradually becoming an important development direction for future intelligent transportation systems.

[0003] Currently, seatbelt detection typically employs traditional image processing algorithms to extract edge texture features or machine learning classification algorithms to train a binary classification model. However, existing technologies have low accuracy in seatbelt detection, which can lead to serious personal injury to vehicle occupants. Summary of the Invention

[0004] This application provides a seat belt detection method, apparatus, device, and storage medium to improve the accuracy of seat belt detection.

[0005] In a first aspect, embodiments of this application provide a seatbelt detection method, the method comprising:

[0006] Acquire the image of the current vehicle to be detected, and perform human body region detection on the image to be detected to obtain the target human body image;

[0007] Based on the target human image, a key point heatmap corresponding to at least one key point in the target human image is obtained using a pre-trained seat belt key point thermal detection model.

[0008] Determine whether the heatmap of the key points meets the preset seat belt detection rules. If so, determine that the occupants of the current vehicle are seat belt wearers.

[0009] Secondly, embodiments of this application provide a seatbelt detection device, the device comprising:

[0010] The target human image acquisition module is used to acquire the image to be detected of the current vehicle, and to detect the human body region in the image to be detected to obtain the target human image.

[0011] The key point heatmap acquisition module is used to obtain a key point heatmap corresponding to at least one key point in the target human image based on the pre-trained seat belt key point heatmap detection model.

[0012] The seat belt detection module is used to determine whether the key point heat map meets the preset seat belt detection rules. If so, it determines that the occupants of the current vehicle are seat belt wearers.

[0013] Thirdly, embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the seat belt detection method as described in any of the embodiments of this application.

[0014] Fourthly, embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the seatbelt detection method as described in any of the embodiments of this application.

[0015] This application embodiment acquires a target human body image of the current vehicle by obtaining a human body region detection within the target human body image. Based on the target human body image, a key point heatmap corresponding to at least one key point in the target human body image is obtained using a pre-trained seat belt key point thermal detection model. It then determines whether the key point heatmap meets preset seat belt detection rules; if so, it identifies the occupants of the current vehicle as seat belt wearers. This solution, by employing a seat belt key point thermal detection model to detect key points in the target human body image and determining whether occupants are wearing seat belts based on the key point heatmap output by the model, achieves accurate detection of seat belt wear, improves the efficiency of seat belt detection, avoids misjudgments due to inaccurate seat belt detection, and reduces potential harm to vehicle occupants. Attached Figure Description

[0016] Figure 1 This is a schematic flowchart of a seat belt detection method according to Embodiment 1 of this application;

[0017] Figure 2 This is a schematic flowchart of a seat belt detection method according to Embodiment 2 of this application;

[0018] Figure 3 This is a structural block diagram of a seat belt detection device according to Embodiment 3 of this application;

[0019] Figure 4 This is a schematic diagram of the structure of an electronic device according to Embodiment 4 of this application. Detailed Implementation

[0020] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0021] Example 1

[0022] Figure 1 This is a flowchart illustrating a seatbelt detection method provided in Embodiment 1 of this application. This embodiment is applicable to detecting whether occupants of a moving vehicle are wearing seatbelts. The method can be executed by a seatbelt detection device, which can be implemented in software and / or hardware, such as... Figure 1 As shown, the method specifically includes the following steps:

[0023] S110. Obtain the image of the current vehicle to be detected, perform human body region detection on the image to be detected, and obtain the target human body image.

[0024] The image of the vehicle to be detected can be acquired by an image acquisition device, which can be a camera. For example, the image acquisition device can be installed at a road checkpoint. The device acquires images of vehicles passing through the checkpoint and uses these images as the image to be detected. The image to be detected can be an image including the upper torso region of a human body acquired by the image acquisition device, or it can be a sequence of images including the upper torso region of a human body acquired by the image acquisition device within a preset number of frames. The number of frames can be predetermined by relevant technicians; for example, the preset number of frames can be 10 frames.

[0025] The human body region can be the area containing the upper torso of a human in the image to be detected. The target human body image can be the image corresponding to the area containing the upper torso of a human in the image to be detected, or it can be an image after the area containing the upper torso of a human in the image to be detected has been marked or bounded. For example, the coordinate points and / or the length and width of the area containing the upper torso of a human can be marked in the image to be detected to obtain the target human body image. Alternatively, the area containing the upper torso of a human in the image to be detected can be bounded to obtain the target human body image.

[0026] A pre-trained torso region detection model can be used to detect human body regions in the image to be detected, thereby obtaining the target human body image. For example, human body region images are acquired, labeled, and used as the training set for training the torso region detection model.

[0027] For example, the label for a human body region image can be the coordinates of the human torso within the image. For instance, a quadrilateral can be used to select the area containing the human body within the image. Specifically, a preset image labeling tool, such as LabelMe, can be used to select this area. Correspondingly, the label for the human body region image can be the coordinates of the vertices of the quadrilateral that selected the human body's location. For example, the label for the human body region image could be "(128, 80), (150, 80), (128, 60), (150, 60)". It should be noted that the image labeling tool can automatically generate the label for the human body region image based on the selected area.

[0028] For example, the label for a human body region image can also be the coordinates of the region where the human body is located, as well as the length and width of the region. For instance, the label for a human body region image could also be "(128, 80), 50px, 30px", where (128, 80) represents the coordinates of the top-left vertex of the quadrilateral that selects the human body region in the image, and 50px and 30px represent the length and width of the quadrilateral region, respectively. The location of the human body in the human body region image can be determined based on the coordinates of the top-left vertex of the quadrilateral and the length and width of the quadrilateral region.

[0029] Labeled human body region images are input as a training set into a pre-defined torso region detection model. The model is trained according to pre-defined training parameters. The torso region detection model can be pre-defined by relevant technical personnel; for example, it can be a CenterNet model. Model training parameters can include the number of iterations, for example, 500 iterations. The trained torso region detection model is then used to detect regions in the image to be detected, yielding the target human body image.

[0030] It should be noted that, in order to improve the accuracy and efficiency of human body region detection in the image to be detected, the location image of the human body region can be determined first based on the image to be detected, and then the target human body image can be determined based on the location image of the human body region.

[0031] In one optional embodiment, detecting human body regions in the image to be detected to obtain a target human body image includes: detecting the windshield region of the image to be detected according to a preset human body region location detection model to obtain a human body region location image; and obtaining the target human body image based on the human body region location image and the preset human body region detection model.

[0032] The personnel location detection model can be pre-trained by relevant technical personnel using a preset first network model, such as the CenterNet model. A training set of samples is obtained to train the first network model. These samples are then manually labeled in small batches to obtain a small batch of labeled training samples. This labeled small batch of training samples is used as input to the first network model. Based on a preset number of iterations, the first network model is iteratively trained in a semi-automatic manner, and the trained first network model is used as the personnel location detection model. The training set required for training the preset first network model can be historical images of the target area obtained from road checkpoints over historical periods.

[0033] For example, 100,000 historical images to be detected are obtained as the sample training set required for training the preset first network model. A small batch of manually labeled samples is used to label 5,000 historical images to be detected. Specifically, a preset image labeling tool can be used to label the windshield region in the historical images to be detected. These 5,000 manually labeled historical images are used as the first sample training set for the first network model to train it, resulting in a first candidate human region location detection model trained on these 5,000 historical images. The remaining 95,000 historical images to be detected are used as input to the first candidate human region location detection model. The model is then used to detect the windshield region on these 95,000 historical images. Images with accurate windshield region detection are extracted and merged with the first sample training set to obtain a second sample training set. The first candidate human region location detection model is then trained again to obtain a second candidate human region location detection model. For example, if the detection result shows that 5,000 out of 95,000 historical images were accurately detected, these 5,000 images are merged with the 5,000 images in the first sample set to obtain the second sample training set. The first candidate person region location detection model is then trained to obtain the second candidate person region location detection model. The remaining 90,000 historical images are used as input to the second candidate person region location detection model. Based on the second candidate person region location detection model, the windshield region of the 90,000 historical images is detected. Based on the detection results, it is determined whether the second candidate person region location detection model should be used as the person region location detection model. Specifically, if the windshield region of all 90,000 historical images has been accurately detected, then the second candidate person region location detection model can be used as the person region location detection model; otherwise, the above steps are repeated to continue training the second candidate person region location detection model.

[0034] By using a small batch of manually labeled training samples, and based on a semi-automated model training method, the workload of manual annotation is reduced, the training efficiency of the model is improved, and thus the determination efficiency of the personnel area location detection model is improved.

[0035] The human region detection model can be pre-trained by relevant technicians using a pre-defined second network model. For example, the pre-defined second network model can be an LSTM (Long Short-Term Memory) model, or it can be the same as the first network model. Images of the human regions to be trained are obtained as the third training set for the second network model. This third training set is labeled and used as input to the second network model. Based on a pre-defined number of iterations, the second network model is iteratively trained in a semi-automatic manner, and the trained second network model is used as the human region detection model. Based on the trained human region detection model, the human region images are detected to obtain the target human image.

[0036] This optional embodiment first detects the windshield area of ​​the image to be detected based on a preset personnel area location detection model to obtain a personnel area location image; then, based on the personnel area location image and a preset human body area detection model, it obtains a target human body image, thereby improving the accuracy and efficiency of human body area detection in the image to be detected.

[0037] S120. Based on the target human body image, and using a pre-trained seat belt key point thermal detection model, obtain a key point thermal map corresponding to at least one key point in the target human body image.

[0038] The seatbelt key point thermal detection model can be pre-trained by relevant technicians. Key points can be coordinate points within the seatbelt area of ​​a target human image. A key point heatmap can be an image used to determine the visibility of the seatbelt location area where the key points are located. The key point heatmap can also be an image used to determine the visibility of the area corresponding to the key point; the area corresponding to the key point can be a local region within the seatbelt area. For example, if there are 5 key points within the seatbelt area, and these 5 key points can be equally divided within the seatbelt area, the area corresponding to each key point can be a local region within the seatbelt area where the 5 key points are located.

[0039] Specifically, the more distinct the features of the seatbelt area to which a key point belongs, the more pronounced the contrast features of the corresponding key point heatmap. The number of key point heatmap images corresponds to the number of key points in the target human body image. For example, if there are 5 key points in the target human body image, then there are 5 key point heatmap images, meaning one key point corresponds to one key point heatmap.

[0040] For example, a seatbelt key point thermal detection model can be pre-constructed by relevant technical personnel. This model could be a MobileNet model. A fourth training set for training the seatbelt key point thermal detection model is obtained. Based on preset model training parameters, the pre-constructed model is trained to obtain the trained seatbelt key point thermal detection model. The fourth training set can consist of labeled images of the human torso with and without seatbelts. The labels can contain the coordinates of at least one key point on the seatbelt area. Model training parameters can include the number of model iterations and the image size of the key point heatmap.

[0041] S130. Determine whether the key point heat map meets the preset seat belt detection rules. If so, determine that the occupants of the current vehicle are seat belt wearers.

[0042] Seatbelt detection rules can be preset by relevant technical personnel. For example, a seatbelt detection rule could be to determine whether the contrast of at least one keypoint heatmap meets a preset contrast judgment condition. This preset contrast judgment condition could be that the contrast of the keypoint heatmap is greater than a preset contrast threshold, for example, a contrast threshold of 0.95. It should be noted that the contrast threshold can be set differently by relevant technical personnel according to actual circumstances. If there are keypoint heatmaps that meet the preset contrast judgment condition, then it is determined whether the number of keypoint heatmaps meeting the contrast judgment condition exceeds a preset number threshold. If yes, then the occupants of the current vehicle are determined to be seatbelt wearers; otherwise, the occupants of the current vehicle are determined to be unbelt wearers. The number threshold can be determined by relevant technical personnel based on the number of keypoint heatmaps. For example, if the number of keypoint heatmaps is 5, then the number threshold can be 4.

[0043] In one optional embodiment, determining whether the key point heatmap meets the preset seat belt detection rules, and if so, determining that the occupants of the current vehicle are seat belt wearers, includes: determining whether there are candidate heat values ​​in at least one key point heatmap that meet the preset heat value size requirements; if so, determining whether the number of candidate heat values ​​is equal to or greater than a preset number threshold; if so, determining that the occupants of the current vehicle are seat belt wearers.

[0044] The heatmap value can be the numerical value corresponding to each pixel in the keypoint heatmap, specifically generated during the detection of the target human image using the seatbelt keypoint heatmap detection model. The preset heatmap value can be pre-set by relevant technical personnel; for example, the preset heatmap value could be 0.8. The candidate heatmap value can be the heatmap value with the highest numerical value in the keypoint heatmap.

[0045] For example, the position coordinates of each pixel in at least one keypoint heatmap are obtained. Based on the obtained position coordinates, the heatmap values ​​are traversed. After the traversal, the heatmap with the largest value in each keypoint heatmap is taken as a candidate heatmap value. It is then determined whether the candidate heatmap values ​​in each keypoint heatmap are greater than or equal to a preset heatmap value. If at least one keypoint heatmap has a candidate heatmap value greater than or equal to the preset heatmap value, it is determined whether the number of keypoint heatmaps satisfying the condition of having a candidate heatmap value greater than or equal to the preset heatmap value is equal to or greater than a preset threshold. If yes, the occupants of the current vehicle are determined to be wearing seat belts; otherwise, the occupants of the current vehicle are determined to be not wearing seat belts. If no at least one keypoint heatmap has a candidate heatmap value greater than or equal to the preset heatmap value, the occupants of the current vehicle are determined to be not wearing seat belts.

[0046] Specifically, the preset heatmap value is 0.8, and the preset quantity threshold is 4 images. The key point heatmaps output by the seat belt key point thermal detection model are heatmap A, heatmap B, heatmap C, heatmap D, and heatmap E. The heatmap values ​​of the five heatmaps are iterated based on the position coordinates of each pixel. After the traversal, the candidate heat value corresponding to heatmap A is 0.75, the candidate heat value corresponding to heatmap B is 0.85, the candidate heat value corresponding to heatmap A is 0.95, the candidate heat value corresponding to heatmap A is 0.95, and the candidate heat value corresponding to heatmap A is 0.9. Therefore, it can be determined that the candidate heat values ​​corresponding to heatmaps B, C, D, and E are greater than or equal to the preset heat value of 0.8, and the candidate heat value corresponding to heatmap A is less than the preset heat value of 0.8. Therefore, it can be determined that the number of key point heatmaps that satisfy the condition of candidate heat values ​​being greater than or equal to the preset heat value is 4, which is equal to the preset threshold of 4. Thus, it can be determined that the occupants of the current vehicle are seatbelt wearers.

[0047] If the candidate heatmap value corresponding to heatmap A is 0.75, the candidate heatmap value corresponding to heatmap B is 0.75, the candidate heatmap value corresponding to heatmap A is 0.95, the candidate heatmap value corresponding to heatmap A is 0.95, and the candidate heatmap value corresponding to heatmap A is 0.9, then it can be determined that the candidate heatmap values ​​corresponding to heatmaps C, D, and E are greater than or equal to the preset heatmap value of 0.8, and the candidate heatmap values ​​corresponding to heatmaps A and B are less than the preset heatmap value of 0.8. Therefore, it can be determined that the number of key point heatmaps that satisfy the condition of candidate heatmap values ​​being greater than or equal to the preset heatmap value is 3, which is less than the preset threshold of 4. Thus, it can be determined that the occupants of the current vehicle are not wearing seat belts.

[0048] This optional embodiment improves the accuracy of determining whether the occupants of the current vehicle are wearing seat belts by judging whether there are candidate heat values ​​that meet the preset heat value size requirements in at least one key point heat map, and judging whether the number of candidate heat values ​​is equal to or greater than a preset number threshold.

[0049] This application embodiment acquires a target human body image of the current vehicle by obtaining a human body region detection within the target human body image. Based on the target human body image, a key point heatmap corresponding to at least one key point in the target human body image is obtained using a pre-trained seat belt key point thermal detection model. It then determines whether the key point heatmap meets preset seat belt detection rules; if so, it identifies the occupants of the current vehicle as seat belt wearers. This solution, by employing a seat belt key point thermal detection model to detect key points in the target human body image and determining whether occupants are wearing seat belts based on the key point heatmap output by the model, achieves accurate detection of seat belt wear, improves the efficiency of seat belt detection, avoids inaccurate seat belt detection, and reduces potential harm to vehicle occupants.

[0050] Example 2

[0051] Figure 2 This is a flowchart illustrating a seatbelt detection method provided in Embodiment 2 of this application. This embodiment is an optimization and improvement based on the above-mentioned technical solutions.

[0052] Furthermore, before the step "acquire the image to be detected of the current vehicle, perform human body region detection on the image to be detected, and obtain the target human body image", add the step "acquire the seat belt image to be trained, and input it into the pre-constructed seat belt key point thermal detection model as a sample training set; based on the sample training set and the pre-constructed seat belt key point thermal detection model, and based on the preset model training parameters, obtain the trained seat belt key point thermal detection model". This improves the training process of the seat belt key point thermal detection model.

[0053] like Figure 2 As shown, the method includes the following specific steps:

[0054] S210. Obtain the seat belt image to be trained and input it into the pre-built seat belt key point thermal detection model as the sample training set of the seat belt key point thermal detection model.

[0055] The acquired seatbelt images to be trained are used as the sample training set for training the seatbelt key point thermal detection model. The seatbelt images to be trained can include images of drivers wearing seatbelts and images of drivers not wearing seatbelts, and the number of samples for each type of image can be preset by relevant technical personnel. For example, the number of samples for images of drivers wearing seatbelts can be 100,000, and the number of samples for images of drivers not wearing seatbelts can be 10,000.

[0056] For example, the seatbelt images to be trained can be pre-labeled, and a pre-built seatbelt keypoint thermal detection model can be trained using supervised model training. The pre-built seatbelt keypoint thermal detection model can be a MobileNet model. The labels for the seatbelt images to be trained can be the coordinates of the keypoints and their corresponding visibility. The locations of the keypoints can be pre-determined by relevant technical personnel. For example, the seatbelt from the shoulder to the waist of the upper torso can be divided into preset equal parts, and the midpoint of each part can be used as a keypoint, with the coordinates of the keypoint being the coordinates of the midpoint. The preset parts can be 5, and correspondingly, the number of keypoints can be 5. The visibility of the keypoint can be 0 or 1. 1 indicates that the seatbelt area to which the keypoint belongs is visible, and 0 indicates that the seatbelt area to which the keypoint belongs is not visible. For example, if the seatbelt area to which the keypoint belongs is not visible due to arm or clothing obstruction, then the visibility of the keypoint is 0. The label for an image of a driver not wearing a seatbelt can be set to -1 by default, and the visibility of key points can be set to 0.

[0057] S220. Based on the sample training set and the pre-built seat belt key point thermal detection model, and using the preset model training parameters, obtain the trained seat belt key point thermal detection model.

[0058] The preset model training parameters may include the image size of the heatmap corresponding to the preset key points or the number of model iterations. Specifically, the image size of the heatmap corresponding to the preset key points can be 64×64, and the number of model iterations can be 500. For example, within the number of model iterations, the sample training set is input into the pre-built seatbelt key point thermal detection model. After iterative training is completed, the trained seatbelt key point thermal detection model is obtained.

[0059] In one optional embodiment, at least one seatbelt key point is marked in the seatbelt image to be trained; correspondingly, based on the sample training set and the pre-built seatbelt key point thermal detection model, and based on preset model training parameters, a trained seatbelt key point thermal detection model is obtained, including: determining the actual value of the thermal map corresponding to at least one seatbelt key point based on the coordinates of the seatbelt key points marked in the seatbelt image to be trained and a preset Gaussian function; determining at least one predicted value of the thermal map based on the seatbelt image to be trained and the thermal map regression function in the pre-built seatbelt key point thermal detection model; determining whether the loss value at the current iteration number meets the preset training completion condition based on the actual value and the predicted value of the thermal map and a preset loss function; if so, the training of the seatbelt key point thermal detection model is completed.

[0060] The seatbelt images to be trained are used as the fourth training set. Before training the seatbelt key point thermal detection model, the image size of the fourth training set can be preset, for example, the image size of the fourth training set can be 256×256. A mapping relationship between the pixel coordinates of the images in the fourth training set and the pixel coordinates of the heatmap images is pre-constructed. This mapping relationship can be achieved by proportionally reducing the pixel coordinates of the corresponding images. For example, if the image size of the fourth training set is 256×256 and the preset heatmap image size is 64×64, and if the pixel coordinates of a key point in any image in the fourth training set are (128, 128), then according to the pre-constructed mapping relationship between the pixel coordinates of the fourth training set and the heatmap images, the pixel coordinates of the corresponding heatmap image can be determined to be (32, 32).

[0061] The preset formula for calculating the Gaussian function can be as follows:

[0062]

[0063] Where G(x,y) is the actual value of the heatmap, (x,y) represents the coordinates of each pixel in the heatmap, σ=1.5. If the image size of the heatmap is 64×64, then x∈(0,64), y∈(0,64). (x k ,y k ) represents the coordinates of the key points of the seat belt.

[0064] The heatmap regression function in the pre-built seatbelt key point thermal detection model can be a linear regression function. Based on the weight parameters and bias parameters generated during the training of the seatbelt key point thermal detection model, at least one predicted heatmap value can be determined using the thermal regression function. For example, if the pre-built seatbelt key point thermal detection model is a MobileNet model, the fully connected layers and classification layers in the MobileNet model can be removed. The removed MobileNet model can then be trained to obtain the weight parameters and bias parameters generated during training. Based on the weight parameters and bias parameters, and using the preset thermal regression function, the predicted heatmap value of the key point to be confirmed can be determined.

[0065] For example, key points determined based on heatmap predictions can be used as key points to be confirmed. If the coordinates of the key points to be confirmed and the coordinates of the key points of the seat belt labeled in the seat belt image to be trained meet a preset coordinate error threshold, then the seat belt key point thermal detection model can be considered to have completed training. The coordinate error threshold can be preset by relevant technical personnel; for example, the coordinate error threshold can be 0.5.

[0066] Based on a preset loss function, it is determined whether the loss value at the current iteration number meets the preset training completion conditions. The loss function can be preset by relevant technical personnel. In an optional embodiment, the formula for the loss function is as follows:

[0067]

[0068]

[0069] C = θ*A - ω*ln(1 + (θ / ε)) α-y );

[0070] Where y is the predicted value from the heatmap. The values ​​are the actual values ​​from the heatmap, while α, ω, θ, and ε are preset parameters. For example, α = 2.1, ω = 14.1, θ = 0.5, and ε = 1.0. Determining the loss value using the above loss function improves the accuracy of the loss value determination.

[0071] In one optional embodiment, determining whether the loss value at the current iteration number meets the preset training completion condition includes: determining whether the loss value at the current iteration number meets the preset loss value change condition; if so, the preset training completion condition is met.

[0072] The preset loss value change condition can be that the range of change in the loss value is less than a preset range threshold, for example, the preset range threshold can be 0.2. The preset training completion condition can be that the seat belt key point thermal detection model converges. If the loss value meets the preset loss value change condition, it can be said that the loss value has stabilized and the seat belt key point thermal detection model has converged.

[0073] For example, based on a preset loss function, it is determined whether the loss value at the current iteration number meets the preset training completion conditions; if yes, the training of the seat belt key point thermal detection model is completed; otherwise, the training of the seat belt key point thermal detection model continues. By judging whether the loss value at the current iteration number meets the preset loss value change conditions, the accuracy of judging whether the training of the seat belt key point thermal detection model is completed is improved.

[0074] Optionally, the number of iterations for model training can be preset. The seat belt key point thermal detection model can be trained within the preset number of iterations until the model training within the preset number of iterations is completed. The trained model is then used as the seat belt key point thermal detection model.

[0075] S230. Obtain the image of the current vehicle to be detected, perform human body region detection on the image to be detected, and obtain the target human body image.

[0076] S240. Based on the target human body image, and using a pre-trained seat belt key point thermal detection model, obtain a key point thermal map corresponding to at least one key point in the target human body image.

[0077] S250. Determine whether the key point heat map meets the preset seat belt detection rules. If so, determine that the occupants of the current vehicle are seat belt wearers.

[0078] This embodiment acquires seatbelt images to be trained and inputs them into a pre-constructed seatbelt key point thermal detection model as a sample training set. Based on the sample training set and the pre-constructed seatbelt key point thermal detection model, and using preset model training parameters, a trained seatbelt key point thermal detection model is obtained. This achieves the acquisition of the seatbelt key point thermal detection model. By using seatbelt images with key point labels to train the preset seatbelt key point thermal detection model, the trained model has strong generalization ability and high training accuracy. This enables accurate detection of whether occupants are wearing seatbelts, improves the efficiency of seatbelt detection, and avoids potential losses to vehicle occupants due to inaccurate seatbelt detection.

[0079] Example 3

[0080] Figure 3 This is a schematic diagram of a seatbelt detection device provided in Embodiment 3 of this application. The seatbelt detection device provided in this embodiment is applicable to detecting whether occupants of a moving vehicle are wearing seatbelts. This device can be implemented using software and / or hardware. Figure 3 As shown, the device specifically includes: a target human image acquisition module 301, a key point heatmap acquisition module 302, and a seatbelt detection module 303. Among them,

[0081] The target human image acquisition module 301 is used to acquire the image to be detected of the current vehicle, and to detect the human body region in the image to be detected to obtain the target human image.

[0082] The key point heatmap acquisition module 302 is used to obtain a key point heatmap corresponding to at least one key point in the target human image based on the pre-trained seat belt key point heatmap detection model.

[0083] The seat belt detection module 303 is used to determine whether the key point heat map meets the preset seat belt detection rules. If so, it determines that the occupants of the current vehicle are seat belt wearers.

[0084] This application embodiment acquires a target human body image of the current vehicle by obtaining a target human body image and performing human body region detection on the target human body image. Based on the target human body image, a key point heatmap corresponding to at least one key point in the target human body image is obtained using a pre-trained seat belt key point thermal detection model. It then determines whether the key point heatmap meets the preset seat belt detection rules; if so, it determines that the occupants of the current vehicle are wearing seat belts. This solution, by employing a seat belt key point thermal detection model to detect key points in the target human body image and determining whether the occupants of the current vehicle are wearing seat belts based on the key point heatmap output by the model, achieves accurate detection of whether occupants are wearing seat belts, improves the efficiency of seat belt detection, avoids misjudgments caused by inaccurate seat belt detection, and reduces losses to the vehicle's occupants.

[0085] Optionally, the target human image acquisition module 301 includes:

[0086] The personnel area location image acquisition unit is used to detect the front window area of ​​the image to be detected according to a preset personnel area location detection model to obtain a personnel area location image;

[0087] The target human body image acquisition unit is used to obtain a target human body image based on the human body region location image and a preset human body region detection model.

[0088] Optionally, the device further includes:

[0089] The seat belt image acquisition module is used to acquire seat belt images to be trained, and input them into the pre-constructed seat belt key point thermal detection model as a sample training set of the seat belt key point thermal detection model.

[0090] The thermal detection model training module is used to obtain the trained thermal detection model of the seat belt key points based on the sample training set and the pre-constructed thermal detection model of the seat belt key points, and based on the preset model training parameters.

[0091] Optionally, at least one key point of the seat belt is marked in the seat belt image to be trained;

[0092] Correspondingly, the thermal detection model training module includes:

[0093] The actual value determination unit of the heat map is used to determine the actual value of the heat map corresponding to at least one key point of the seat belt based on the coordinates of the key points of the seat belt marked on the seat belt image to be trained and a preset Gaussian function.

[0094] The heatmap prediction value determination unit is used to determine at least one heatmap prediction value based on the seat belt image to be trained and the heatmap regression function in the pre-built seat belt key point thermal detection model.

[0095] The training completion condition judgment unit is used to determine whether the loss value at the current iteration number meets the preset training completion condition based on the actual value and predicted value of the heat map and a preset loss function.

[0096] The thermal detection model training unit is used to complete the training of the thermal detection model for the key points of the seat belt if the loss value at the current iteration number meets the preset training completion conditions.

[0097] Optionally, the training completion condition judgment unit includes:

[0098] The loss value judgment subunit is used to determine whether the loss value at the current iteration number meets the preset loss value change conditions;

[0099] The training completion condition judgment subunit is used to determine if the training completion condition is met if the loss value at the current iteration number meets the preset loss value change condition.

[0100] Optionally, the formula for the loss function is as follows:

[0101]

[0102]

[0103] C = θ*A - ω*ln(1 + (θ / ε)) α-y );

[0104] Where y is the predicted value from the heatmap. The values ​​are the actual values ​​from the heatmap, while α, ω, θ, and ε are preset parameters.

[0105] Optionally, the seatbelt detection module 303 includes:

[0106] The candidate thermal value determination unit is used to determine whether there are candidate thermal values ​​that meet the preset thermal value size requirements in at least one key point thermal map;

[0107] The thermal value quantity determination unit is used to determine whether the quantity of the candidate thermal values ​​is equal to or greater than a preset quantity threshold if there are candidate thermal values ​​that meet the preset thermal value size requirements in at least one key point thermal map.

[0108] The seatbelt occupant determination unit is used to determine that the occupants of the current vehicle are seatbelt occupants if the number of candidate thermal values ​​is equal to or greater than a preset threshold.

[0109] The aforementioned seat belt detection device can execute the seat belt detection method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects for executing each seat belt detection method.

[0110] Example 4

[0111] Figure 4 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of this application. Figure 4 A block diagram is shown of an exemplary electronic device 400 suitable for implementing embodiments of the present application. Figure 4 The electronic device 400 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0112] like Figure 4 As shown, the electronic device 400 is presented in the form of a general-purpose computing device. The components of the electronic device 400 may include, but are not limited to: one or more processors or processing units 401, system memory 402, and bus 403 connecting different system components (including system memory 402 and processing unit 401).

[0113] Bus 403 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0114] Electronic device 400 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 400, including volatile and non-volatile media, removable and non-removable media.

[0115] System memory 402 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 404 and / or cache memory 405. Electronic device 400 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 406 may be used to read and write non-removable, non-volatile magnetic media (… Figure 4 Not shown; usually referred to as a "hard drive"). Although Figure 4Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 403 via one or more data media interfaces. Memory 402 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application.

[0116] A program / utility 408 having a set (at least one) of program modules 407 may be stored, for example, in memory 402. Such program modules 407 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 407 typically perform the functions and / or methods described in the embodiments of this application.

[0117] Electronic device 400 can also communicate with one or more external devices 409 (e.g., keyboard, pointing device, display 410, etc.), and with one or more devices that enable a user to interact with the electronic device 400, and / or with any device that enables the electronic device 400 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 411. Furthermore, electronic device 400 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 412. As shown, network adapter 412 communicates with other modules of electronic device 400 via bus 403. It should be understood that, although... Figure 4 As not shown, other hardware and / or software modules may be used in conjunction with electronic device 400, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0118] The processing unit 401 executes various functional applications and data processing by running programs stored in the system memory 402, such as implementing a seat belt detection method provided in the embodiments of this application.

[0119] Example 5

[0120] This application also provides a storage medium containing computer-executable instructions, on which a computer program is stored. When the program is executed by a processor, it implements the seat belt detection method provided in this application, including: acquiring a target image of the current vehicle; detecting human body regions in the target image to obtain a target human body image; obtaining a key point heatmap corresponding to at least one key point in the target human body image based on a pre-trained seat belt key point thermal detection model; determining whether the key point heatmap meets the preset seat belt detection rules, and if so, determining that the occupants of the current vehicle are seat belt wearers.

[0121] The computer storage medium in this application embodiment can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0122] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0123] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0124] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0125] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A seat belt detection method characterized by, The method comprises the following steps: obtaining a to-be-detected image of a current vehicle, detecting a human body region of the to-be-detected image, and obtaining a target human body image; obtaining, according to the target human body image, a key point heat map corresponding to at least one key point in the target human body image based on a pre-trained safety belt key point heat detection model; determining whether the key point heat map meets a preset safety belt detection rule, and if so, determining that an occupant in the current vehicle is a safety belt wearing person, wherein the determination comprises the following steps: determining whether a candidate heat value meeting a preset heat value size requirement exists in at least one key point heat map; if so, determining whether the number of the candidate heat values is equal to or greater than a preset number threshold; if so, determining that the occupant in the current vehicle is the safety belt wearing person; before the step of obtaining the to-be-detected image of the current vehicle, detecting the human body region of the to-be-detected image, and obtaining the target human body image, the method further comprises the following steps: obtaining a to-be-trained safety belt image as a sample training set of the safety belt key point heat detection model, and inputting the to-be-trained safety belt image into a pre-constructed safety belt key point heat detection model; obtaining a trained safety belt key point heat detection model based on a preset model training parameter according to the sample training set and the pre-constructed safety belt key point heat detection model.

2. The method of claim 1, wherein, The step of detecting the human body region of the to-be-detected image and obtaining the target human body image comprises the following steps: detecting a front window region of the to-be-detected image according to a preset personnel region position detection model, and obtaining a personnel region position image; obtaining the target human body image based on a preset human body region detection model according to the personnel region position image.

3. The method of claim 1, wherein, at least one safety belt key point marked in the to-be-trained safety belt image; correspondingly, the step of obtaining the trained safety belt key point heat detection model based on the preset model training parameter according to the sample training set and the pre-constructed safety belt key point heat detection model comprises the following steps: determining an actual value of a heat map corresponding to at least one safety belt key point according to a coordinate of the safety belt key point marked in the to-be-trained safety belt image and a preset Gaussian function; determining at least one heat map prediction value according to the to-be-trained safety belt image and a heat map regression function in the pre-constructed safety belt key point heat detection model; determining whether a loss value at a current iteration meets a preset training completion condition based on a preset loss function according to the actual value and the prediction value of the heat map; if so, completing the training of the safety belt key point heat detection model.

4. The method of claim 3, wherein, The step of determining whether the loss value at the current iteration meets the preset training completion condition comprises the following steps: determining whether the loss value at the current iteration meets a preset loss value change condition; if so, the preset training completion condition is met.

5. The method of claim 3, wherein, The formula of the loss function is as follows: ； ； ； wherein, is the heat map predicted value, is the heat map actual value, and is a preset parameter.

6. A seat belt detection device characterized by comprising: The method comprises the following steps: a target human body image obtaining module is configured to obtain a to-be-detected image of a current vehicle, detect a human body region of the to-be-detected image, and obtain a target human body image; The key point heat map acquisition module is configured to obtain a key point heat map corresponding to at least one key point in the target human body image based on a pre-trained seat belt key point heat detection model according to the target human body image. The seat belt detection module is configured to determine whether the key point heat map satisfies a preset seat belt detection rule, and if yes, determine that the occupant in the current vehicle is a seat belt wearing person, which includes: determining whether there is a candidate heat value that satisfies a preset heat value size requirement in at least one key point heat map; if yes, determining whether the number of the candidate heat values is equal to or greater than a preset number threshold; if yes, determining that the occupant in the current vehicle is a seat belt wearing person; The key point heat detection model training module is configured to obtain a to-be-trained seat belt image as a sample training set of the seat belt key point heat detection model, and input the sample training set into a pre-constructed seat belt key point heat detection model; and further configured to obtain a trained seat belt key point heat detection model based on preset model training parameters according to the sample training set and the pre-constructed seat belt key point heat detection model. The processor executes the program to implement the seat belt detection method of any one of claims 1-5.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The program is executed by the processor to implement the seat belt detection method of any one of claims 1-5.

8. A computer readable storage medium having stored thereon a computer program, characterized in that, ​

Images