Safety belt wearing monitoring method, electronic device and vehicle

Abstract

The application provides a safety belt wearing monitoring method, an electronic device and a vehicle. When the safety belt and the driver's foreign object color have high similarity, multi-modal verification data is used for cross verification, the monitoring problem caused by the same color of the safety belt and the clothes is fundamentally solved, the accuracy of the safety belt wearing monitoring is greatly improved, and false judgments are effectively eliminated. The multi-modal verification data is used for cross verification, the heterogeneous multi-modal verification data is deeply fused and mutually backed up, even if the single mode recognition fails or is disturbed, other modes can still provide effective information, the robustness of the wearing monitoring task as a whole is ensured, and the stability of the safety belt wearing monitoring in all scenes and all weather conditions is ensured. After the safety belt wearing state is determined, a graded early warning prompt is given, the harsh alarm is changed into natural guidance under different monitoring results, and the user experience and safety are significantly improved.

Description

Technical Field

[0001] This application relates to the field of seat belt wearing monitoring technology, and in particular to a seat belt wearing monitoring method, electronic equipment, and vehicle. Background Technology

[0002] During vehicle operation, seat belts are the primary passive safety device protecting occupants' lives. Mainstream seat belt monitoring technologies primarily rely on physical sensors (such as buckle switches and pressure sensors) or camera-based visual recognition systems. However, when the seat belt color is highly similar to or completely identical to the occupant's clothing color, purely visual recognition systems are prone to misjudgment, leading to incorrect prompts and severely undermining the reliability of the monitoring system and user trust. Summary of the Invention

[0003] In view of this, the purpose of this application is to propose a seat belt wearing monitoring method, electronic device and vehicle, which improves the accuracy of seat belt wearing recognition in similar color scenarios by multimodal fusion seat belt wearing monitoring.

[0004] To achieve the above objectives, this application provides a method for detecting seatbelt wearing, comprising: Determine the color similarity between the theoretical seat belt area and the clothing area based on monitoring images of the seat belt wearing area; In response to the color similarity being greater than or equal to a preset first similarity threshold, multimodal verification data is acquired, and multiple single-modal judgment results are determined based on the monitored image and the multimodal verification data; The seatbelt wearing status is determined based on multiple judgment results and the color similarity, and graded warning prompts are issued based on the seatbelt wearing status.

[0005] Optionally, determining the color similarity between the theoretical seatbelt area and the clothing area based on the monitoring image of the seatbelt wearing area includes: In the monitoring image, a first identification color is used to identify the theoretical seat belt area and a second identification color is used to identify the clothing background area; The similarity between the first identified color and the second identified color is calculated to obtain the color similarity.

[0006] Optionally, the multimodal verification data includes thermal imaging data of the seatbelt wearing area and pressure distribution data of the driver's seat; the determination result includes structural modal wearing status, thermal imaging modal wearing status, and pressure modal wearing status; the determination of multiple single modalities based on the monitoring images and the multimodal verification data includes: Visible light features are extracted from the theoretical seat belt area in the monitoring image to obtain edge continuity features and texture structure features, and the wearing state of the structural mode is determined based on the edge continuity features and texture structure features; Based on the thermal imaging data, a thermal distribution image and a temperature gradient map of the seatbelt wearing area are determined, and the wearing status of the thermal imaging mode is determined based on the thermal distribution image and the temperature gradient map. The pressure distribution center and pressure distribution uniformity are determined based on the pressure distribution data, and the pressure mode wearing state is determined based on the pressure distribution center and the pressure distribution uniformity.

[0007] Optionally, determining the structural modality wearing state based on the edge continuity feature and the texture structure feature includes: In response to the edge continuity feature being edge continuous and the texture structure feature being textured with differences, wearing a seatbelt is determined as the structural mode wearing state; In response to the edge continuity feature being an edge break, or the texture structure feature being the absence of texture differences, the absence of a seatbelt is determined as the structural modality wearing state.

[0008] Optionally, determining the wearing status of the thermal imaging modality based on the thermal distribution image and the temperature gradient map includes: In response to the presence of peak and valley features in the temperature gradient map, the target image corresponding to the seat belt is matched in the thermal distribution image based on the image similarity to obtain the image matching result; In response to the graphic matching result indicating that the target graphic exists in the thermal distribution image, wearing a seatbelt is determined as the wearing state of the thermal imaging modality; In response to the graphic matching result indicating that the target graphic does not exist in the thermal distribution image, the absence of a seatbelt is determined as the wearing state in the thermal imaging modality.

[0009] Optionally, determining the pressure mode wearing state based on the pressure distribution center and the pressure distribution uniformity includes: Determine the standard pressure center range and standard uniformity range corresponding to the standard sitting posture; In response to the existence of the pressure distribution center within the standard pressure center range and the existence of the pressure distribution uniformity within the standard uniformity range, wearing a seat belt is determined as the pressure mode wearing state; In response to the absence of the pressure distribution center within the standard pressure center range, or the absence of the pressure distribution uniformity within the standard uniformity range, the absence of a seatbelt is determined as the pressure mode wearing state.

[0010] Optionally, determining the seatbelt wearing status based on multiple determination results and the color similarity includes: In response to the color similarity being less than or equal to a preset second similarity threshold, the determination result is the number of results for wearing the full band and the total number of modalities; wherein, the second similarity threshold is greater than the first similarity threshold; Determine the ratio of the number of results to the total number of modes; In response to the ratio being greater than a preset ratio threshold, the wearing of the seat belt is determined as the seat belt wearing state; In response to the ratio being less than or equal to a preset ratio threshold, the state of not wearing a seatbelt is determined as the seatbelt wearing state.

[0011] Optionally, determining the seatbelt wearing status based on multiple determination results and the color similarity includes: In response to the color similarity being greater than a preset second similarity threshold and the ratio being greater than a preset ratio threshold, the determination result is determined to be the target determination result of wearing the full strap; Determine the confidence level of each target determination result to obtain a confidence level set; In response to the maximum confidence level in the confidence set being greater than or equal to a preset confidence threshold, the wearing of the seat belt is determined as the seat belt wearing state; In response to the maximum confidence level in the confidence set being less than a preset confidence threshold, an abnormal wearing condition is determined as the seatbelt wearing status.

[0012] Based on the same inventive concept, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method described above when executing the computer program.

[0013] Based on the same inventive concept, this application also provides a vehicle including the electronic equipment described above.

[0014] As can be seen from the above, the seat belt wearing monitoring method, electronic device, and vehicle provided in this application can determine the color similarity between the theoretical seat belt area and the clothing area based on the monitoring image of the seat belt wearing area; when the color similarity is greater than or equal to a preset first similarity threshold, multimodal verification data is acquired, and multiple single-modal judgment results are determined based on the monitoring image and multimodal verification data; the seat belt wearing status is determined based on multiple judgment results and color similarity, and graded warning prompts are given according to the seat belt wearing status. When the color similarity between the seat belt and the driver's foreign object is high, multimodal verification data is used for cross-verification, fundamentally overcoming the monitoring problem caused by the seat belt and clothing being the same color, significantly improving the accuracy of seat belt wearing monitoring, and effectively eliminating false judgments. By using multimodal verification data for cross-verification, heterogeneous multimodal verification data are deeply integrated and mutually backed up. Even if the recognition of a single modality fails or is interfered with, other modalities can still provide effective information, ensuring the overall robustness of the wearing monitoring task and ensuring the stability of seat belt wearing monitoring in all scenarios and all weather conditions. After determining the seatbelt wearing status, tiered warning prompts are provided, transforming rigid alarms into natural guidance based on different monitoring results, significantly improving user experience and safety. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart of a seatbelt wearing monitoring method according to an embodiment of this application; Figure 2 This is a schematic diagram of a seatbelt wearing monitoring device according to an embodiment of this application; Figure 3 This is a schematic diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0018] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0019] In this article, it is important to understand that any number of elements in the accompanying figures is for illustrative purposes and not for limitation, and any naming is for distinction only and has no limiting meaning.

[0020] Based on the above background description, the following situations also exist in the related technologies: In related technologies, seatbelt monitoring technology primarily relies on physical sensors or camera-based visual recognition methods. Monitoring via physical sensors involves installing sensors inside the vehicle. The most common method is to install sensors (such as Hall effect sensors or microswitches) inside the seatbelt buckle, detecting whether the latch is inserted into the buckle to determine if the seatbelt is worn. This is the most basic and widely used method. Torque sensors can also be used to detect changes in tension as the seatbelt retracts. When the seatbelt is pulled out and worn, its rebound force exhibits a specific sequence of changes; comparing this sequence with a preset reference sequence can improve the accuracy of the detection.

[0021] Camera-based visual recognition methods primarily utilize images captured by cameras, which are then analyzed using image processing and pattern recognition techniques. First, high-definition cameras capture images of occupants inside the vehicle. These cameras have high resolution, enabling them to clearly capture details. The next crucial and complex step is accurately locating the area where seatbelts might be worn. Based on face or upper body detection, algorithms (such as the Viola-Jones algorithm and deformable part models) are further used within the located window area to detect faces or upper bodies, thereby defining the seatbelt detection area (ROI) based on relative position. Within the defined area, advanced image analysis algorithms are used to identify the seatbelts.

[0022] Detecting whether the latch is inserted into the buckle to determine whether the seat belt is worn can lead to false recognition due to some non-seat belt props being inserted. When using computer vision for recognition, pure vision recognition systems are prone to misjudgment when the seat belt color is highly similar to or exactly the same as the occupant's clothing color, resulting in incorrect prompts and seriously undermining the reliability of the monitoring system and user trust.

[0023] The seatbelt wearing monitoring method, electronic device, and vehicle provided in this application can determine the color similarity between the theoretical seatbelt area and the clothing area based on the monitoring image of the seatbelt wearing area. When the color similarity is greater than or equal to a preset first similarity threshold, multimodal verification data is acquired, and multiple single-modal judgment results are determined based on the monitoring image and multimodal verification data. The seatbelt wearing status is determined based on multiple judgment results and color similarity, and graded warning prompts are given according to the seatbelt wearing status. When the color similarity between the seatbelt and the driver's foreign object is high, multimodal verification data is used for cross-verification, fundamentally overcoming the monitoring problem caused by the seatbelt and clothing being the same color, significantly improving the accuracy of seatbelt wearing monitoring, and effectively eliminating false judgments. By using multimodal verification data for cross-verification, heterogeneous multimodal verification data are deeply integrated and mutually backed up. Even if the recognition of a single modality fails or is interfered with, other modalities can still provide effective information, ensuring the overall robustness of the wearing monitoring task and ensuring the stability of seatbelt wearing monitoring in all scenarios and all weather conditions. After determining the seatbelt wearing status, tiered warning prompts are provided, transforming rigid alarms into natural guidance based on different monitoring results, significantly improving user experience and safety.

[0024] The following describes in detail, with reference to the accompanying drawings, the seat belt wearing monitoring method provided by the embodiments of this application.

[0025] In some embodiments, such as Figure 1 As shown, a method for monitoring seat belt wearing includes steps 101-103.

[0026] Step 101: Determine the color similarity between the theoretical seat belt area and the clothing area based on the monitoring image of the seat belt wearing area.

[0027] In practice, after the vehicle is powered on, a functional self-test is performed first. Upon vehicle startup or detection of the driver being seated, the seatbelt wearing monitoring system performs a self-test to ensure that data acquisition modules such as the high-definition camera, infrared thermal imaging sensor, seat pressure sensor, and microphone array are functioning correctly. The multimodal fusion decision module is then initialized, loading the pre-trained deep learning model and decision logic parameters.

[0028] Then, multimodal data is collected synchronously. The multimodal data includes monitoring images of the entire seat belt wearing area and multimodal verification data. The multimodal verification data includes thermal imaging data of the seat belt wearing area and pressure distribution data of the driver's seat (more modal data can also be added for assistance, which is not limited here).

[0029] The monitoring images are visible light images, captured by high-definition cameras deployed on the A-pillars, rearview mirror, or center console, collecting RGB images covering the upper body of the occupant and the seatbelt area. Infrared thermal imaging data is also collected, with infrared sensors simultaneously acquiring thermal imaging data of the seatbelt wearing area to obtain temperature distribution information. Pressure distribution data acquisition utilizes a matrix of pressure sensors installed in the seat to collect data such as the occupant's pressure distribution and center of gravity position. An in-vehicle microphone array continuously collects in-vehicle voice commands and ambient sounds to promptly respond to user interaction requests.

[0030] Then, the multimodal data undergoes data preprocessing and standardization. The monitoring images are processed with denoising, grayscale conversion, and histogram equalization to enhance image quality and reduce the impact of illumination variations. Non-uniformity correction and temperature calibration are performed on the thermal imaging data to ensure accuracy. Pressure distribution data is filtered to eliminate noise interference such as vehicle vibration. Noise reduction and endpoint detection are performed on the speech signal to prepare for subsequent recognition.

[0031] Once an occupant is detected seated, the seatbelt wearing status is determined using multimodal data, with high-reliability identification specifically for the extreme scenario where the seatbelt and clothing are the same color. This requires determining the color similarity between the theoretical seatbelt area and the clothing area based on the monitored image. First, region localization is performed in the monitored image, using object detection networks (such as YOLO or SSD) to accurately identify the "occupant's upper body," "seatbelt shoulder strap area," "seatbelt lap belt area," and "main clothing area." Then, color feature extraction is performed. Within the localized area, histograms in the RGB and HSV color spaces are calculated, and the color similarity between the seatbelt area and the clothing area is calculated using the CIEDE2000 color difference formula.

[0032] In some embodiments, determining the color similarity between the theoretical seatbelt area and the clothing area based on a monitoring image of the seatbelt wearing area includes: In the monitored image, a first identification color is used to identify the theoretical seat belt area and a second identification color is used to identify the clothing background area; The similarity between the first and second identified colors is calculated to obtain the color similarity.

[0033] In practice, the first step is to identify the primary color of the theoretical seatbelt area and the secondary color of the clothing background area. The theoretical seatbelt area includes the shoulder strap area and the lap belt area. Both the primary and secondary colors are represented using RGB values. The similarity calculation process includes: color space conversion, where both the primary and secondary colors (usually RGB values) are converted to the CIELAB color space. The CIELAB color space is an approximately uniform color space, where L represents lightness, and A and B represent the opposing dimensions of red / green and yellow / blue. Then, auxiliary variables are calculated. Based on the L, A, and B values, a set of intermediate variables, such as L' (average lightness), C' (average chroma), and h' (average hue angle), are calculated to prepare for subsequent calculations. Next, the difference components are calculated, specifically the differences in lightness (ΔL'), chroma (ΔC'), and hue (ΔH') between the primary and secondary colors. Finally, weighting and correction are performed, introducing weighting coefficients (S_L, S_C, S_H) to correct for differences in human eye sensitivity to different lightness, chroma, and hue values. For example, the human eye is more sensitive to changes in the hue of the blue region. The formula corrects for this using S_H and the rotation function R_T. Substituting all the parameters into the CIEDE2000 formula, the calculated output is the color difference ΔE. Color similarity and color difference are negatively correlated; that is, the smaller the color difference ΔE, the larger the color similarity T, indicating that the first color and the second color are more similar.

[0034] For example, if color similarity is simply represented as the reciprocal of color difference, when ΔE∈(10.0,+∞), T∈(0,1 / 10), it means the color difference between the first and second colors is large, so the color similarity between the first and second colors is small, the first and second colors are significantly different, and the first and second colors are completely different. When ΔE∈(3.5,10.0], T∈[1 / 10,1 / 7), it means the color difference between the first and second colors is slightly large, so the color similarity between the first and second colors is slightly small, the first and second colors have a perceptible difference, and the first and second colors are clearly different. When ΔE∈(2.0,3.5], T∈[1 / 7,1 / 2), it means the color difference between the first and second colors is small, so the color similarity between the first and second colors is large, the first and second colors have a slight difference, and the first and second colors can be noticed when displayed side by side. When ΔE∈(1.0,2.0], T∈[1 / 2,1), it means the color difference between the first and second colors is very small, indicating a high degree of similarity between them. The difference is only noticeable upon close observation. When ΔE∈(0,1.0], T∈[1,+∞), it means the color difference between the first and second colors is almost negligible, indicating an extremely high degree of similarity. The difference is indistinguishable to the naked eye and with ordinary equipment, and can only be detected using professional color analysis equipment. Therefore, 1 / 2 can be used as the first similarity threshold.

[0035] Step 102: In response to a color similarity greater than or equal to a preset first similarity threshold, acquire multimodal verification data, and determine the judgment results of multiple single modalities based on the monitoring image and the multimodal verification data.

[0036] In practice, if the color similarity is greater than or equal to the preset first similarity threshold, it means that the color of the seat belt and the color of the occupant's clothing are highly similar or extremely similar, and cannot be distinguished by simple vision. In this case, it is necessary to further verify through multimodal verification data and monitoring images to determine the preliminary monitoring results and obtain the judgment result.

[0037] In some embodiments, the multimodal verification data includes thermal imaging data of the seatbelt wearing area and pressure distribution data of the driver's seat; the determination results include structural modal wearing status, thermal imaging modal wearing status, and pressure modal wearing status; and the determination results of multiple single modalities are determined based on the monitoring images and multimodal verification data, including: Visible light features are extracted from the theoretical seat belt area in the monitoring image to obtain edge continuity features and texture structure features, and the structural mode wearing status is determined based on the edge continuity features and texture structure features; Based on thermal imaging data, determine the thermal distribution image and temperature gradient map of the seat belt wearing area, and determine the wearing status of the thermal imaging mode based on the thermal distribution image and temperature gradient map; The pressure distribution center and pressure distribution uniformity are determined based on the pressure distribution data, and the pressure mode wearing status is determined based on the pressure distribution center and pressure distribution uniformity.

[0038] In practice, for monitoring images, edge continuity and structural verification are performed using visible light feature extraction. This analyzes whether the seatbelt edges are continuous and complete, and whether the geometric structure conforms to wearing characteristics (such as the diagonal shape of the shoulder strap and the horizontal shape of the lap belt). If the edges are broken, blurred, or blended with clothing edges, the judgment is more likely that the seatbelt is not being worn. Thermal imaging data is used to analyze the heat distribution in the seatbelt wearing area. Because fabric seatbelts and ordinary clothing have different materials and heat capacities, their thermal radiation characteristics usually differ. If a continuous band-like area with a significantly different temperature from the surrounding clothing is detected, it strongly supports the judgment that the seatbelt is being worn; if the heat distribution is uniform and there are no band-like features, it supports the judgment that the seatbelt is not being worn. Pressure distribution data is used for auxiliary verification of pressure and posture. Based on the pressure distribution data, the occupant's body posture is determined to determine whether the occupant is in a standard sitting posture that can be effectively restrained by the seatbelt. If the center of pressure is severely deviated (such as excessive forward leaning) or the posture is abnormal, even if the seatbelt is detected, there is a possibility of misjudgment, and the judgment of "not wearing a seatbelt" is more likely, even if the occupant meets the requirements of a standard sitting posture. The system tends to favor the "wearing" of a seatbelt. Each modality's data yields a judgment result; combining multiple judgment results with color similarity allows for a more accurate identification. The specific judgment process for each modality is illustrated in the following example.

[0039] In some embodiments, determining the structural modality wearing state based on edge continuity features and texture structure features includes: In response to the edge continuity feature being edge continuity and the texture structure feature being texture difference, wearing a seat belt is determined as the structural modality wearing state; In response to the edge continuity feature being an edge break or the texture structure feature being the absence of texture differences, the absence of a seatbelt is determined as the structural modal wearing state.

[0040] In practice, for determining the edge structure mode, since the seat belt and the occupant's clothing are highly similar in color, the analysis needs to be based on the texture and structural features of the seat belt as input. If the edge continuity feature is continuous, it indicates whether the edge of the seat belt is continuous and complete. If the texture structure feature shows texture differences, it indicates that there are certain differences in the geometric structure of the seat belt and the clothing, and no edge merging occurs. If both conditions are met, the seat belt is determined to have continuous and complete edges, a clear geometric structure, and the edge does not merge with the edge of the clothing, conforming to wearing characteristics (such as the diagonal shape of the shoulder strap and the horizontal shape of the lap belt), and the wearing of the seat belt is determined as the structural mode of wearing. If there are edge breaks, blurring, or merging with the edge of the clothing, the judgment tends to be that the seat belt is not being worn. In this case, if either the edge continuity feature is broken or the texture structure feature shows no texture differences, the not wearing of the seat belt can be determined as the structural mode of wearing.

[0041] In some embodiments, determining the thermal imaging modality wearing status based on a thermal distribution image and a temperature gradient map includes: In response to the presence of peak and valley features in the temperature gradient map, the target image corresponding to the seat belt is matched in the thermal distribution image based on the image similarity to obtain the image matching result; In response to the image matching result indicating the presence of a target image in the thermal distribution image, the wearing of the seat belt is determined as the wearing state in the thermal imaging modality; In response to the image matching result indicating that the target image does not exist in the thermal distribution image, the absence of a seatbelt is identified as the wearing status in the thermal imaging modality.

[0042] In practice, the process of determining the infrared thermal imaging mode first involves thermal region registration, where the thermal distribution image and the monitoring image are registered pixel-level to ensure that the same physical area is being analyzed. Then, absolute temperature analysis is performed to read the absolute temperature values ​​of the seatbelt area and the clothing area. Since seatbelts are typically not in direct contact with the skin and are made of different materials, their temperature may differ from that of clothing.

[0043] Next, relative temperature gradient analysis is performed to calculate the temperature gradient perpendicular to the seatbelt direction. A taut seatbelt will form a "thermal groove" in contact with the body or a "low-temperature band" due to its fabric properties, thus showing obvious peak and valley characteristics on the gradient map. "Fake seatbelt" patterns printed on clothing will not exhibit this gradient characteristic. Finally, thermal distribution pattern recognition is performed, using a small neural network to determine whether the thermal distribution image contains a continuous, narrow, band-shaped target pattern, which corresponds to the physical shape of a seatbelt.

[0044] If peak-valley characteristics are present and a target image is visible in the thermal distribution image, it indicates that the thermal imaging data clearly confirms that a seatbelt is being worn. Therefore, seatbelt wearing is identified as the seatbelt wearing status in the thermal imaging modality with a high confidence level. If peak-valley characteristics are absent or a target image is not visible in the thermal distribution image, it indicates that the thermal imaging characteristics of seatbelt wearing are not obvious. Therefore, seatbelt not wearing is identified as the seatbelt wearing status in the thermal imaging modality.

[0045] In some embodiments, determining the pressure mode wearing state based on the pressure distribution center and pressure distribution uniformity includes: Determine the standard pressure center range and standard uniformity range corresponding to the standard sitting posture; In response to the existence of a pressure distribution center within the standard pressure center range and a pressure distribution uniformity within the standard uniformity range, wearing a seat belt is defined as the pressure mode wearing state. In response to the absence of a pressure distribution center within the standard pressure center range, or the absence of pressure distribution uniformity within the standard uniformity range, the absence of a seatbelt is determined as a pressure mode wearing state.

[0046] In practice, pressure distribution-based seating posture classification requires the use of a classifier (such as SVM or lightweight CNN) to categorize occupant postures into "standard sitting posture," "forward leaning," "side leaning," and "reclining," etc. The restraint state and visible shape of the seat belt differ under different postures.

[0047] Optionally, it can also complement visual pose estimation. If a camera-based skeleton recognition algorithm (such as OpenPose) is configured, its output joint information can be fused with pressure distribution information to more accurately determine body angles, such as whether the torso is at a reasonable angle to the seat back, which is a prerequisite for effective seat belt restraint.

[0048] When classifying sitting postures, it is only necessary to determine whether the user's sitting posture is relatively standard. Therefore, the standard pressure center range and standard uniformity range corresponding to the standard sitting posture are first determined. If a pressure distribution center exists within the standard pressure center range and a pressure distribution uniformity exists within the standard uniformity range, it indicates that the occupant is in a standard sitting posture that can be effectively restrained by the seat belt, and wearing the seat belt is determined as the pressure modal wearing state. If there is no pressure distribution center within the standard pressure center range, or no pressure distribution uniformity within the standard uniformity range, it indicates that the occupant is in a non-standard sitting posture that cannot be effectively restrained by the seat belt, and not wearing a seat belt is determined as the pressure modal wearing state.

[0049] Optionally, if the color similarity is less than a preset first similarity threshold, it indicates that the computer vision-based judgment is relatively accurate, and the determination of whether a seatbelt is worn can be made directly based on the recognition result. Optionally, to ensure accuracy, further verification can be performed based on multimodal verification data if sufficient computing resources are available.

[0050] Step 103: Determine the seat belt wearing status based on multiple judgment results and color similarity, and provide graded warning prompts based on the seat belt wearing status.

[0051] In practice, if the color similarity is less than or equal to the preset second similarity threshold (e.g., 1), it indicates that there is a certain possibility of wearing a seat belt during computer vision recognition, but the probability is small. Therefore, the influence of color similarity is not considered when performing multimodal judgment. It is only necessary to determine the seat belt wearing status based on the judgment results of structural mode, thermal imaging mode and pressure mode.

[0052] If the structural mode determination result is "wearing a seat belt", the thermal imaging mode determination result is "wearing a seat belt", and the pressure mode determination result is "wearing a seat belt", it means that the determination result of all three modes is "wearing a seat belt", the result is reliable, and the seat belt wearing status is "wearing a seat belt".

[0053] If the structural mode determines that the seatbelt is not being worn, the thermal imaging mode determines that the seatbelt is being worn, and the pressure mode determines that the seatbelt is being worn, this means that two out of the three modes determine that the seatbelt is being worn, and one mode determines that the seatbelt is not being worn. Furthermore, the judgment based on computer vision is unreliable because the seatbelt and the foreign object are similar in color. The majority of the seatbelt determinations are based on the majority judgment, and the seatbelt wearing status is determined to be wearing.

[0054] If the structural mode determines that the seatbelt is being worn, the thermal imaging mode determines that the seatbelt is not being worn, and the pressure mode determines that the seatbelt is being worn, this means that two out of the three modes determine that the seatbelt is being worn, and one mode determines that the seatbelt is not being worn. Furthermore, the judgment based on computer vision is unreliable because the seatbelt and the foreign object are similar in color. The majority of the seatbelt determinations are based on the majority judgment, and the seatbelt wearing status is determined to be wearing.

[0055] If the structural mode determines that the seatbelt is being worn, the thermal imaging mode determines that the seatbelt is being worn, and the pressure mode determines that the seatbelt is not being worn, this means that two out of the three modes determine that the seatbelt is being worn, and one mode determines that the seatbelt is not being worn. Furthermore, the computer vision-based judgment is unreliable because the seatbelt and the foreign object are similar in color. The Petty seatbelt judgment has the majority, so the majority judgment is taken as the standard, and the seatbelt wearing status is determined to be wearing a seatbelt.

[0056] If the structural mode assessment result is "not wearing a seatbelt", the thermal imaging mode assessment result is "not wearing a seatbelt", and the pressure mode assessment result is "wearing a seatbelt", it means that one of the three modes' assessment results is "wearing a seatbelt", and two modes' assessment results are "not wearing a seatbelt". Furthermore, the judgment based on computer vision is unreliable because the seatbelt and the foreign object are similar in color. The majority of the assessment results are "not wearing a seatbelt". Therefore, the majority assessment result is taken as the standard, and the seatbelt wearing status is determined to be "not wearing a seatbelt".

[0057] If the structural mode determines that a seatbelt is not being worn, the thermal imaging mode determines that a seatbelt is being worn, and the pressure mode determines that a seatbelt is not being worn, this means that one of the three modes determines that a seatbelt is being worn, and two modes determine that a seatbelt is not being worn. Furthermore, the judgment based on computer vision is unreliable because the seatbelt and the foreign object are similar in color. The majority of the judgments indicates that a seatbelt is not being worn. Therefore, the majority judgment is used to determine that the seatbelt is not being worn.

[0058] If the structural mode determines that a seatbelt is being worn, the thermal imaging mode determines that a seatbelt is not being worn, and the pressure mode determines that a seatbelt is not being worn, this indicates that one of the three modes determines that a seatbelt is being worn, and two modes determine that a seatbelt is not being worn. Furthermore, the computer vision-based judgment is unreliable because the seatbelt and the foreign object are similar in color. Therefore, the majority of the judgments indicates that a seatbelt is not being worn. The majority judgment is used to determine the seatbelt wearing status. The process for determining seatbelt wearing status when the color similarity is less than or equal to a preset second similarity threshold is shown in the following embodiment.

[0059] In some embodiments, determining the seatbelt wearing status based on multiple determination results and color similarity includes: In response to a color similarity less than or equal to a preset second similarity threshold, the determination result is the number of results for wearing the full band and the total number of modalities; wherein, the second similarity threshold is greater than the first similarity threshold; Determine the ratio of the number of results to the total number of modes; When the ratio is greater than a preset ratio threshold, the wearing of the seat belt is determined as the seat belt wearing status; In response to a ratio less than or equal to a preset ratio threshold, the absence of a seatbelt is determined as a seatbelt-wearing status.

[0060] In practical implementation, computer vision-based judgments are unreliable due to the similarity in color between seat belts and foreign objects. Therefore, it is only necessary to determine the seat belt wearing status based on the judgment results of structural, thermal imaging, and pressure modes, resulting in a total of 3 modes. Further, the number of judgment results indicating seat belt wearing is determined, and the ratio of this number to the total number of modes is used to determine the seat belt wearing status. When the total number of modes is odd, it is always possible to distinguish between the number of judgment results indicating seat belt wearing and those indicating not wearing, and the majority of these results is taken as the final seat belt configuration status. When the total number of modes is even, there may be cases where the number of judgment results indicating seat belt wearing and those indicating not wearing are equal. In this case, since the visual judgment results indicate similar colors, there is a greater tendency to determine that the seat belt is not worn, and this is determined as the seat belt wearing status. Therefore, 0.5 can be used as a ratio threshold to determine the seat belt wearing status. When the ratio is greater than the preset ratio threshold, the majority of the judgments are that the seat belt is worn, and the seat belt wearing status is determined. When the ratio is less than or equal to the preset ratio threshold, the majority of the judgments are that the seat belt is not worn, or the number of the two judgments is the same, and the seat belt not being worn is determined.

[0061] Optionally, if a scenario where the number of both judgment results is the same is defined as wearing a seatbelt, 0.5 can also be selected as the ratio threshold to determine the seatbelt wearing status. In this case, when the ratio is greater than or equal to the preset ratio threshold, the judgment result of wearing a seatbelt is the majority, or the number of both judgment results is the same, and the seatbelt wearing status is determined; when the ratio is less than the preset ratio threshold, the judgment result of not wearing a seatbelt is the majority, and the seatbelt not wearing status is determined.

[0062] For cases where the color similarity exceeds a preset second similarity threshold, an excessively high color similarity is equivalent to determining that the seatbelt is not being worn. However, to avoid coincidences where the occupant is wearing clothing that is extremely similar in color to the seatbelt, multimodal verification is still required. The process of determining the seatbelt wearing status in this scenario is shown in the following embodiment.

[0063] In some embodiments, determining the seatbelt wearing status based on multiple determination results and color similarity includes: In response to a color similarity greater than a preset second similarity threshold and a ratio greater than a preset ratio threshold, the determination result is determined as the target determination result of wearing the full strap; Determine the confidence level of each target's judgment result to obtain a confidence level set; When the maximum confidence level in the confidence set is greater than or equal to a preset confidence threshold, the wearing of a seat belt is determined as the seat belt wearing status. If the maximum confidence level in the confidence set is less than the preset confidence threshold, the wearing abnormality is determined to be a seat belt wearing status.

[0064] In practice, since cases where clothing and seatbelt colors are so similar that specialized equipment is required to distinguish them are almost nonexistent (except for rare, accidental occurrences), the computer vision-based determination of "not wearing a seatbelt" is more likely. In this situation, the accuracy of determinations with ratios exceeding a preset threshold in multimodal validation is affected; that is, the seatbelt wearing status may be inaccurate (but this has no impact on cases where the seatbelt is not worn, and actually increases credibility). This is because including the computer vision determination might yield the opposite seatbelt wearing status. Therefore, it is necessary to further determine the credibility of each determination. Confidence levels are used to measure the credibility of each target determination. By determining the confidence level of each target determination, a confidence set is obtained, which includes the confidence level for each determination of "wearing a seatbelt."

[0065] If the maximum confidence level in the confidence level set is greater than or equal to the preset confidence level threshold, it means that there is a highly reliable judgment result that can support the final judgment of wearing a seat belt. In this scenario, the confidence level of the judgment result of the computer vision dimension is lower, and the overall judgment tends to be more confident, thus determining that wearing a seat belt is the seat belt wearing status.

[0066] If the maximum confidence level in the confidence set is less than the preset confidence threshold, it means there is no highly reliable judgment result to support the final determination of seat belt wearing. For safety reasons, a more conservative control strategy is adopted, and abnormal wearing is determined as a seat belt wearing status. Because user safety is more important than user experience, safety is given priority, and a seat belt wearing prompt will be triggered when abnormal wearing occurs to ensure safety.

[0067] After determining the seatbelt patency status, tiered warning prompts can be issued based on the seatbelt patency status through a heads-up display and voice interaction. The heads-up display tiered warning prompt mechanism is as follows: Level 1 Reminder (Preventative): After the vehicle is started or passengers are seated, the head-up display shows the icon and text "Please fasten your seatbelt," accompanied by a gentle reminder sound.

[0068] Level 2 alert (early warning): When the system determines that the seat belt is not properly worn, the head-up display will show "Seat belt status detection in progress, please check if it is fastened" or "Seat belt is the same color as clothing, it is recommended to confirm that it is being worn", and a yellow warning icon will flash.

[0069] Level 3 prompt (mandatory): When the system determines that the seat belt is not worn, the head-up display shows a red "seat belt not fastened" icon, text prompt and countdown (such as "please fasten your seat belt in 3 seconds"), accompanied by a continuous beep or vibration reminder.

[0070] If the system determines that the seat belt is being worn, no warning or prompt is required.

[0071] Voice interaction mechanism: Voice command recognition: The system listens in real time and recognizes occupant voice commands, such as "fasten your seat belt", "how to fasten your seat belt", "the seat belt is not fastened", etc.

[0072] Voice feedback and guidance: If the seat belt is not worn, the system will immediately issue a voice reminder through the car audio: "Your seat belt is not fastened. Please fasten it immediately to ensure safety." If the system recognizes "How to fasten the seat belt", it can provide voice guidance: "Please cross the shoulder strap diagonally from your shoulder to your hip, keep the lap belt close to your hip bones, and fasten the buckle." Voice confirmation and closed loop: After the passenger fastens their seat belt, if they say "I have fastened my seat belt", the system will restart the multimodal verification. If the verification is successful, the head-up display and voice will simultaneously prompt "Your seat belt is fastened. Have a pleasant journey!", completing the closed loop and ensuring driving safety.

[0073] For example, consider a scenario where the passenger is wearing a black hoodie and a black seatbelt.

[0074] The computer vision recognition module detected that the seat belt and clothing colors were highly similar, triggering a high-risk process for color matching.

[0075] The edge detection module found that the edge of the seat belt was almost invisible in the image, indicating poor continuity.

[0076] The infrared thermal imaging module detected a continuous strip of heat zone (slightly lower temperature, consistent with the heat dissipation characteristics of webbing) in the area where the seat belt was worn, which had a clear thermal gradient with the background of the clothing (uniform temperature).

[0077] The pressure sensor indicates that the occupant's posture is a standard sitting position and the pressure distribution is normal.

[0078] Based on the combined assessment: the edge structure verification result was "not wearing a seatbelt," while the infrared thermal imaging and attitude pressure verification results were both "wearing a seatbelt," with significant infrared features and high confidence. Therefore, the final determination was that the seatbelt was "wearing a seatbelt," the head-up display showed a green "seatbelt fastened" icon, and there was no voice prompt.

[0079] The corresponding voice interaction example is: The passenger said, "The seatbelt wasn't fastened properly." The voice recognition module identifies and analyzes the information, triggering a voice reminder: "We have detected that you are not wearing your seatbelt. Please fasten it immediately to ensure your safety." After fastening their seatbelts, the passenger replied, "I've fastened my seatbelt." The system performs multimodal verification again. After confirming that the seat belt is "wearing seat belt", the head-up display shows "seat belt is fastened" and the voice prompts "seat belt is fastened, have a pleasant journey!", thus completing the closed loop.

[0080] In summary, the embodiments of this application have the following technical effects: Precisely addresses color-matching interference: Through a four-dimensional cross-validation logic of "color similarity → edge structure → infrared thermal imaging → posture pressure", the monitoring accuracy is significantly improved in scenarios where seat belts and clothing are the same color, avoiding misjudgments from single visual recognition.

[0081] Multimodal fusion with strong robustness: By comprehensively utilizing heterogeneous data from multiple sources such as visible light, infrared, pressure, and voice, even if one modality fails or is interfered with (such as insufficient light or clothing obstruction), other modalities can still provide effective information, ensuring the overall robustness of the system.

[0082] Proactive interaction for a better experience: Integrating head-up displays with voice interaction enables multi-channel, hierarchical, and interactive prompts that combine visual and auditory elements, improving user perception and response efficiency and avoiding "rigid prompts".

[0083] Safety closed-loop management: Supports voice command input and system feedback, forming a safety closed loop of "detection → reminder → user response → re-detection → confirmation" to ensure the effectiveness of seat belt wearing.

[0084] Highly adaptable and versatile: suitable for different vehicle models, different occupants (adults / children), and different environments (daytime / nighttime, sunny / rainy days), with broad practical value.

[0085] It should be noted that the method in this embodiment can be executed by a single device, such as a computer or server. The method can also be applied in a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method in this embodiment, and the multiple devices will interact with each other to complete the method described.

[0086] It should be noted that the above description describes some embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0087] Based on the same inventive concept, corresponding to any of the above embodiments, this application also provides a seat belt wearing monitoring device.

[0088] refer to Figure 2 The seatbelt wearing monitoring device includes: Computer vision module 10 is configured to: determine the color similarity between the theoretical seat belt area and the clothing area based on the monitored image of the seat belt wearing area; The multimodal verification module 20 is configured to: in response to a color similarity greater than or equal to a preset first similarity threshold, acquire multimodal verification data, and determine the judgment results of multiple single modalities based on the monitoring image and the multimodal verification data; The integrated decision module 30 is configured to: determine the seat belt wearing status based on multiple judgment results and color similarity, and provide graded warning prompts based on the seat belt wearing status.

[0089] Optionally, the computer vision module 10 is also configured as follows: In the monitored image, a first identification color is used to identify the theoretical seat belt area and a second identification color is used to identify the clothing background area; The similarity between the first and second identified colors is calculated to obtain the color similarity.

[0090] Optionally, the multimodal verification module 20 is also configured as follows: Visible light features are extracted from the theoretical seat belt area in the monitoring image to obtain edge continuity features and texture structure features, and the structural mode wearing status is determined based on the edge continuity features and texture structure features; Based on thermal imaging data, determine the thermal distribution image and temperature gradient map of the seat belt wearing area, and determine the wearing status of the thermal imaging mode based on the thermal distribution image and temperature gradient map; The pressure distribution center and pressure distribution uniformity are determined based on the pressure distribution data, and the pressure mode wearing status is determined based on the pressure distribution center and pressure distribution uniformity.

[0091] Optionally, the multimodal verification module 20 is also configured as follows: In response to the edge continuity feature being edge continuity and the texture structure feature being texture difference, wearing a seat belt is determined as the structural modality wearing state; In response to the edge continuity feature being an edge break or the texture structure feature being the absence of texture differences, the absence of a seatbelt is determined as the structural modal wearing state.

[0092] Optionally, the multimodal verification module 20 is also configured as follows: In response to the presence of peak and valley features in the temperature gradient map, the target image corresponding to the seat belt is matched in the thermal distribution image based on the image similarity to obtain the image matching result; In response to the image matching result indicating the presence of a target image in the thermal distribution image, the wearing of the seat belt is determined as the wearing state in the thermal imaging modality; In response to the image matching result indicating that the target image does not exist in the thermal distribution image, the absence of a seatbelt is identified as the wearing status in the thermal imaging modality.

[0093] Optionally, the multimodal verification module 20 is also configured as follows: Determine the standard pressure center range and standard uniformity range corresponding to the standard sitting posture; In response to the existence of a pressure distribution center within the standard pressure center range and a pressure distribution uniformity within the standard uniformity range, wearing a seat belt is defined as the pressure mode wearing state. In response to the absence of a pressure distribution center within the standard pressure center range, or the absence of pressure distribution uniformity within the standard uniformity range, the absence of a seatbelt is determined as a pressure mode wearing state.

[0094] Optionally, the integrated decision module 30 is also configured as follows: In response to a color similarity less than or equal to a preset second similarity threshold, the determination result is the number of results for wearing the full band and the total number of modalities; wherein, the second similarity threshold is greater than the first similarity threshold; Determine the ratio of the number of results to the total number of modes; When the ratio is greater than a preset ratio threshold, the wearing of the seat belt is determined as the seat belt wearing status; In response to a ratio less than or equal to a preset ratio threshold, the absence of a seatbelt is determined as a seatbelt-wearing status.

[0095] Optionally, the integrated decision module 30 is also configured as follows: In response to a color similarity greater than a preset second similarity threshold and a ratio greater than a preset ratio threshold, the determination result is determined as the target determination result of wearing the full strap; Determine the confidence level of each target's judgment result to obtain a confidence level set; When the maximum confidence level in the confidence set is greater than or equal to a preset confidence threshold, the wearing of a seat belt is determined as the seat belt wearing status. If the maximum confidence level in the confidence set is less than the preset confidence threshold, the wearing abnormality is determined to be a seat belt wearing status.

[0096] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.

[0097] The apparatus described above is used to implement the corresponding seat belt wearing monitoring method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0098] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the seat belt wearing monitoring method described in any of the above embodiments.

[0099] Figure 3 This embodiment illustrates a more specific hardware structure of an electronic device. The device may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0100] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0101] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0102] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0103] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0104] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0105] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0106] The electronic devices described above are used to implement the corresponding seat belt wearing monitoring method in any of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0107] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the seat belt wearing monitoring method as described in any of the above embodiments.

[0108] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0109] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the seat belt wearing monitoring method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0110] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides a vehicle, including the electronic device or seat belt wearing monitoring device of the above embodiments, and executes the seat belt wearing monitoring method as described in any of the above embodiments through the electronic device or seat belt wearing monitoring device of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0111] It is understood that before using the technical solutions of the various embodiments in this application, users will be informed of the type, scope of use, and usage scenarios of the personal information involved in an appropriate manner, and user authorization will be obtained.

[0112] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose, based on the prompt message, whether to provide personal information to the software or hardware such as electronic devices, applications, servers, or storage media performing the operations described in this application.

[0113] As an optional but not limited implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0114] It is understood that the above notification and user authorization process are merely illustrative and do not limit the implementation of this application. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this application.

[0115] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application is limited to these examples; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in detail for the sake of brevity.

[0116] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0117] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0118] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the claims of this application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.

Claims

1. A seat belt wearing monitoring method characterized by, include: Determine the color similarity between the theoretical seat belt area and the clothing area based on monitoring images of the seat belt wearing area; In response to the color similarity being greater than or equal to a preset first similarity threshold, multimodal verification data is acquired, and multiple single-modal judgment results are determined based on the monitored image and the multimodal verification data; The multimodal verification data includes thermal imaging data of the seatbelt wearing area and pressure distribution data of the driver's seat; the determination results include structural modal wearing status, thermal imaging modal wearing status, and pressure modal wearing status; the determination of multiple single modalities based on the monitoring image and the multimodal verification data includes: extracting visible light features from the theoretical seatbelt area in the monitoring image to obtain edge continuity features and texture structure features, and determining the structural modal wearing status based on the edge continuity features and texture structure features; determining the thermal distribution image and temperature gradient map of the seatbelt wearing area based on the thermal imaging data, and determining the thermal imaging modal wearing status based on the thermal distribution image and temperature gradient map; determining the pressure distribution center and pressure distribution uniformity based on the pressure distribution data, and determining the pressure modal wearing status based on the pressure distribution center and pressure distribution uniformity. The seatbelt wearing status is determined based on multiple judgment results and the color similarity, and graded warning prompts are issued based on the seatbelt wearing status. The step of determining the seat belt wearing status based on multiple determination results and the color similarity includes: when the color similarity is less than or equal to a preset second similarity threshold, determining the status corresponding to the determination result that accounts for the majority of the multiple determination results as the seat belt wearing status; the second similarity threshold is greater than the first similarity threshold.

2. The seatbelt wearing monitoring method according to claim 1, characterized in that, The step of determining the color similarity between the theoretical seatbelt area and the clothing area based on the monitoring image of the seatbelt wearing area includes: In the monitoring image, a first identification color is used to identify the theoretical seat belt area and a second identification color is used to identify the clothing background area; The similarity between the first identified color and the second identified color is calculated to obtain the color similarity.

3. The seatbelt wearing monitoring method according to claim 2, characterized in that, Determining the structural modality wearing state based on the edge continuity features and the texture structure features includes: In response to the edge continuity feature being edge continuous and the texture structure feature being textured with differences, wearing a seatbelt is determined as the structural mode wearing state; In response to the edge continuity feature being an edge break, or the texture structure feature being the absence of texture differences, the absence of a seatbelt is determined as the structural modality wearing state.

4. The seatbelt wearing monitoring method according to claim 2, characterized in that, Determining the wearing status of the thermal imaging modality based on the thermal distribution image and the temperature gradient map includes: In response to the presence of peak and valley features in the temperature gradient map, the target image corresponding to the seat belt is matched in the thermal distribution image based on the image similarity to obtain the image matching result; In response to the graphic matching result indicating that the target graphic exists in the thermal distribution image, wearing a seatbelt is determined as the wearing state of the thermal imaging modality; In response to the graphic matching result indicating that the target graphic does not exist in the thermal distribution image, the absence of a seatbelt is determined as the wearing state in the thermal imaging modality.

5. The seatbelt wearing monitoring method according to claim 2, characterized in that, Determining the pressure mode wearing state based on the pressure distribution center and the pressure distribution uniformity includes: Determine the standard pressure center range and standard uniformity range corresponding to the standard sitting posture; In response to the existence of the pressure distribution center within the standard pressure center range and the existence of the pressure distribution uniformity within the standard uniformity range, wearing a seat belt is determined as the pressure mode wearing state; In response to the absence of the pressure distribution center within the standard pressure center range, or the absence of the pressure distribution uniformity within the standard uniformity range, the absence of a seatbelt is determined as the pressure mode wearing state.

6. The seatbelt wearing monitoring method according to claim 1, characterized in that, The process of determining the seatbelt wearing status based on multiple determination results and the color similarity includes: In response to the color similarity being less than or equal to a preset second similarity threshold, the determination result is the number of results for wearing the full band and the total number of modalities; wherein, the second similarity threshold is greater than the first similarity threshold; Determine the ratio of the number of results to the total number of modes; In response to the ratio being greater than a preset ratio threshold, the wearing of the seat belt is determined as the seat belt wearing state; In response to the ratio being less than or equal to a preset ratio threshold, the state of not wearing a seatbelt is determined as the seatbelt wearing state.

7. The seatbelt wearing monitoring method according to claim 6, characterized in that, The process of determining the seatbelt wearing status based on multiple determination results and the color similarity includes: In response to the color similarity being greater than a preset second similarity threshold and the ratio being greater than a preset ratio threshold, the determination result is determined to be the target determination result of wearing the full strap; Determine the confidence level of each target determination result to obtain a confidence level set; In response to the maximum confidence level in the confidence set being greater than or equal to a preset confidence threshold, the wearing of the seat belt is determined as the seat belt wearing state; In response to the maximum confidence level in the confidence set being less than a preset confidence threshold, an abnormal wearing condition is determined as the seatbelt wearing status.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 7.

9. A vehicle, characterized in that, Including the electronic device as described in claim 8.

Images