In-vehicle area abnormity identification method and device, electronic equipment and storage medium
By combining visual and radar sensors, a multimodal sensor system has been developed, which solves the problem of limited in-vehicle detection solutions and enables efficient identification and alarm of in-vehicle anomalies in different environments, thereby improving vehicle safety.
Patent Information
- Application Number
- CN202511212685.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-28
- Publication Date
- 2025-12-12
AI Technical Summary
Existing vehicle security detection solutions are limited in scope, resulting in poor recognition efficiency and effectiveness, and are unable to effectively identify abnormal situations inside the vehicle in different environments.
A multimodal sensor system combining visual and radar sensors is used to determine the visual and radar anomaly levels by acquiring images and radar information of the target area, and to calculate the comprehensive confidence level based on environmental status information for alarm processing.
It improves the accuracy and flexibility of in-vehicle anomaly detection, and can dynamically adjust the confidence of vision and radar under different environmental conditions to ensure that abnormal situations are promptly notified to users, thereby improving the overall vehicle safety.
Smart Images
Figure CN121106006A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of vehicle safety technology, and in particular to a method, device, electronic device, and storage medium for identifying anomalies in the interior of a vehicle. Background Technology
[0002] There are generally many ways to handle vehicle driving safety features. For example, seat belts can be controlled by judging whether there are children in the seat through seat pressure distribution. Another method is to identify the distance between the occupant and the danger zone to provide in-vehicle reminders and handle dangerous behaviors of children.
[0003] However, the detection schemes described above are relatively simple, resulting in poor recognition efficiency and effectiveness. Summary of the Invention
[0004] In order to solve the above-mentioned technical problems, or at least partially solve the above-mentioned technical problems, at least one embodiment of the present disclosure provides a method, apparatus, electronic device and storage medium for identifying anomalies in vehicle interior areas.
[0005] Firstly, this disclosure provides a method for identifying anomalies in a vehicle interior area, including:
[0006] The target area image is acquired based on a pre-set visual sensor inside the vehicle, and the target visual anomaly level is determined based on the target area image.
[0007] When the target visual anomaly level is greater than a preset visual anomaly threshold, radar information of the target area is acquired based on a preset radar sensor inside the vehicle, and the target radar anomaly level is determined based on the target area radar information.
[0008] When the target radar anomaly level is greater than a preset radar anomaly threshold, the target visual confidence level and the target radar confidence level are obtained; wherein, the current environmental state information is obtained based on a preset multimodal sensor in the vehicle, and the target visual confidence level and the target radar confidence level are determined based on the current environmental state information;
[0009] A comprehensive confidence level is calculated based on the target visual anomaly level, the target radar anomaly level, the target visual confidence level, and the target radar confidence level. An alarm is then triggered based on the comprehensive confidence level and a preset alarm threshold.
[0010] Secondly, this disclosure provides an in-vehicle area anomaly detection device, including:
[0011] The vision processing module is used to acquire images of the target area based on preset vision sensors inside the vehicle, and to determine the level of visual anomaly of the target area based on the images of the target area.
[0012] The radar processing module is used to acquire radar information of the target area based on a preset radar sensor in the vehicle when the visual anomaly level of the target is greater than a preset visual anomaly threshold, and to determine the radar anomaly level of the target based on the radar information of the target area.
[0013] The acquisition module is used to acquire the target visual confidence level and the target radar confidence level when the target radar anomaly level is greater than a preset radar anomaly threshold; wherein, the current environmental state information is acquired based on a preset multimodal sensor in the vehicle, and the target visual confidence level and the target radar confidence level are determined based on the current environmental state information;
[0014] The processing module is used to calculate a comprehensive confidence level based on the target visual anomaly level, the target radar anomaly level, the target visual confidence level, and the target radar confidence level, and to perform alarm processing based on the comprehensive confidence level and a preset alarm threshold.
[0015] Thirdly, this disclosure provides an electronic device, including: a processor and a memory;
[0016] The processor executes any of the in-vehicle area anomaly identification methods provided in this disclosure by calling the programs or instructions stored in the memory.
[0017] Fourthly, this disclosure provides a computer-readable storage medium storing a program or instructions that cause a computer to execute any of the in-vehicle area anomaly identification methods provided in the embodiments of this disclosure.
[0018] Fifthly, this disclosure provides a computer program product for executing any of the in-vehicle area anomaly recognition methods provided in the embodiments of this disclosure.
[0019] The technical solution provided in this disclosure has at least the following advantages compared with the prior art:
[0020] In this embodiment, an image of the target area is acquired based on a pre-set visual sensor inside the vehicle, and the visual anomaly level of the target area is determined based on the image. When the visual anomaly level exceeds a pre-set visual anomaly threshold, radar information of the target area is acquired based on a pre-set radar sensor inside the vehicle, and the radar anomaly level of the target area is determined based on the radar information. When the radar anomaly level exceeds a pre-set radar anomaly threshold, the target visual confidence score and the target radar confidence score are acquired. Furthermore, current environmental state information is acquired based on a pre-set multi-modal sensor inside the vehicle, and the target visual confidence score and the target radar confidence score are determined based on this information. A comprehensive confidence score is calculated based on the target visual anomaly level, the target radar anomaly level, the target visual confidence score, and the target radar confidence score, and an alarm is triggered based on the comprehensive confidence score and a pre-set alarm threshold. By employing the above technical solution, visual recognition combined with radar assistance is used to detect whether there are anomalies in target areas such as the steering wheel and dashboard inside the vehicle. The visual confidence score and radar confidence score can be dynamically adjusted under different environmental conditions for anomaly detection, and an alarm can be triggered to notify the user when an anomaly is detected in the target area, further improving overall vehicle safety. Attached Figure Description
[0021] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.
[0022] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.
[0023] Figure 1 A flowchart illustrating an in-vehicle area anomaly identification method provided as an exemplary embodiment of this disclosure;
[0024] Figure 2 A flowchart illustrating a method for identifying anomalies in a vehicle interior area, provided as another exemplary embodiment of this disclosure;
[0025] Figure 3 A flowchart illustrating a method for identifying anomalies in an in-vehicle area, provided as another exemplary embodiment of this disclosure;
[0026] Figure 4 This is a schematic diagram of the structure of an in-vehicle area anomaly recognition device provided in an embodiment of this disclosure. Detailed Implementation
[0027] To better understand the above-described objectives, features, and advantages of this disclosure, the present disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the described embodiments are only some, not all, of the embodiments of this disclosure. The specific embodiments described herein are merely for explaining this disclosure and not for limiting it. Unless otherwise specified, the embodiments of this disclosure and the features within them can be combined with each other. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure are within the scope of protection of this disclosure.
[0028] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.
[0029] In existing methods, the active control of the intelligent airbag device is achieved by identifying the relative motion of the external environment or obstacles in front of the vehicle, judging the possible intensity and angle of a collision based on a collision prediction model, and intelligently adjusting the deployment time and intensity of the airbags. Alternatively, the deployment time and intensity of the airbags can be intelligently adjusted by using sensing elements such as camera vision and seat pressure distribution sensors to classify the body shape of the occupants or distinguish whether there are child seats in the seats, based on the age, body shape, and presence of child seats. Alternatively, the relative distance of passengers to dangerous areas can be identified and compared with a preset safe distance. When the distance is too small, a vehicle alert is issued to achieve in-vehicle safety protection. At the same time, dangerous actions by children can be accurately identified and warned, reducing driving hazards.
[0030] In other words, there are many ways to handle vehicle driving safety functions. For example, identifying the external environment to control power and airbag deployment; identifying whether there is a child seat by means of seat pressure distribution to control the timing of seat belt and airbag deployment; and identifying the distance between the occupant and the danger zone to provide in-vehicle reminders and handle dangerous behaviors of children.
[0031] However, the aforementioned existing technologies all have limitations, such as a single recognition and detection scheme, the inability to differentiate processing according to different usage scenarios, which affects the recognition rate, or the lack of processing and reminder functions for narrow spaces, multiple overlapping targets, and small objects used in vehicles. In other words, the detection scheme is relatively simple, resulting in poor recognition efficiency and recognition effect.
[0032] To address the aforementioned issues, this disclosure provides a method for identifying anomalies in in-vehicle areas. The method involves acquiring images of target areas using pre-set visual sensors within the vehicle and determining the visual anomaly level based on these images. When the visual anomaly level exceeds a pre-set visual anomaly threshold, radar information of the target area is acquired using pre-set radar sensors within the vehicle, and the radar anomaly level is determined based on this information. When the radar anomaly level exceeds a pre-set radar anomaly threshold, target visual confidence and target radar confidence are acquired. Furthermore, the method utilizes pre-set multi-modal sensors within the vehicle to acquire current environmental state information and determines target visual confidence and target radar confidence based on this information. A comprehensive confidence score is calculated based on the target visual anomaly level, target radar anomaly level, target visual confidence, and target radar confidence score, and an alarm is triggered based on the comprehensive confidence score and a pre-set alarm threshold. This technical solution detects anomalies in target areas such as the steering wheel and dashboard within the vehicle using a combination of visual recognition and radar assistance. It dynamically adjusts the visual and radar confidence scores under different environmental conditions for anomaly detection and alerts the user when an anomaly is detected, further enhancing overall vehicle safety.
[0033] The following detailed explanation, in conjunction with the accompanying drawings, illustrates the specific implementation methods of the in-vehicle area anomaly identification method, device, electronic equipment, and storage medium disclosed herein.
[0034] Figure 1 This is a flowchart illustrating an in-vehicle area anomaly recognition method provided in an exemplary embodiment of the present disclosure. The method can be executed by an in-vehicle area anomaly recognition device provided in an embodiment of the present disclosure. The in-vehicle area anomaly recognition device can be implemented in software and / or hardware, and is generally integrated into an electronic device.
[0035] like Figure 1 As shown, the method for anomaly identification in the vehicle interior may include the following steps:
[0036] Step 101: Determine the target visual anomaly level by acquiring images of the target area based on the pre-set visual sensors inside the vehicle.
[0037] The preset visual sensors can be understood as visual sensors, such as cameras, set up in one or more locations inside the vehicle. In actual vehicle application scenarios, there are usually multiple preset visual sensors. Specifically, one or more target areas inside the vehicle are pre-determined. The target area refers to the dangerous area inside the vehicle to be detected, such as the area affected by the explosion of the steering wheel airbag or the area affected by the explosion of the dashboard airbag. The specific settings are selected according to the actual application scenario. After the target area is determined, visual sensors are set up in the corresponding locations to perform visual recognition of the target area. For example, cameras are set up to capture images of the target area. That is to say, there can be one or more target area images, but usually there are multiple target area images.
[0038] In this embodiment of the disclosure, target area images of each target area can be acquired by a vision sensor pre-installed in the vehicle. After acquiring the target area image, anomaly identification can be performed on the target area image to obtain the target visual anomaly level. In other words, by processing the target area image, it can be determined whether there is an anomaly in the corresponding target area.
[0039] In some embodiments, visual recognition is performed on the target region image to obtain one or more of the following as target pixel feature parameters: pixel size, pixel grayscale, pixel color, and pixel quantity. The target pixel feature parameters are then compared with preset pixel feature parameters in a preset visual database to obtain the target visual contrast difference. Finally, the target visual anomaly level is determined based on the target visual contrast difference and a preset visual difference threshold.
[0040] In another embodiment, a target detection model is pre-trained using image samples with target labels, thereby enabling visual recognition of target region images based on the preset target detection model, obtaining target recognition results, and comparing the target recognition results with preset results (such as decorations, ornaments, bags, or dangerous user behaviors) to determine the target visual anomaly level.
[0041] The above two methods are only for determining the visual anomaly level of a target based on the target area image. The embodiments of this disclosure do not impose specific restrictions on the implementation method of determining the visual anomaly level of a target based on the target area image.
[0042] Step 102: When the visual anomaly level of the target is greater than the preset visual anomaly threshold, obtain radar information of the target area based on the preset radar sensor in the vehicle, and determine the radar anomaly level of the target based on the radar information of the target area.
[0043] In this embodiment of the disclosure, for example, a higher visual anomaly level indicates a higher level of danger in the target area. A visual anomaly threshold is preset, such as level one or level two. After obtaining the visual anomaly level of the target, the visual anomaly level of the target is compared with the visual anomaly threshold. When the visual anomaly level of the target is greater than the preset visual anomaly threshold, it indicates that there is an anomaly in the target area. Further judgment is needed based on the preset radar sensor in the vehicle to ensure the accuracy of the anomaly identification result.
[0044] The preset radar sensors can be understood as radar sensors set in one or more locations inside the vehicle. In actual vehicle application scenarios, there are usually multiple preset radar sensors. Specifically, one or more target areas inside the vehicle are pre-determined. The target area refers to the dangerous area inside the vehicle to be detected, such as the area affected by the explosion of the steering wheel airbag or the area affected by the explosion of the dashboard airbag. The specific settings are selected according to the actual application scenario. After the target area is determined, radar sensors are set in the corresponding locations to identify the target area and obtain radar information of the target area. That is to say, there can be one or more radar information for the target area. Usually, there are multiple images of the target area, such as the detection distance to the signal source, identification material characteristics, signal source reflection attenuation rate, and other information.
[0045] In this embodiment of the disclosure, target area radar information of each target area can be obtained through radar sensors pre-installed in the vehicle. After obtaining the target area radar information, anomaly identification can be performed on the target area radar information to obtain the target radar anomaly level. In other words, by processing the target area radar information, it can be determined whether there is an anomaly in the corresponding target area.
[0046] In some embodiments, target radar feature parameters are obtained based on target area radar information, and target radar feature parameters are compared with preset radar feature parameters in a preset radar database to obtain target radar contrast difference degree. The target radar anomaly level is determined based on the target radar contrast difference degree and a preset radar contrast difference threshold.
[0047] In other embodiments, a preset radar signal processing algorithm (such as adding short-range anti-multipath interference filtering or a radar signal processing algorithm based on micro-Doppler features for liveness detection) is used to sense micro-movements such as the target user's breathing, pet's heartbeat, abnormal occlusion, and standard movements of non-in-vehicle objects, and the target radar anomaly level is determined based on the sensing results.
[0048] The above two methods are only for determining the target radar anomaly level based on the target area radar information. The embodiments of this disclosure do not impose specific restrictions on the implementation method of determining the target radar anomaly level based on the target area radar information.
[0049] Specifically, a dual-stage, multimodal, heterogeneous sensor architecture, employing a visual sensor for priority selection and radar for precise verification, works collaboratively for anomaly identification. First, the visual sensor (e.g., a wide-angle camera) quickly identifies spatial anomalies (e.g., object occupancy, abnormal user behavior) within the vehicle's target area based on high-resolution pixel data. Then, only when an anomaly is visually detected is the radar sensor (e.g., a four-dimensional imaging radar) triggered to verify the same target area using distance, material characteristics, or reflection attenuation (e.g., detecting distance to the source, identifying material characteristics, and source reflection attenuation rate). This avoids the energy waste of continuous radar operation, further improving anomaly identification efficiency and effectiveness. Consequently, the false alarm rate for anomaly identification can be effectively reduced (e.g., preventing visual misinterpretation of shadows as stationary objects), while also being more energy-efficient than a continuous dual-sensor solution.
[0050] Step 103: When the target radar anomaly level is greater than the preset radar anomaly threshold, obtain the target visual confidence level and the target radar confidence level; wherein, the current environmental state information is obtained based on the preset multimodal sensors in the vehicle, and the target visual confidence level and the target radar confidence level are determined based on the current environmental state information.
[0051] Step 104: Calculate the comprehensive confidence level based on the target visual anomaly level, target radar anomaly level, target visual confidence level, and target radar confidence level, and then perform alarm processing based on the comprehensive confidence level and a preset alarm threshold.
[0052] In this embodiment of the disclosure, for example, a higher target radar anomaly level indicates a higher level of danger in the target area. A radar anomaly threshold is preset, such as level one or level two. After obtaining the target radar anomaly level, the target radar anomaly level is compared with the radar anomaly threshold. When the target radar anomaly level is greater than the preset visual anomaly threshold, it indicates that there is an anomaly in the target area. Further calculation is needed by combining the target visual confidence level and the target radar confidence level to determine whether to alarm to notify the user, thereby ensuring normal safety.
[0053] Specifically, the system acquires current environmental state information based on pre-set multimodal sensors within the vehicle, and determines the target's visual confidence and radar confidence based on this information. In some embodiments, the system acquires current ambient light, current ambient humidity, and current ambient temperature based on the multimodal sensors, and calculates the target's visual confidence using a pre-set visual confidence calculation formula based on these parameters. In other embodiments, the system acquires current ambient light and current ambient temperature based on the multimodal sensors, and calculates the target's radar confidence using the current ambient humidity, current ambient temperature, and a pre-set radar confidence calculation formula.
[0054] It should be noted that the sound sensor in the multimodal sensor can be used to collect environmental noise, the vibration sensor can be used to collect vibration in the vehicle environment, and the rain sensor can be used to collect rain in the external environment to calculate the target visual confidence and target radar confidence. The specific settings should be selected according to the actual application scenario.
[0055] In practical applications, some environmental conditions may significantly affect visual or radar sensors, potentially leading to inaccurate anomaly identification results. Therefore, the target visual confidence and target radar confidence can be adjusted based on real-time environmental conditions to calculate the overall confidence score, thereby improving the accuracy of the final warning.
[0056] In this embodiment of the disclosure, after obtaining the target visual confidence score and the target radar confidence score, a comprehensive confidence score is calculated based on the target visual anomaly level, the target radar anomaly level, the target visual confidence score, and the target radar confidence score. Specifically, the target visual anomaly score can be determined based on the target visual anomaly level and the target radar anomaly score can be determined based on the target radar anomaly level. Then, the target visual confidence score, the target radar confidence score, the target visual anomaly score, and the target radar anomaly score are weighted and summed to obtain the comprehensive confidence score.
[0057] Furthermore, the overall confidence level is compared with the preset alarm threshold. For example, if the alarm threshold is preset to 0.9, a first-level alarm signal is generated when the overall confidence level is greater than the alarm threshold, and alarms are triggered according to the first-level alarm information, such as by combining auditory and visual reminders. When the overall confidence level is less than or equal to the alarm threshold, a second-level alarm signal is generated, and alarms are triggered according to the second-level alarm information, such as by sending reminder information. Among these, the first-level alarm signal takes precedence over the second-level alarm information.
[0058] The in-vehicle area anomaly recognition method of this disclosure acquires target area images based on preset visual sensors inside the vehicle and determines the target visual anomaly level based on the target area images. When the target visual anomaly level is greater than a preset visual anomaly threshold, radar information of the target area is acquired based on preset radar sensors inside the vehicle, and the target radar anomaly level is determined based on the target radar information. When the target radar anomaly level is greater than a preset radar anomaly threshold, target visual confidence and target radar confidence are acquired. The method also acquires current environmental state information based on preset multimodal sensors inside the vehicle and determines target visual confidence and target radar confidence based on the current environmental state information. A comprehensive confidence score is calculated based on the target visual anomaly level, target radar anomaly level, target visual confidence, and target radar confidence score, and an alarm is triggered based on the comprehensive confidence score and a preset alarm threshold. By employing the above technical solution, visual recognition combined with radar assistance is used to detect whether there are anomalies in target areas such as the steering wheel and dashboard inside the vehicle. The visual confidence and radar confidence scores can be dynamically adjusted under different environmental conditions for anomaly judgment, and an alarm can be triggered to notify the user when an anomaly is detected in the target area, further improving overall vehicle safety.
[0059] In one optional embodiment of this disclosure, determining the target visual anomaly level based on a target region image includes: performing visual recognition on the target region image to obtain target pixel feature parameters; wherein, the target pixel feature parameters include one or more of pixel size, pixel grayscale, pixel color, and pixel quantity; comparing the target pixel feature parameters with preset pixel feature parameters in a preset visual database to obtain a target visual contrast difference; and determining the target visual anomaly level based on the target visual contrast difference and a preset visual difference threshold.
[0060] Specifically, a visual database is pre-set. Data samples are collected from normal states across multiple scenes, modalities, and vehicle models, and the collected visual standard database is entered into the visual database. After acquiring the target region image, the target region image can be processed to obtain target pixel feature parameters such as pixel size, pixel grayscale, pixel color, and pixel quantity. These parameters are then compared with the preset pixel feature parameters in the visual database to obtain a target visual contrast difference. The target visual contrast difference is then compared with a preset visual difference threshold to determine the target visual anomaly level. Multiple different visual difference thresholds can be pre-set, with different visual difference thresholds corresponding to different target visual anomaly levels.
[0061] It should be noted that preset pixel feature parameters can be obtained from the visual database based on the current environment information to ensure the accuracy of anomaly identification by comparing pixel feature parameters under the same environment.
[0062] In this embodiment of the disclosure, the target region image can also be visually recognized based on a preset target detection model to obtain the target recognition result. The target recognition result is compared with the preset result to determine the target visual anomaly level.
[0063] Specifically, a target detection model can be pre-trained to perform visual recognition on target region images, obtain target recognition results, and compare the target recognition results with preset results to determine the target visual anomaly level. For example, if the target recognition result identifies a decoration, and the preset result has a decoration with a corresponding visual anomaly level, then when the target recognition result and the preset result match, the visual anomaly level corresponding to the preset result is taken as the target visual anomaly level.
[0064] This involves constructing a target dataset of the vehicle's interior (e.g., images of a child's head stuck in a car seat gap, a thermos spilling liquid, or a mobile phone slipping into the brake pedal gap) to train a lightweight target detection model. This improves the detection accuracy of small targets (typically less than 100 pixels) and occluded targets. Consequently, it can directly support object recognition and differentiation without comparing against a visual database. For example, it can identify human movements, limb positions, object outlines, and object materials to determine if the currently identified content exhibits abnormal behavior. Therefore, the recall rate for small target detection is significantly improved compared to traditional models, and the false positive rate for liveness detection is effectively controlled.
[0065] In one optional embodiment of this disclosure, determining the target radar anomaly level based on target area radar information includes: acquiring target radar feature parameters based on target area radar information; comparing the target radar feature parameters with preset radar feature parameters in a preset radar database to obtain a target radar comparison difference degree; and determining the target radar anomaly level based on the target radar comparison difference degree and a preset radar difference threshold.
[0066] Specifically, a visual database is pre-set, and data samples are collected from normal states across multiple scenarios, modalities, and vehicle models. The collected radar standard database is then entered into the radar database. The radar sensor performs a high-resolution, rapid scan of the target area, recording target radar characteristic parameters within the target area, including low-frequency vibration rate, reflection of minute object movements, point cloud reflection parameters, reflected wave attenuation rate, and radar reflection distance. These parameters are then compared with preset radar characteristic parameters in the radar database to obtain a target radar contrast difference degree. This degree is then compared with a preset radar contrast difference threshold to determine the target radar anomaly level. Multiple different radar contrast difference thresholds can be pre-set, with different thresholds corresponding to different target radar anomaly levels.
[0067] It should be noted that preset radar characteristic parameters can be obtained from the radar database based on the current environmental information to ensure the accuracy of anomaly identification by comparing radar characteristic parameters under the same environment.
[0068] In this embodiment of the disclosure, for short-range (e.g., 0.5-2 meters) detection scenarios inside the vehicle, the radar signal processing algorithm can be optimized (e.g., by adding short-range anti-multipath interference filtering and liveness detection based on micro-Doppler features) to improve the perception capability of micro-movements of target users such as children's breathing, pets' heartbeats, abnormal occlusion, and standard movements of objects outside the vehicle.
[0069] In order to define the baseline requirements for the normal state of the target area inside the vehicle and resolve the ambiguity of the abnormal state, and to support the consistency of multimodal data, it is necessary to establish a visual database and a radar database. The establishment of the visual database and the radar database can also enable the arbitration module to achieve adaptive adjustment in dynamic scenarios, and to adaptively adjust the comparison threshold and confidence level changes for the normal state in different sub-scenarios.
[0070] Specifically, data samples are collected from normal states across multiple scenarios, modalities, and vehicle models. This includes: Scenario dimension: standard data samples are collected covering typical scenarios such as "empty vehicle," "single-person driving," "multiple passengers (including children or pets)," "item placement (backpacks, water bottles, or toys)," and "special states (seat heating or ventilation, window opening / closing)." Modal dimension: visual (RGB (red, yellow, blue) or infrared images), radar (millimeter-wave point cloud or four-dimensional imaging data), and environmental sensor (light intensity, ambient temperature, ambient weather, seat pressure) data are collected simultaneously to form a multimodal fusion sample of "visual + radar + environment." Vehicle model dimension: sub-databases of normal states for different vehicle models are established to account for the differences in in-vehicle layout. Based on the collected data, the boundary ranges of normal / abnormal states are determined, and visual feature labels such as the pixel mean or variance of the target area, radar feature labels such as the dielectric constant of the object and the attenuation of prohibited object reflections, and abnormal correlation attributes are completed. The collected standard databases are then entered into the visual database and the radar database respectively.
[0071] In one optional embodiment of this disclosure, the current environmental state information is obtained based on a preset multimodal sensor inside the vehicle, and the target visual confidence and the target radar confidence are determined based on the current environmental state information. This includes: obtaining the current ambient light, current ambient humidity, and current ambient temperature based on the multimodal sensor; calculating the target visual confidence based on the current ambient light, current ambient humidity, current ambient temperature, and a preset visual confidence calculation formula; and calculating the target radar confidence based on the current ambient humidity, current ambient temperature, and a preset radar confidence calculation formula.
[0072] Specifically, multimodal sensors are pre-set, such as light sensors to collect ambient light, sound sensors to collect ambient noise, vibration sensors to collect vibrations inside the vehicle, rain sensors to collect rainfall outside the vehicle to determine the weather conditions, and temperature sensors to collect the ambient temperature of the vehicle.
[0073] In this embodiment of the disclosure, the confidence levels of the visual sensor and the radar sensor are dynamically adjusted according to the scene after the current scene is identified through self-learning of the multimodal sensor, visual sensor or radar sensor.
[0074] Specifically, the formula for calculating visual confidence is as follows: Wherein, Visual is the visual confidence parameter; the baseline value of the highest confidence benchmark for visual recognition under ideal in-vehicle environment (e.g., ambient light is ∞, relative humidity is 70%, and in-vehicle temperature is 25℃) is 90; e^(-lux / 2000) is the exponential decay effect of light intensity; humidity is the relative humidity in the in-vehicle environment; and temp is the in-vehicle temperature (degrees Celsius).
[0075] Specifically, the radar confidence calculation formula is Radar=85-0.4×max(0,humidity-75)-0.25*|temp-25|; where Radar is the radar confidence; the baseline value of the highest confidence of radar recognition under ideal in-vehicle environment (e.g., relative humidity of 75% and in-vehicle temperature of 25℃) is 85; humidity is the relative humidity of the in-vehicle environment; and temp is the in-vehicle temperature (degrees Celsius).
[0076] It should be noted that temperature changes in radar sensors can cause frequency drift, signal attenuation, and expansion of antenna substrate materials, which can increase the temperature ratio requirement. Typically, visual confidence and radar confidence require a range of visual[40, 90] and a radar confidence range of radar[60, 85].
[0077] Finally, the overall confidence score is calculated as follows: Overall Confidence Score = Visual Confidence Score × Visual Anomaly Score (determined based on the visual anomaly level) + Radar Confidence Score × Radar Anomaly Score (determined based on the visual anomaly level). An alarm is triggered when the overall confidence score exceeds a preset alarm threshold (e.g., 0.9). This allows the system to adapt to complex in-vehicle environments (such as strong light, low light, rain, and snow), further improving the reliability of anomaly recognition.
[0078] In this embodiment of the disclosure, updated environmental state information of the vehicle can also be obtained based on multimodal sensors, and the target visual confidence and target radar confidence can be adjusted based on the updated environmental state information.
[0079] Specifically, when the environmental condition is judged to be unfavorable to radar recognition but favorable to visual recognition, the confidence level of radar recognition is reduced and the confidence level of visual recognition is increased; when the environmental condition is judged to be unfavorable to visual recognition but favorable to radar recognition, the confidence level of visual recognition is reduced and the confidence level of radar recognition is increased; if the environmental condition is judged to be favorable or unfavorable to both visual and radar recognition, the confidence levels of both visual and radar recognition are adjusted in a balanced manner.
[0080] Specifically, the updated environmental state information of the vehicle can be obtained based on multimodal sensors. If the updated environmental state information indicates that the visual sensor recognition is unreliable, the recognition result of the visual sensor can be ignored. In this case, the target visual confidence can be reduced, thus having a relatively small impact on the final overall confidence.
[0081] Therefore, the confidence levels of visual and radar anomalies are dynamically adjusted for different environments, and alarm processing is initiated when the alarm thresholds set for different alarm stages are exceeded, thereby further improving the flexibility and accuracy of identification.
[0082] In this embodiment, when the target visual anomaly level is greater than a preset visual anomaly threshold, and further when the target radar anomaly level is less than or equal to a preset radar anomaly threshold, the visual sensor and radar sensor are controlled to synchronously identify and obtain updated visual anomaly levels and updated radar anomaly levels. Then, when the updated visual anomaly level is greater than both the visual anomaly threshold and the updated radar anomaly level are greater than the radar anomaly threshold, an updated comprehensive confidence level is calculated based on the updated visual confidence level, updated radar confidence level, updated visual anomaly level, and updated radar anomaly level. An alarm is then triggered based on the updated comprehensive confidence level and an alarm threshold. Alternatively, when the updated visual anomaly level is greater than the visual anomaly threshold and the updated radar anomaly level is less than or equal to the radar anomaly threshold, an alarm is triggered according to a preset alarm method. Alternatively, when the updated visual anomaly level is less than or equal to both the visual anomaly threshold and the updated radar anomaly level is less than or equal to the radar anomaly threshold, the step of determining the visual anomaly level based on the area image acquired by the visual sensor is updated. Alternatively, when the updated visual anomaly level is less than or equal to the visual anomaly threshold and the updated radar anomaly level is greater than the radar anomaly threshold, the visual sensor is controlled to continue identifying, and based on the identification result, a judgment is made according to the aforementioned method to determine whether to trigger an alarm and the specific alarm method.
[0083] In other words, when the visual anomaly level of the target is greater than the preset visual anomaly threshold and the radar anomaly level of the target is less than or equal to the preset radar anomaly threshold, it is necessary to control the visual sensor and the radar sensor to perform anomaly identification again through synchronous tracking, so as to further improve the accuracy of anomaly identification.
[0084] For example, multimodal sensors such as a light sensor for ambient light during exhalation, a sound sensor for ambient noise, a vibration sensor for ambient vibration, a rain sensor for ambient weather, and a temperature sensor for ambient temperature can be pre-set to obtain current environmental status information. Based on this information, the visual confidence level of the visual sensor and the radar confidence level of the radar sensor can be determined. The visual confidence level of the visual sensor and the radar confidence level of the radar sensor can be adjusted in real time according to the multimodal environment, thereby achieving efficient and effective dynamic real-time anomaly identification under different environments.
[0085] Specifically, the target area is identified using a visual sensor to acquire an image of the target area. Target pixel feature parameters are then identified based on the image and compared with preset pixel feature parameters in a pre-defined visual database to determine the target visual contrast difference. Based on the target visual contrast difference and a preset visual contrast difference threshold, the target visual anomaly level is determined. If the target visual anomaly level exceeds the preset visual anomaly threshold, target radar feature parameters are acquired based on radar information of the target area and compared with preset radar feature parameters in a pre-defined radar database to obtain the target radar contrast difference. Based on the target radar contrast difference and a preset radar contrast difference threshold, the target radar anomaly level is determined. Finally, a comprehensive confidence level is calculated based on the target visual anomaly level, target radar anomaly level, target visual confidence level, and target radar confidence level. An alarm is then triggered based on the comprehensive confidence level and a preset alarm threshold.
[0086] It should be noted that when the visual anomaly level of the target exceeds the visual anomaly threshold, the radar sensor can be controlled to perform a low-power radar scan to determine the moving object. If no anomaly is found, a high-power radar scan is performed to confirm the object's outline, material, etc. If no anomaly is found, a subsequent synchronous tracking and recognition strategy is implemented to further improve the recognition accuracy.
[0087] For example, such as Figure 2 As shown, the process includes step 2.1: the vision sensor acquires and processes images of the target area inside the vehicle to obtain target visual feature parameters (such as target pixel feature parameters); step 2.2: the target visual feature parameters are compared with data in the visual database, that is, the corresponding standard visual feature parameters are obtained from the visual database based on the current environmental state information and compared with the target visual feature parameters, and a target visual anomaly level can be determined based on the comparison result; step 2.3: it is determined whether the target visual anomaly level is greater than a preset visual anomaly threshold, that is, the target visual anomaly level is compared with the preset visual anomaly threshold.
[0088] Further, in step 2.4, when the target's visual anomaly level is greater than a preset visual anomaly threshold, the radar sensor is controlled to operate; in step 2.5, the radar sensor performs a high-resolution scan to acquire the target's radar feature parameters, and compares the target's visual feature parameters with the data in the radar database. That is, based on the current environmental state information, the corresponding standard radar feature parameters are obtained from the radar database and compared with the target's radar feature parameters. Based on the comparison result, a target radar anomaly level can be determined; in step 2.6, it is determined whether the target radar anomaly level is within the preset radar anomaly threshold. That is, the target radar anomaly level is compared with the preset radar anomaly threshold.
[0089] Further, in step 2.7, when the target radar anomaly level is greater than the preset radar anomaly threshold, a comprehensive confidence level is calculated based on the target visual anomaly level, target radar anomaly level, target visual confidence level, and target radar confidence level; in step 2.8, it is determined whether the comprehensive confidence level is greater than the preset alarm threshold; in step 2.9, when the comprehensive confidence level is greater than the preset alarm threshold, an alarm signal is generated to trigger an alarm; in step 2.10, when the target radar anomaly level is less than or equal to the preset radar anomaly threshold, a synchronous tracking signal is triggered to provide synchronous tracking to the visual sensor and radar sensor. That is, the target area is identified by the visual sensor and radar sensor, and the updated visual anomaly level and updated radar anomaly level are obtained to determine whether to issue an alarm; in step 2.11, when the calculated new comprehensive confidence level is greater than the preset alarm threshold, an alarm signal is generated to trigger an alarm; in step 2.12, when no alarm is needed, anomaly samples are recorded, the visual processing algorithm is updated, and subsequent detection is optimized.
[0090] For example, a camera is used to collect in-vehicle images and record visual feature parameters. These parameters are compared with a visual database, and the degree of difference between the parameters and the database determines the level of visual anomaly. The visual database contains standard in-vehicle visual image states for multiple scenarios to support the visual judgment module in determining the level of visual anomalies in target areas within the vehicle. Similarly, radar sensors record radar feature parameters of the in-vehicle environment, and these parameters are compared with a radar database. The degree of difference between these parameters determines the level of radar anomalies. The radar database contains standard in-vehicle radar reflection states for multiple scenarios to determine the level of anomalies in target areas within the vehicle. A comprehensive confidence score is calculated based on the visual anomaly level, the radar anomaly level, and the confidence scores for dynamically adjusting visual and radar anomalies for different environments. When the comprehensive confidence score exceeds a threshold set for different alarm stages, an alarm level signal is sent, triggering different levels of alarms.
[0091] As an example, such as Figure 3As shown, step 3.1 involves dynamically processing the confidence level based on real-time environmental state information acquired by a preset multimodal sensor. This means determining the target visual confidence level of the visual sensor and the target radar confidence level of the radar sensor based on the environmental state information. Step 3.2 involves acquiring the visual anomaly level. Step 3.3 involves determining whether the visual anomaly level exceeds a set threshold. If not, step 3.4 involves controlling the visual sensor to continuously monitor the target area, i.e., continuously acquiring and processing images of the target area and identifying the visual anomaly level. If yes, step 3.5 involves recording the visual anomaly level and controlling the radar sensor to perform high-power, rapid scanning. Step 3.6 involves acquiring the radar anomaly level. Normal level; Step 3.7 Determine if the radar anomaly level is greater than the set threshold; If not, proceed to step 3.8 to trigger the synchronization tracking mechanism and send a synchronization tracking signal to the visual sensor and the radar sensor; If yes, proceed to step 3.9 to determine the target visual confidence level of the visual sensor and the target radar confidence level of the radar sensor, as well as the visual anomaly level and the radar anomaly level, and calculate the comprehensive confidence level based on the real-time environmental status information; Step 3.10 Determine if the comprehensive confidence level exceeds the high-risk threshold; If not, proceed to step 3.11 to determine the low-risk alarm signal and perform alarm processing; If yes, proceed to step 3.12 to determine the high-risk alarm signal and perform alarm processing.
[0092] The process includes the following steps after step 3.8: Step 3.13: Obtaining new visual anomaly levels and radar anomaly levels; Step 3.14: Determining whether the new visual anomaly levels and radar anomaly levels exceed the corresponding anomaly thresholds; If both exceed the thresholds, Step 3.15: Calculating the comprehensive confidence score based on the synchronously tracked visual anomaly levels and radar anomaly levels, and determining whether the comprehensive confidence score is greater than the high-risk threshold; if so, Step 3.16: Performing alarm processing based on the high-risk alarm method; if not, Step 3.17: Performing alarm processing based on the low-risk alarm method; If the visual anomaly exceeds the threshold and the radar exceeds the threshold, Step 3.17 is executed; If both are below the thresholds, Step 3.18: No alarm is triggered, the anomaly sample is recorded, the visual algorithm is updated, and the subsequent detection scheme is optimized; If the visual anomaly exceeds the threshold and the radar exceeds the threshold, Step 3.19: Controlling the visual sensor to perform visual recognition and judgment again.
[0093] First, multimodal sensors collect environmental feature parameters inside and outside the vehicle as environmental state information to adjust the confidence levels of the visual and radar sensors. When the environmental state is judged to be unfavorable to radar recognition and favorable to visual recognition, the confidence level of radar recognition is reduced and the confidence level of visual recognition is increased. When the environmental state is judged to be unfavorable to visual recognition and favorable to radar recognition, the confidence level of visual recognition is reduced and the confidence level of radar recognition is increased. If the environmental state is judged to be favorable or unfavorable to both visual and radar recognition, the confidence levels of visual and radar recognition are adjusted in a balanced manner.
[0094] Furthermore, the vehicle interior is divided into target areas based on its layout, such as seat gaps, steering wheel airbags, dashboard airbags, passenger seat, door and window seams, seats, and floor. Images of these target areas are captured using cameras as visual sensors. Visual processing of these images yields pixel feature parameters, including pixel size, grayscale, color, and number. Visual standard states in the same environment are compared with these visual feature parameters in a visual database to determine the degree of difference between the in-vehicle visual feature parameters and the pixel feature parameters in the database. Anomaly levels are then classified based on different difference thresholds to complete preliminary visual recognition and determine the visual anomaly level.
[0095] Furthermore, if the visual anomaly level exceeds the set anomaly threshold, it is recorded and sent to control the operation of the radar sensor; if it is below the anomaly threshold, the visual sensor is required to perform continuous detection. The radar sensor first performs a high-resolution rapid scan of the target area, recording radar characteristic parameters within the target area, including low-frequency vibration rate, reflection of minute object movements, point cloud reflection parameters, reflected wave attenuation rate, radar reflection distance, etc. It also obtains the standard radar state in the same environment from the radar database and compares it with the radar characteristic parameters to confirm the degree of difference between the radar characteristic parameters and the radar characteristic parameters in the radar database. Based on different difference thresholds, the anomaly level is classified, completing the high-resolution rapid scan judgment and obtaining the radar anomaly level.
[0096] Furthermore, if the radar anomaly level exceeds the high-resolution rapid scan set anomaly threshold, it is recorded, and a comprehensive confidence level is determined based on the aforementioned dynamic confidence level and visual anomaly level. When the comprehensive confidence level exceeds the high-risk alarm set threshold, a high-risk alarm signal is sent for alarm processing; when the comprehensive confidence level is lower than the high-risk alarm set threshold, a low-risk alarm signal is sent for alarm processing. If the radar anomaly level is lower than the high-resolution rapid scan set anomaly threshold, it is recorded, and a radar-visual synchronous tracking mechanism is triggered. Simultaneously, a synchronous tracking signal is sent to the visual sensor and the radar sensor to simultaneously perform image acquisition and radar acquisition after receiving the synchronous tracking signal, and to obtain new visual anomaly levels and radar anomaly levels by combining visual feature parameters and radar feature parameters.
[0097] Finally, if both the visual anomaly level and radar anomaly level from synchronous tracking exceed the set anomaly threshold, a comprehensive confidence level is determined based on the aforementioned dynamic confidence level and the visual and radar anomaly levels from synchronous tracking. If the comprehensive confidence level exceeds the high-risk alarm threshold, a high-risk alarm signal is sent for alarm processing; if the comprehensive confidence level is below the high-risk alarm threshold, a low-risk alarm signal is sent for alarm processing. If the visual anomaly level from synchronous tracking is higher than the set anomaly threshold and the radar anomaly level is lower than the set anomaly threshold, a low-risk alarm signal is sent for alarm processing. If the visual anomaly level from synchronous tracking is lower than the set anomaly threshold and the radar anomaly level is lower than the set anomaly threshold, the misjudgment is eliminated, no alarm is triggered, the anomaly sample is recorded, the visual algorithm is updated, and the subsequent detection scheme is optimized. If the visual anomaly level from synchronous tracking is lower than the anomaly threshold and the radar anomaly level is higher than the anomaly threshold, visual recognition is performed again, and the aforementioned processing steps are executed based on the visual anomaly level determined by visual recognition.
[0098] The in-vehicle area anomaly recognition method of this disclosure detects whether there are anomalies in the target area inside the vehicle through a scheme of visual recognition plus radar assistance, and performs a dynamic weighted fusion decision scheme under different environmental conditions. At the same time, it is combined with the target area target detection model, radar signal processing algorithm, etc., to inform the user of the current danger through sound and visual reminders when anomalies are detected in the target area, thereby further improving the safety of the vehicle.
[0099] To achieve the above embodiments, this disclosure also provides an in-vehicle area anomaly recognition device, which can be implemented using software and / or hardware.
[0100] Figure 4 This is a schematic diagram of the structure of an in-vehicle area anomaly recognition device provided in an embodiment of this disclosure, as shown below. Figure 4 As shown, the in-vehicle area anomaly recognition device 40 may include: a vision processing module 410, a radar processing module 420, an acquisition module 430, and a processing module 440.
[0101] The vision processing module 410 is used to acquire images of the target area based on a preset vision sensor inside the vehicle, and to determine the target visual anomaly level based on the target area image.
[0102] The radar processing module 420 is used to acquire radar information of the target area based on a preset radar sensor in the vehicle when the visual anomaly level of the target is greater than a preset visual anomaly threshold, and to determine the radar anomaly level of the target based on the radar information of the target area.
[0103] The acquisition module 430 is used to acquire the target visual confidence level and the target radar confidence level when the target radar anomaly level is greater than a preset radar anomaly threshold; wherein, the current environmental state information is acquired based on a preset multimodal sensor in the vehicle, and the target visual confidence level and the target radar confidence level are determined based on the current environmental state information;
[0104] The processing module 440 is used to calculate a comprehensive confidence level based on the target visual anomaly level, the target radar anomaly level, the target visual confidence level, and the target radar confidence level, and to perform alarm processing based on the comprehensive confidence level and a preset alarm threshold.
[0105] Optionally, the visual processing module 410 is specifically used for:
[0106] The target area image is acquired based on a pre-set vision sensor inside the vehicle, and visual recognition is performed on the target area image to obtain target pixel feature parameters; wherein, the target pixel feature parameters include one or more of the following: pixel size, pixel grayscale, pixel color, and pixel quantity;
[0107] The target pixel feature parameters are compared with preset pixel feature parameters in a preset visual database to obtain the target visual contrast difference.
[0108] The visual anomaly level of the target is determined based on the target visual contrast difference and a preset visual difference threshold.
[0109] Optionally, the visual processing module 410 is further configured to:
[0110] When the visual anomaly level of the target is greater than the preset visual anomaly threshold, radar information of the target area is obtained based on the preset radar sensor in the vehicle.
[0111] Visual recognition of the target region image is performed based on a preset target detection model to obtain the target recognition result;
[0112] The target visual anomaly level is determined by comparing the target recognition result with the preset result.
[0113] Optionally, the radar processing module 420 is specifically used for:
[0114] Target radar characteristic parameters are obtained based on the radar information of the target area;
[0115] The target radar feature parameters are compared with the preset radar feature parameters in the preset radar database to obtain the target radar comparison difference degree.
[0116] The anomaly level of the target radar is determined based on the target radar contrast difference and a preset radar difference threshold.
[0117] Optionally, the device further includes an environmental sensing module for...
[0118] The current ambient light, current ambient humidity, and current ambient temperature are obtained based on the multimodal sensor.
[0119] The target visual confidence score is calculated based on the current ambient light, current ambient humidity, current ambient temperature, and the preset visual confidence score calculation formula.
[0120] The target radar confidence level is calculated based on the current ambient humidity, the current ambient temperature, and the preset radar confidence level calculation formula.
[0121] Optionally, the device further includes: an update module, configured to:
[0122] The updated environmental status information of the vehicle is obtained based on the multimodal sensor;
[0123] The target visual confidence level and the target radar confidence level are adjusted based on the updated environmental state information.
[0124] Optionally, when the target visual anomaly level is greater than a preset visual anomaly threshold, the device further includes: a synchronous tracking module, used for:
[0125] When the target radar anomaly level is less than or equal to a preset radar anomaly threshold, the visual sensor and the radar sensor are controlled to synchronously identify and obtain updated visual anomaly levels and updated radar anomaly levels.
[0126] When the updated visual anomaly level is greater than the visual anomaly threshold and the updated radar anomaly level is greater than the radar anomaly threshold, an updated comprehensive confidence score is calculated based on the updated visual confidence score, the updated radar confidence score, the updated visual anomaly level, and the updated radar anomaly level, and an alarm is triggered based on the updated comprehensive confidence score and the alarm threshold; or,
[0127] When the updated visual anomaly level is greater than the visual anomaly threshold and the updated radar anomaly level is less than or equal to the radar anomaly threshold, an alarm is triggered according to a preset alarm method; or,
[0128] When the updated visual anomaly level is less than or equal to the visual anomaly threshold and the updated radar anomaly level is less than or equal to the radar anomaly threshold, the visual anomaly level determination algorithm based on the region image acquired by the visual sensor is updated; or,
[0129] When the updated visual anomaly level is less than or equal to the visual anomaly threshold and the updated radar anomaly level is greater than the radar anomaly threshold, the visual sensor is controlled to identify the target area and determine the anomaly identification result.
[0130] The in-vehicle area anomaly recognition device provided in this disclosure can execute the in-vehicle area anomaly recognition method provided in this disclosure, and has the corresponding functional modules and beneficial effects of the method execution. Content not described in detail in the device embodiments of this disclosure can be referred to the description in any method embodiment of this disclosure.
[0131] This disclosure also provides an electronic device, including a processor and a memory; the processor executes the steps of the aforementioned embodiments of the in-vehicle area anomaly identification method by calling programs or instructions stored in the memory. To avoid repetition, these steps will not be repeated here.
[0132] This disclosure also provides a computer-readable storage medium that is non-transitory and stores a program or instructions that cause a computer to perform the steps of the aforementioned embodiments of the in-vehicle area anomaly identification method. To avoid repetition, these steps will not be repeated here.
[0133] This disclosure also provides a computer program product for performing the steps of the aforementioned embodiments of the in-vehicle area anomaly recognition method.
[0134] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0135] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for identifying anomalies in a vehicle interior area, characterized in that, include: The target area image is acquired based on a pre-set visual sensor inside the vehicle, and the target visual anomaly level is determined based on the target area image. When the target visual anomaly level is greater than a preset visual anomaly threshold, radar information of the target area is acquired based on a preset radar sensor inside the vehicle, and the target radar anomaly level is determined based on the target area radar information. When the target radar anomaly level is greater than a preset radar anomaly threshold, the target visual confidence level and the target radar confidence level are obtained; wherein, the current environmental state information is obtained based on a preset multimodal sensor in the vehicle, and the target visual confidence level and the target radar confidence level are determined based on the current environmental state information; A comprehensive confidence level is calculated based on the target visual anomaly level, the target radar anomaly level, the target visual confidence level, and the target radar confidence level. An alarm is then triggered based on the comprehensive confidence level and a preset alarm threshold.
2. The method according to claim 1, characterized in that, Determining the target visual anomaly level based on the target region image includes: Visual recognition is performed on the target region image to obtain target pixel feature parameters; wherein, the target pixel feature parameters include one or more of the following: pixel size, pixel grayscale, pixel color, and pixel quantity; The target pixel feature parameters are compared with preset pixel feature parameters in a preset visual database to obtain the target visual contrast difference. The visual anomaly level of the target is determined based on the target visual contrast difference and a preset visual difference threshold.
3. The method according to claim 1, characterized in that, The method further includes: Visual recognition of the target region image is performed based on a preset target detection model to obtain the target recognition result; The target visual anomaly level is determined by comparing the target recognition result with the preset result.
4. The method according to claim 1, characterized in that, Determining the target radar anomaly level based on the target area radar information includes: Target radar characteristic parameters are obtained based on the radar information of the target area; The target radar feature parameters are compared with the preset radar feature parameters in the preset radar database to obtain the target radar comparison difference degree. The anomaly level of the target radar is determined based on the target radar contrast difference and a preset radar difference threshold.
5. The method according to claim 1, characterized in that, The step of acquiring current environmental state information based on pre-set multimodal sensors inside the vehicle, and determining the target visual confidence level and the target radar confidence level based on the current environmental state information, includes: The current ambient light, current ambient humidity, and current ambient temperature are obtained based on the multimodal sensor. The target visual confidence score is calculated based on the current ambient light, current ambient humidity, current ambient temperature, and the preset visual confidence score calculation formula. The target radar confidence level is calculated based on the current ambient humidity, the current ambient temperature, and the preset radar confidence level calculation formula.
6. The method according to claim 5, characterized in that, The method further includes: The updated environmental status information of the vehicle is obtained based on the multimodal sensor; The target visual confidence level and the target radar confidence level are adjusted based on the updated environmental state information.
7. The method according to claim 1, characterized in that, When the target visual anomaly level is greater than a preset visual anomaly threshold, the method further includes: When the target radar anomaly level is less than or equal to a preset radar anomaly threshold, the visual sensor and the radar sensor are controlled to synchronously identify and obtain updated visual anomaly levels and updated radar anomaly levels. When the updated visual anomaly level is greater than the visual anomaly threshold and the updated radar anomaly level is greater than the radar anomaly threshold, an updated comprehensive confidence score is calculated based on the updated visual confidence score, the updated radar confidence score, the updated visual anomaly level, and the updated radar anomaly level, and an alarm is triggered based on the updated comprehensive confidence score and the alarm threshold; or, When the updated visual anomaly level is greater than the visual anomaly threshold and the updated radar anomaly level is less than or equal to the radar anomaly threshold, an alarm is triggered according to a preset alarm method; or, When the updated visual anomaly level is less than or equal to the visual anomaly threshold and the updated radar anomaly level is less than or equal to the radar anomaly threshold, the visual anomaly level determination algorithm based on the region image acquired by the visual sensor is updated; or, When the updated visual anomaly level is less than or equal to the visual anomaly threshold and the updated radar anomaly level is greater than the radar anomaly threshold, the visual sensor is controlled to identify the target area and determine the anomaly identification result.
8. A vehicle interior area anomaly detection device, characterized in that, The device includes: The vision processing module is used to acquire images of the target area based on preset vision sensors inside the vehicle, and to determine the level of visual anomaly of the target area based on the images of the target area. The radar processing module is used to acquire radar information of the target area based on a preset radar sensor in the vehicle when the visual anomaly level of the target is greater than a preset visual anomaly threshold, and to determine the radar anomaly level of the target based on the radar information of the target area. The acquisition module is used to acquire the target visual confidence level and the target radar confidence level when the target radar anomaly level is greater than a preset radar anomaly threshold; wherein, the current environmental state information is acquired based on a preset multimodal sensor in the vehicle, and the target visual confidence level and the target radar confidence level are determined based on the current environmental state information; The processing module is used to calculate a comprehensive confidence level based on the target visual anomaly level, the target radar anomaly level, the target visual confidence level, and the target radar confidence level, and to perform alarm processing based on the comprehensive confidence level and a preset alarm threshold.
9. An electronic device, characterized in that, include: Processor and memory; The processor executes the in-vehicle area anomaly identification method as described in any one of claims 1 to 7 by calling the program or instructions stored in the memory.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program or instructions that cause a computer to perform the in-vehicle area anomaly identification method as described in any one of claims 1 to 7.