Vehicle abnormity monitoring data processing method, device and equipment, vehicle and medium

The image data is processed through the target detection model, and combined with the vibration sensor data, the vehicle abnormality situation is comprehensively determined, which solves the problem of low accuracy of vehicle abnormality warning in the prior art, and achieves more accurate warning results.

CN118942064APending Publication Date: 2024-11-12ZHEJIANG ZEEKR INTELLIGENT TECH CO LTD +1

Patent Information

Application Number
CN202411000081.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-24
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In the prior art, only relying on the vibration sensor on the vehicle to monitor abnormal vibrations is likely to lead to misidentification and misidentification, resulting in low accuracy of vehicle abnormal warning results.

Method used

The object detection model is used to detect the image data, and combined with the vibration sensor data, a comprehensive judgment is made through the preset target area and vibration perception threshold to control the vehicle to enter the early warning state.

Benefits of technology

It improves the accuracy of vehicle abnormal warning results, reduces false alarms and missed reports, and ensures the accuracy of warning information.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118942064A_ABST
    Figure CN118942064A_ABST
Patent Text Reader

Abstract

The invention provides a vehicle abnormity monitoring data processing method, device and equipment, a vehicle and a medium, and belongs to the technical field of vehicles, and the method comprises the steps: obtaining the image data of a target object; performing target detection processing on the image data by using a target detection model, and obtaining a target vibration sensing threshold value corresponding to a scene where the target object is located; when it is detected that the target object is located in the preset target area, the staying position data and the staying duration of the target object in the preset target area are obtained; when it is detected that the staying duration reaches a first staying threshold value, first vibration position data and first vibration strength data are obtained; and when it is detected that the first vibration intensity data is larger than the target vibration sensing threshold value and the vibration position data is matched with the parking position data, the vehicle is controlled to enter a first early warning state. According to the invention, the problem of low accuracy of the vehicle abnormity early warning result is solved, the situation of false alarm or missing alarm can be reduced, and the accuracy of the vehicle abnormity early warning result is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, specifically to the field of vehicle safety technology, and in particular to a method, device, equipment, vehicle and medium for processing vehicle abnormality monitoring data. Background Art

[0002] As the popularity of cars continues to increase, car monitoring and protection technologies have also begun to develop. In order to avoid abnormal approach incidents such as deliberate scratching, smashing or vehicle scrapes, car monitoring and protection technologies such as vehicle sentinel mode that use cameras installed on the vehicle to monitor and perceive vehicle abnormalities have also begun to develop.

[0003] At present, in the existing technology, in addition to using cameras to monitor abnormal vehicle approach events, there are also vibration sensors installed on the vehicle to monitor abnormal vehicle vibrations to prevent the camera from missing abnormalities generated by the vehicle based on the monitoring data due to environmental and blind spot issues, thereby improving the accuracy of abnormal situation warning results.

[0004] However, the inventors have discovered that the prior art has at least the following technical problems: relying solely on the vibration sensors on the vehicle to monitor abnormal vibrations of the vehicle can easily lead to misidentification and missed identification of abnormal vibrations of the vehicle, thereby resulting in low accuracy of abnormal warning results for the vehicle. Summary of the invention

[0005] The present application provides a vehicle abnormality monitoring data processing method, device, equipment, vehicle and medium, which are used to solve the problem of low accuracy of vehicle abnormality warning results.

[0006] In a first aspect, the present application provides a method for processing vehicle abnormality monitoring data, comprising:

[0007] Acquire image data of the target object;

[0008] Performing target detection processing on the image data using a target detection model to detect whether the target object is located in a preset target area, and obtaining a target vibration perception threshold corresponding to the scene where the target object is located;

[0009] When it is detected that the target object is located in the preset target area, the staying position data and the staying time of the target object in the preset target area are obtained;

[0010] When it is detected that the stay time reaches a first stay threshold, first vibration position data and first vibration intensity data are acquired;

[0011] When it is detected that the first vibration intensity data is greater than a target vibration perception threshold and the vibration position data matches the stay position data, the vehicle is controlled to enter a first warning state.

[0012] In one possible implementation, the target detection model is obtained through a model training process, and the model training process includes: obtaining at least one group of sample images, each group of sample images corresponding to a target object type and an abnormal event scene type; performing a frame cut operation on each group of sample images to obtain a group of positive sample data and a group of negative sample data; assigning a first weight value to the positive sample data and a second weight value to the negative sample data; inputting the positive sample data assigned the first weight value and the negative sample data assigned the second weight value into the initial detection model to perform a model training operation to obtain the target detection model.

[0013] In a possible implementation, before inputting the positive sample data assigned with the first weight value and the negative sample data assigned with the second weight value into the initial detection model, it also includes: performing sample interpolation processing on the positive sample data assigned with the first weight value and the negative sample data assigned with the second weight value.

[0014] In a possible implementation, controlling the vehicle to enter a first warning state includes: controlling the vehicle to perform warning light operations, sending first target warning information to a user terminal, and collecting monitoring image data of the target object in a preset target area.

[0015] In a possible implementation, it also includes: when it is detected that the image clarity of the image data of the target object is lower than the clarity threshold, the radar monitoring position data and the radar monitoring time are obtained; when it is detected that the radar monitoring time reaches the second stay threshold, the second vibration position data and the second vibration intensity data are obtained; when it is detected that the radar monitoring position data matches the second vibration position data, the vehicle is controlled to enter the second warning state.

[0016] In a possible implementation, it also includes: when it is detected that the target object is outside the preset target area, obtaining third vibration intensity data and vibration duration; when it is detected that the third vibration intensity data reaches the vibration scene intensity threshold, and the vibration duration reaches the preset vibration duration threshold, controlling the vehicle to enter a third warning state.

[0017] In a second aspect, the present application also provides a vehicle abnormality monitoring data processing device, comprising:

[0018] An acquisition module, used for acquiring image data of a target object;

[0019] A visual detection module, used to perform target detection processing on the image data using a target detection model to detect whether the target object is located in a preset target area, and obtain a target vibration perception threshold corresponding to the scene where the target object is located;

[0020] The acquisition module is further used to acquire the stay position data and stay duration of the target object in the preset target area when it is detected that the target object is located in the preset target area;

[0021] a fusion judgment module, configured to obtain first vibration position data and first vibration intensity data when it is detected that the stay time reaches a first stay threshold;

[0022] The fusion judgment module is further used to control the vehicle to enter a first warning state when it is detected that the first vibration intensity data is greater than a target vibration perception threshold and the vibration position data matches the stop position data.

[0023] In a third aspect, the present application provides a vehicle abnormality monitoring data processing device, the device comprising:

[0024] A surround-view camera, used to collect image data of objects around the vehicle;

[0025] Radar, used to collect radar monitoring location data and radar monitoring duration around the vehicle;

[0026] A vibration sensor, used to collect first vibration position data, first vibration intensity data, second vibration position data, second vibration intensity data, third vibration position data and vibration duration of the vehicle;

[0027] one or more processors;

[0028] A storage device is used to store one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in the first aspect.

[0029] In a fourth aspect, the present application provides a vehicle, comprising a vehicle body and a vehicle abnormality monitoring data processing device as described in the third aspect, wherein the vehicle abnormality monitoring data processing device is deployed on the vehicle body.

[0030] In a fifth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the vehicle abnormality monitoring data processing method described in the first aspect is implemented.

[0031] The vehicle abnormal monitoring data processing method, device, equipment, vehicle and medium provided by the present application, wherein the method first obtains the image data of the target object, and then uses the target detection model to perform target detection processing on the image data to obtain the target vibration perception threshold. When the target object is detected to be located in the preset target area, the target object's residence position data and residence time in the preset target area are obtained. Then, when the residence time is detected to reach the first residence threshold, the first vibration position data and the first vibration intensity data are obtained. When the first vibration intensity data is detected to be greater than the target vibration perception threshold, and the first vibration position data matches the residence position data, the vehicle is controlled to enter the first warning state. In the whole process, by using the target vibration perception threshold corresponding to the scene where the target object is located, a judgment basis that is more in line with the current scene is provided for the subsequent detection of the first vibration intensity data, and the accuracy of the detection is improved. Through the secondary residence position data matching, it is further ensured that the result of controlling the vehicle to enter the first warning state is more accurate. The above process can obtain the final target warning information through the comprehensive judgment of visual monitoring and vibration monitoring, reduce the situation of false alarm or missed alarm, and improve the accuracy of the vehicle's warning results for abnormal events. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0033] Figure 1 A schematic diagram of an application scenario of the vehicle abnormality monitoring data processing method provided in an embodiment of the present application;

[0034] Figure 2 A schematic diagram of a process flow of a vehicle abnormality monitoring data processing method provided in an embodiment of the present application;

[0035] Figure 3 A schematic diagram of an image data transmission route provided in an embodiment of the present application;

[0036] Figure 4 A schematic diagram of the structure of the vehicle abnormality monitoring data processing device provided in the embodiment of the present application Figure 1 ;

[0037] Figure 5 A schematic diagram of the structure of the vehicle abnormality monitoring data processing device provided in the embodiment of the present application Figure 2 ;

[0038] Figure 6 A schematic diagram of the hardware structure of a vehicle abnormality monitoring data processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0040] At present, in order to prevent the vehicle from being parked in a parking lot or in an outdoor scene without security cameras, and there is no user in the vehicle, when abnormal events such as smashing the car window and deliberately damaging the vehicle occur, causing losses to the car owner, more and more car companies are beginning to develop car monitoring and protection technology. For example, the sentry mode is commonly configured on vehicles.

[0041] In the prior art, the general industry solution for the sentry mode is to simply monitor abnormal approach events of people based on the surround-view camera, and determine whether there are real abnormal approach events such as deliberate scratching of the vehicle and other behaviors that damage the vehicle based on the monitoring data of the camera, and use the vibration sensor to sense abnormal vibration events encountered by the vehicle such as vehicle scratches or collisions. However, when using the sentry mode of the existing vehicle, the inventor found that due to relying solely on the monitoring data collected by the vibration sensor to sense the scratches or collisions encountered by the vehicle, a large number of misidentifications and missed identifications will occur. For example: when a heavily loaded vehicle passes by the monitored vehicle, fireworks and firecrackers, remotely turning on the air conditioner in the car, etc., will cause the vehicle to vibrate, causing the sentry mode to mistakenly issue an early warning, affecting the user experience of using the sentry mode, and resulting in the problem of low accuracy of the vehicle abnormal warning results.

[0042] In order to solve the above technical problems, the inventors proposed the following technical concept for solving the above problems in the embodiments of the present application: based on the visual monitoring data collected by the camera, the visual monitoring data is visually judged using a pre-trained visual model, and then combined with the vehicle vibration monitoring data collected by the vibration sensor, different abnormal collision thresholds are set in advance according to experiments for different vehicle abnormal vibration scenarios. Then, the vehicle will judge whether the set abnormal collision threshold is reached based on the direction and intensity of the vibration uploaded by the vibration sensor, thereby reducing the occurrence of misidentification and missed identification, and improving the accuracy of the vehicle abnormal warning results.

[0043] Embodiment 1:

[0044] Figure 1 A schematic diagram of an application scenario of the vehicle abnormality monitoring data processing method provided in an embodiment of the present application, such as Figure 1 As shown, it includes: a vehicle terminal 101 and a vehicle 102.

[0045] The vehicle terminal 101 is used to receive various monitoring data and install visual recognition models, and determine the abnormal warning results of the vehicle based on the monitoring data. The vehicle 102 includes a vehicle body and a surround view camera and a vibration sensor installed on the vehicle, wherein the surround view camera is used to collect visual monitoring data, and the vibration sensor is used to collect vehicle vibration monitoring data. The hardware equipment and physical transmission performance on the vehicle 102 meet the requirements of the sentinel mode.

[0046] Figure 2 This is a flow chart of the vehicle abnormality monitoring data processing method provided in the embodiment of the present application. The execution subject of this embodiment can be Figure 1 The vehicle terminal 102 in the illustrated embodiment may also be other computer-related devices, and this embodiment is not particularly limited.

[0047] like Figure 2 As shown, the vehicle abnormality monitoring data processing method includes:

[0048] S201: Acquire image data of a target object.

[0049] In this embodiment, the image data of the target object may be a picture or video of the target object collected by a surround view camera installed on the vehicle. The target object may be a pedestrian, a vehicle, a non-motor vehicle, or other object that may cause damage to the vehicle.

[0050] S202: Performing target detection processing on the image data using a target detection model to detect whether the target object is located in a preset target area, and obtaining a target vibration perception threshold corresponding to the scene where the target object is located.

[0051] In this embodiment, the target detection model can be an existing object recognition model, such as a convolutional neural network model CNN. The working principle of the target detection model is to extract the bounding box bbox of the identified target object in a specific area, use the CNN model for high-dimensional abstraction (emmbedding) processing, extract deep feature information, and finally return a multi-dimensional feature value. The feature value can be used for comparison. For example, the feature value can be data used to represent local features of a face.

[0052] In this embodiment, the target detection processing is performed on the image data to identify the type of the target object, whether the target object is in the preset target area, and the scene classification when the target detection processing is performed, and then the corresponding target vibration perception threshold is indexed according to the scene classification. In this embodiment, the preset target area can be a pre-set spatial range that can trigger an alarm. For example, the area within 1 meter of the vehicle is set as the preset target area.

[0053] In this embodiment, the target vibration perception threshold changes with the scene in which the target object is located and the type of the target object. Specifically, in an optional embodiment of the present application, the target vibration perception threshold for the scene of intentional scratching can be a first vibration perception threshold of a, and the target vibration perception threshold for the scene of a moving electric vehicle can be a second vibration perception threshold of b, where a and b are both positive numbers, and a is less than b. In an optional embodiment of the present application, the target detection model can also be obtained by using image data in different scenes as sample images for model training. Specifically, the target detection model in step S202 is obtained through a model training process, and the model training process includes:

[0054] S202a: Acquire at least one group of sample images, each group of sample images corresponding to a target object type and an abnormal event scene type.

[0055] In this embodiment, the sample images can be obtained by taking pictures with the surround-view camera on the vehicle, or they can be taken from the driving recorder of other vehicles that have experienced abnormal events. In this embodiment, the abnormal event scene types may include takeaway electric vehicle scenes, low-speed car scenes, high-altitude falling objects scenes, unintentional human scratch scenes, car door scratch scenes, and intentional human scratch scenes. In the above-mentioned scenes, the model is used for training. The target object types may include electric vehicles, pedestrians, cars, bicycles, and other objects that may cause abnormal events to vehicles. For example: a set of sample images needs to be collected for the scene of intentional scratches by pedestrians, and a set of sample images also needs to be collected for the scene of unintentional human scratches by pedestrians.

[0056] S202b: Perform a frame clipping operation on each group of sample images to obtain a group of positive sample data and a group of negative sample data; assign a first weight value to the positive sample data, and assign a second weight value to the negative sample data.

[0057] In this embodiment, the frame capture operation may be a process of capturing a video at n seconds and m frames to obtain a picture, where n and m are numbers greater than 0. For example: 2 frames per second. A positive sample picture refers to a picture in which a target object exists, and a negative sample picture refers to a picture in which a target object does not exist. The judgment criteria for the presence and absence of a target object may be preset, for example: when more than 50% of the target object appears in a picture, the picture may be considered a positive sample picture.

[0058] The positive sample data is assigned a first weight value, and the negative sample data is assigned a second weight value in order to avoid the problem of imbalance between the positive sample data and the negative sample data, wherein the first weight value and the second weight value may be different weight values. In this embodiment, the weights of the positive sample data and the negative sample data as well as the difficulty classification are dynamically adjusted by adopting different weight values. For example, when the number of positive sample images is less than the number of negative sample images, the weight assigned to the positive sample images is greater than the weight of the negative samples.

[0059] S202c: Input the positive sample data assigned with the first weight value and the negative sample data assigned with the second weight value into the initial detection model to perform a model training operation to obtain a target detection model.

[0060] In this embodiment, the model training operation refers to the process of inputting positive sample images and negative sample images into the initial detection model, optimizing the relevant parameters of the initial detection model, and finally obtaining a detection model that can be used to identify the target object and classify the scene. For example, the initial detection model can be a YOLO series model or other deep learning model with object detection capabilities.

[0061] In an optional embodiment of the present application, before inputting the positive sample data assigned with the first weight value and the negative sample data assigned with the second weight value into the initial detection model in step S202c, the following further includes:

[0062] Step S202d: performing sample interpolation processing on the positive sample data assigned with the first weight value and the negative sample data assigned with the second weight value.

[0063] In this embodiment, interpolation processing refers to mixing positive sample data and negative sample data in a specific ratio, fusing the two training samples and their corresponding labels by linear interpolation to generate more training samples, so as to improve the generalization ability of the model, reduce overfitting and enhance robustness, thereby improving the classification ability of the final target detection model.

[0064] To facilitate understanding of the model training process, the actual training process of the target detection model trained by the inventor is used as an example:

[0065] 1,000 targets are set for the monitored vehicle to simulate different intrusion scenarios around the real vehicle, including ground parking scenarios, underground parking scenarios, parking scenarios with good visual conditions, and parking scenarios with poor visual conditions. There is also a simulation process of riding a bicycle and driving past the monitored vehicle. Then, 700,000 images of the monitored vehicle are obtained as samples by cutting the video at 2 frames per second, and divided into positive sample images and negative sample images. Then, the YOLO series model is used for target detection training, and the categories are set to 5 categories, namely pedestrians, bicycles, electric vehicles, tricycles, and cars. The training strategy adds focal loss (by assigning different weights to positive sample images and negative sample images, making positive sample images more prominent and negative sample images more obvious), and then performs multi-sample mixing (mixup) processing to improve the generalization ability of the model, and accelerates model fitting through large batch training to obtain the final target detection model. After verification of the sample images, the mAP value of the final target detection model reached 0.98.

[0066] S203: When it is detected that the target object is located in the preset target area, the staying position data and the staying time of the target object in the preset target area are obtained.

[0067] In this embodiment, the target object's stay position and stay duration in the preset target area can be obtained by collecting image data of the target object in the preset target area from a surround view camera on the vehicle, and analyzing the image data to obtain the target object's stay position and stay duration. The target object's stay position can be the relative position of the target object around the vehicle, for example, the target object is located in front of the front of the vehicle 1 meter away. The stay duration can be the total time the target object is in the preset target area.

[0068] S204: When it is detected that the stay time reaches a first stay threshold, first vibration position data and first vibration intensity data are obtained.

[0069] In this embodiment, the first stay threshold refers to the set time threshold for the target object to stay in the preset target area. For example, the first stay threshold is 10 seconds. The first vibration position data and the first vibration intensity data can be obtained by collecting the vibration sensor on the vehicle and analyzing and processing the vehicle terminal. It can also be the data output by some vibration sensors after directly detecting the vibration source.

[0070] Specifically, in an optional embodiment of the present application, the first vibration position data and the first vibration intensity data are vibration information uploaded by vibration sensors at various positions on the vehicle. These vibration signals can be imu values ​​measured by the inertial measurement unit IMU, that is, the acceleration and angular velocity (or angular increment) measured by the accelerometer and gyroscope in the imu sensor coordinate system. These imu values ​​can be classified into vibration intensity and vibration position. For example: the vibration position refers to the time when the body of the vehicle encounters an abnormality, such as a collision with a takeaway electric vehicle, and the collision point generates vibration. After being collected by the vibration sensor, a vibration signal is generated. The vibration signal makes an approximate position judgment based on the difference in acceleration to obtain the position where the vibration is generated, such as the left front of the vehicle.

[0071] S205: When it is detected that the first vibration intensity data is greater than the target vibration perception threshold, and the vibration position data matches the stay position data, the vehicle is controlled to enter a first warning state.

[0072] In this embodiment, when the first vibration intensity data is greater than the target vibration perception threshold, it indicates that the vehicle is encountering an abnormal event. If the vibration position data matches the target object's stop position data in the preset target area at this time, then after secondary confirmation, it can be determined that the vehicle is encountering damage to the target object, and it is necessary to generate an early warning message. The vehicle entering the first early warning state may be that the vehicle generates an alarm signal or information so that the user and owner of the vehicle can obtain the early warning message.

[0073] On the contrary, when the vibration position data does not match the target object's stop position data in the preset target area, after secondary confirmation, it can be obtained that the damage encountered by the vehicle is not the target object. For example, when a pedestrian passes through the preset target area, a heavy truck passes by, causing the vehicle body to vibrate. At this time, no warning information is generated, or the vehicle is not controlled to enter the first warning state.

[0074] Specifically, in an optional embodiment of the present application, controlling the vehicle to enter the first warning state in step S205 includes: controlling the vehicle to perform warning light operation, sending first target warning information to the user terminal, and collecting monitoring image data of the target object in the preset target area.

[0075] In this embodiment, the warning light operation may be a double flashing and yellow light on the vehicle. After sending the first target warning information to the user terminal, the user terminal may display the received first target warning information through the application software to remind the user that the vehicle is encountering an abnormal event. The collected monitoring image data of the target object in the preset target area may be recorded and saved, and these recorded videos may be used by the user to find the cause of the abnormal event later.

[0076] In summary, the vehicle abnormal monitoring data processing method provided by the embodiment of the present application first obtains the image data of the target object, and then uses the target detection model to perform target detection processing on the image data to obtain the target vibration perception threshold. When the target object is detected to be located in the preset target area, the target object's residence position data and residence time in the preset target area are obtained. Then when it is detected that the residence time reaches the first residence threshold, the first vibration position data and the first vibration intensity data are obtained. When it is detected that the first vibration intensity data is greater than the target vibration perception threshold, and the first vibration position data matches the residence position data, the vehicle is controlled to enter the first warning state. In the whole process, by using the target vibration perception threshold corresponding to the scene where the target object is located, a judgment basis that is more in line with the current scene is provided for the subsequent detection of the first vibration intensity data, and the accuracy of the detection is improved. Through the secondary residence position data matching, it is further ensured that the result of controlling the vehicle to enter the first warning state is more accurate. The above process can obtain the final target warning information through the comprehensive judgment of visual monitoring and vibration monitoring, reduce the situation of false alarm or missed alarm, and improve the accuracy of the vehicle's warning results for abnormal events. Therefore, the present application has the advantages of reducing the false alarm rate of vehicle warning and improving the accuracy of warning results.

[0077] At the same time, by using a large amount of image data corresponding to different scenes and different targets for model training, the final target detection model can further improve the accuracy of the alarm and prevent false alarms.

[0078] Based on the above embodiment, in an optional embodiment of the present application, the target vibration perception threshold may also be determined according to the target scene classification information to determine which corresponding target vibration perception threshold needs to be used to obtain a higher accuracy.

[0079] Specifically, in an optional embodiment of the present application, the process of determining the target vibration perception threshold includes:

[0080] S202a: Determine the target scene according to the target scene classification information.

[0081] In this embodiment, the target scene classification information is used to determine which specific scene the vehicle to be tested is in, and the result of the determination is used as the target scene.

[0082] S202b: According to the target scene, the corresponding vibration perception value is extracted from the correspondence between the preset target scene category and the vibration perception value gradient to be determined as the target vibration perception threshold, wherein the target scenes include the takeaway electric vehicle scene, the low-speed driving car scene, the pure vibration perception scene, the human scratch scene, the door opening scratch scene and the scene with poor visual conditions, and different scenes correspond to different vibration perception values.

[0083] In this embodiment, the correspondence between the preset target scene category and the vibration perception value gradient refers to a table or index relationship between various scene categories and vibration perception values ​​obtained in advance based on a large amount of monitoring data after analysis. Among them, the takeaway electric vehicle scene refers to a scene in which an electric vehicle rider driving an electric vehicle for takeaway delivery passes or stays by the vehicle to be monitored. The low-speed car scene can be a scene in which a car with a driving speed of less than 10 kilometers per hour passes by the vehicle to be monitored. The pure vibration perception scene refers to a scene in which the vehicle vibrates due to falling objects, thunder or fireworks near the vehicle to be monitored without specific targets. The artificial scratching scene can be a scene in which a car is deliberately scratched by a human or a car is unintentionally scratched by a human. The door opening scratching scene can be a scene in which a person in a vehicle parked next to the vehicle to be monitored touches the vehicle to be monitored when opening the door. Scenes with poor visual conditions can include underground parking scenes, parking scenes without direct light during the day, and parking scenes without lighting conditions at night. Different scene categories should have a corresponding vibration perception threshold. The vibration perception value gradient refers to the vibration perception value distributed from a low threshold to a high threshold. For example, the vibration perception value corresponding to the door scratching scene is a low threshold, and the pure vibration perception scene corresponds to a high threshold.

[0084] In an optional embodiment of the present application, the target object's staying position data and staying time in the preset target area may be obtained by calling a tracking algorithm to monitor and shoot the target object to obtain the staying image data of the target object in the preset target area. The process of obtaining the staying image data includes:

[0085] S203a: Calling a tracking algorithm to monitor and photograph the target object, and obtaining multiple frames of images to be processed of the target object in a preset target area.

[0086] In this embodiment, the tracking algorithm may be a tracking algorithm based on pedestrian re-identification (ReID track algorithm). The multiple frames of target images to be processed in the preset target area refer to multiple consecutive frames of images, which can be used to determine whether the target objects are the same object.

[0087] S203b: Perform high-dimensional abstract processing on the previous frame image to be processed, obtain a first eigenvalue, store it in a preset feature library as the previous frame eigenvalue, and assign a target feature identifier to the previous frame image to be processed.

[0088] In this embodiment, high-dimensional abstract processing refers to the process of using a neural network model CNN to do emmbedding to extract deep feature information and return a multi-dimensional first feature value. Target feature identification refers to assigning a unique identification ID to the first feature value.

[0089] S203c: Perform high-dimensional abstract processing on the subsequent frame to be processed image after the previous frame to be processed image to obtain the subsequent frame feature value, and perform feature comparison processing on the subsequent frame feature value and the previous frame feature value in the preset feature library to obtain the comparison result.

[0090] In this embodiment, the principle of obtaining the subsequent feature value is similar to that of the previous frame feature value, so this embodiment will not be repeated here. The preset feature library refers to a database for storing feature values ​​obtained after each high-dimensional abstract processing.

[0091] S203d: When it is detected that the comparison result is greater than the preset feature comparison threshold, the same target feature identifier is assigned to the subsequent frame to be processed image, and the step of performing high-dimensional abstract processing on the subsequent frame to be processed image after the previous frame to be processed image is returned, and all the images to be processed whose feature identifiers are the target feature identifiers are determined as the stay image data.

[0092] In this embodiment, the preset feature comparison threshold may be a pre-set value, the comparison result may be a feature similarity, and the preset feature comparison threshold may be 95% or other values ​​indicating a high similarity and being considered as the same object. At this time, two adjacent previous frame images to be processed and subsequent frame images to be processed may be considered as images captured for tracking of the same object, that is, object tracking of the previous and subsequent frames is completed. Until the final stop image data is obtained.

[0093] Based on the above embodiment, in an optional embodiment of the present application, the dwell time is the cumulative time after the target enters the preset target area or the number of continuous video stream frames after the target enters the preset target area. Accordingly, in step S204, it is detected that the dwell time reaches the first dwell threshold, including:

[0094] S204a: It is detected that the accumulated duration reaches a first stay threshold.

[0095] In this embodiment, the accumulated duration refers to the time count starting from when the target object enters the preset target area until the time count reaches a first stay threshold, and the first stay threshold may be a preset fixed duration value, for example, 0.2 seconds.

[0096] S204b: Or it is detected that the number of continuous video stream frames reaches a preset frame number threshold.

[0097] In this embodiment, the data corresponding to the duration in the monitored video data may also be the number of frames in the video, for example, for a 30fps video stream, the corresponding preset frame number threshold may be 6 consecutive frames. When the target object is detected in the preset target area in six consecutive frames of images, it indicates that an abnormal event may have occurred.

[0098] In an optional embodiment of the present application, the vehicle abnormality monitoring data processing method, after controlling the vehicle to enter the first warning state in step S205, further includes:

[0099] When it is detected that the target object leaves the preset target area and / or the acquired first vibration position data and first vibration intensity data are zero, and the duration of zero reaches the preset no-vibration signal duration threshold, the vehicle is controlled to release the first warning state.

[0100] In this embodiment, the alert state can be released by stopping the double flashing and the yellow light, stopping obtaining the recorded video of the target object, and sending the warning state release information to the user device. In an optional embodiment of the present application, when the first vibration intensity data is less than a preset vibration intensity threshold that does not trigger the first warning state of the vehicle, and the duration reaches a preset no vibration signal duration threshold, the vehicle can also be controlled to release the first warning state.

[0101] Based on the above embodiment, a vehicle abnormality monitoring data processing method provided in an optional embodiment of the present application further includes:

[0102] Step A: When the image clarity of the image data of the detected target object is lower than the clarity threshold, the radar monitoring position data and the radar monitoring duration are obtained.

[0103] In this embodiment, when it is detected that the image clarity of the visual monitoring data is lower than the clarity threshold, it means that the target object captured by the camera on the vehicle cannot be identified. For example, in scenes with poor visual conditions such as unlit parking scenes, the radar installed on the vehicle can collect the target objects around the vehicle. On the contrary, the image data of the target object can be collected only by the surround camera. In this embodiment, the radar can be a laser radar, an ultrasonic radar, and a millimeter wave radar. According to the direction of the radar transmitting signal and the time difference between sending and receiving signals, the specific position of the target object can be calculated, and the specific position is the radar monitoring position data. The radar monitoring duration can be based on the total time from the time the target enters the preset target area to the time it leaves the preset target area.

[0104] Step B: When it is detected that the radar monitoring intrusion duration reaches the second dwell threshold, the second vibration position data and the second vibration intensity data are acquired.

[0105] Step C: When it is detected that the radar monitoring position data matches the vibration position data, the vehicle is controlled to enter the second warning state.

[0106] In this embodiment, when it is detected that the radar monitoring intrusion duration data reaches the second stay threshold, it means that the target object may cause an abnormal event to the vehicle. At this time, the vehicle needs to be controlled to enter the second warning state.

[0107] In this embodiment, the second dwell time threshold may be the same as the first dwell time threshold, or it may be a pre-set time threshold applicable to the radar monitoring scenario. The difference between the second warning state and the first warning state in the above embodiment is that the warning information sent to the user terminal in this embodiment is slightly different, and there is no need to collect monitoring image data of the target object in the preset area. For example, the first target warning information may be "Your vehicle is being scratched by object A", and the warning information in this embodiment may be "Your vehicle is being scratched in the morning, and the object causing the scratch is unknown."

[0108] In this embodiment, the second vibration position data and the first vibration position data, the second vibration intensity data and the first vibration intensity data may be the same or different, and may be specifically set according to different scenarios in which the vehicle is located.

[0109] When the vehicle enters the second warning state, if the light generated by the warning light operation can make the image clarity of the image data collected by the surround-view camera greater than or equal to the clarity threshold, then controlling the vehicle to enter the second warning state also includes: monitoring image data of the target object in the preset area can be collected.

[0110] Of course, in an optional embodiment of the present application, radar monitoring data is used to identify the specific location of the target object around the vehicle. It is also possible to obtain a radar monitoring recognition model by performing model training on a sample set of radar monitoring data, and then use the radar monitoring model to identify subsequent radar monitoring data, so as to make the radar monitoring location data and radar monitoring intrusion duration data more accurate.

[0111] In summary, the method provided in the embodiment of the present application also performs double judgment by using the radar monitoring position data, radar monitoring duration, and the second vibration position data and second vibration intensity data monitored by the vibration sensor, thereby further improving the accuracy of the target warning information obtained in parking scenarios with poor visual conditions.

[0112] Based on the above embodiment, a vehicle abnormality monitoring data processing method provided in an optional embodiment of the present application further includes:

[0113] Step D: When it is detected that the target object is outside the preset target area, third vibration intensity data and vibration duration are obtained.

[0114] Step E: When it is detected that the third vibration intensity data reaches the vibration scene intensity threshold, and the vibration duration reaches the preset vibration duration threshold, the vehicle is controlled to enter the third warning state.

[0115] In this embodiment, when the target detection result is that the target object is not in the preset target area, but there is body vibration monitoring data, it indicates that the vehicle may be in a scene where someone deliberately scratches the vehicle, that is, the target object will deliberately avoid being photographed by the camera. At this time, a comprehensive judgment can be made based on the third vibration intensity data and vibration duration data collected by the vibration sensor on the vehicle. When the third vibration intensity data reaches the vibration scene intensity threshold, and the vibration duration data reaches the preset vibration duration threshold, it indicates that the vehicle is experiencing an abnormal event, which is an act of deliberately scratching the vehicle or damaging the vehicle body.

[0116] The vibration scene intensity threshold may be a vibration intensity value obtained by the inventor in advance based on a test in a scene where no target object is detected in a preset target area and the false alarm rate is less than or equal to 5%.

[0117] In this embodiment, when the vehicle enters the third warning state, the warning information sent to the user terminal is a warning information of no target object, which can be the same as or different from the target warning information in the above embodiment. And there is no need to collect monitoring image data of the target object in the preset target area. For example, at different times, the warning information of no target object can be "Your vehicle is being damaged intentionally by humans, and the target object cannot be photographed. Please go to the parking site to check as soon as possible."

[0118] In summary, the method provided in the embodiment of the present application also obtains warning information by judging whether the vehicle needs to be controlled in a vibration scenario without a target object only through data collected by the vibration sensor when the target object cannot be determined based on image data, thereby improving the warning performance of the vehicle in parking scenarios where the vehicle is deliberately damaged.

[0119] Embodiment 2:

[0120] The above embodiments describe the vehicle abnormality monitoring data processing method for some newly produced vehicles or relatively recent vehicles that support the sentinel mode. For some old models, in order to enable these models to support the sentinel mode, target warning information can be generated when parking and the vehicle can be controlled to enter the warning state. The inventor also found that the difference from the above embodiments is that the transmission mode of the video stream can be adjusted by software control so that the vehicle terminal of the old model can obtain visual monitoring data.

[0121] Figure 3 This is a schematic diagram of the image data transmission route provided in the embodiment of the present application. Figure 3 As shown in the figure, the old vehicle models are equipped with multiple cameras, wide-angle visual modules WAM, driving recorders DVR, body gateway modules VGM, and driving information or entertainment host DHU. When the camera collects visual monitoring data (i.e. video stream) for the target object, it will transmit the visual monitoring data to WAM through a physical transmission line, and then to DVR through Ethernet, and then the driving recorder will transmit it to VGM through Ethernet, and finally VGM will transmit the visual monitoring data to DHU through Ethernet. Compared with WAM directly transmitting to DHU through a physical transmission line, the transmission performance is better, providing more real-time visual monitoring data for subsequent visual tracking and obtaining target warning information.

[0122] In this regard, a vehicle abnormality monitoring data processing method provided in an optional embodiment of the present application further includes:

[0123] Step F: When it is detected that the speed of acquiring the visual monitoring data of the target object is lower than the preset transmission speed, the wide-angle visual module WAM is controlled to receive the visual monitoring data of the target object collected by multiple cameras.

[0124] In this embodiment, the preset transmission speed may be a video stream transmission speed that has been pre-tested to meet the conditions for use of the sentinel mode, which may indicate that the physical transmission line does not meet the transmission requirements. First, the WAM is controlled to receive visual monitoring data for a target object collected by multiple cameras.

[0125] In an optional embodiment of the present application, before performing step G, Ethernet wake-up and activation operations need to be performed.

[0126] First, when the state machine of DHU jumps from 0 / 1 to 2, it pulls the identifier PNC23 of a specific network controller or network interface to wake up the Ethernet. When the state jumps to 2, a wake-up signal is sent to PNC23 or other network controllers. You can also use a timeout setting VFC Time Out Delay to set it to 30 seconds. When the timer reaches the specified timeout setting of 30 seconds, perform the IP wake-up operation, namely IPWakeUp, to wake up the Ethernet, so that the DVR, WAM and VGM devices can transmit data through the Ethernet. The DHU then sends a request to the devices DVR, WAM and VGM that use Ethernet transmission through the vehicle's CAN bus. The request is group number RG=4, priority Prio=0, and the maximum timeout limit is 30 seconds. To ensure that the request is responded within 30 seconds.

[0127] Then, a timer is set in the software, and the DHU sends an IPactivity packet to the DVR, WAM, VGM, and DHU every second. This can be achieved through the Keep-alive mechanism in the TCP / IP protocol stack.

[0128] Finally, after the DVR, WAM, VGM and DHU return the response of IPactivity to the DHU, the DHU uses the network broadcast function of Ethernet to send a broadcast signal VstdModeSts signal with the specified value value=2 / 3 / 4.

[0129] Step G: Control the wide-angle visual module WAM to send the visual monitoring data to the driving recorder DVR via Ethernet.

[0130] Step H: Control the driving recorder DVR to send the visual monitoring data to the vehicle body gateway module VGM via Ethernet.

[0131] Step I: Control the vehicle body gateway module VGM to complete the video stream transmission operation of the visual monitoring data through Ethernet to obtain the final visual monitoring data, and transfer to the step of performing target detection processing on the visual monitoring data using the target detection model.

[0132] In this embodiment, the wide-angle visualization module WAM receives the visual monitoring data captured by multiple cameras through physical transmission. Before receiving, a self-wake-up mechanism is set in WAM, which can be based on a timer, event trigger or other logical conditions. Then WAM sends it to the driving recorder DVR via Ethernet. When sending, it is transmitted to the driving recorder through the virtual local area network Vlan3. Then the DVR maintains self-wake-up through the built-in self-wake-up mechanism. And after waking up, the network interface is configured to convert the virtual local area network Vlan3 into the virtual local area network Vlan2, and then the visual monitoring data is transmitted to VGM through the video monitoring data on Vlan2. At the same time, the target IP address sent is DHU, and VGM finally sends the video data to DHU through Ethernet. After that, you can jump to the steps described in Example 1 to control the vehicle to enter the corresponding warning state.

[0133] In summary, the vehicle abnormality monitoring data processing method provided in the embodiment of the present application uses Ethernet to complete the transmission of visual monitoring data through the camera, WAM, DRV and VGM in sequence and finally to the DHU. This allows old models to report abnormal events encountered by the vehicle to the user in a timely manner, thereby improving the applicability and popularity of the method.

[0134] Figure 4 A schematic diagram of the structure of the vehicle abnormality monitoring data processing device provided in the embodiment of the present application Figure 1 The device includes: an acquisition module 41, a visual detection module 42 and a fusion judgment module 43.

[0135] The acquisition module 41 is used to acquire image data of the target object.

[0136] The visual detection module 42 is used to perform target detection processing on the image data using the target detection model to detect whether the target object is located in a preset target area and obtain a target vibration perception threshold corresponding to the scene where the target object is located.

[0137] The acquisition module 41 is further configured to acquire the staying position data and the staying time of the target object in the preset target area when it is detected that the target object is in the preset target area.

[0138] The fusion judgment module 43 is used to obtain the first vibration position data and the first vibration intensity data when it is detected that the stay time reaches the first stay threshold.

[0139] The fusion judgment module 43 is further configured to control the vehicle to enter a first warning state when it is detected that the first vibration intensity data is greater than a target vibration perception threshold and the vibration position data matches the stop position data.

[0140] In an optional embodiment of the present application, the target detection model used by the visual detection module 41 is obtained through a model training process, and the model training process includes: obtaining at least one group of sample images, each group of sample images corresponding to a target object type and an abnormal event scene type; performing a frame capture operation on each group of sample images to obtain a group of positive sample data and a group of negative sample data; assigning a first weight value to the positive sample data and a second weight value to the negative sample data; inputting the positive sample data assigned the first weight value and the negative sample data assigned the second weight value into the initial detection model to perform a model training operation to obtain a target detection model.

[0141] In an optional embodiment of the present application, the target detection model used by the visual detection module 41 includes: performing sample interpolation processing on the positive sample data assigned to the first weight value and the negative sample data assigned to the second weight value before inputting the positive sample data assigned to the first weight value and the negative sample data assigned to the second weight value into the initial detection model during the model training process.

[0142] In an optional embodiment of the present application, the fusion judgment module 43 is used to control the vehicle to enter a first warning state, including: controlling the vehicle to perform warning light operations, sending first target warning information to the user terminal, and collecting monitoring image data of the target object in a preset target area.

[0143] In an optional embodiment of the present application, the acquisition module 41 is further used to: when the image clarity of the image data of the detected target object is lower than the clarity threshold, the radar monitoring position data and the radar monitoring duration are acquired; when the radar monitoring intrusion duration is detected to reach the second dwell threshold, the second vibration position data and the second vibration intensity data are acquired. The fusion judgment module 43 is further used to: when it is detected that the radar monitoring position data matches the second vibration position data, the vehicle is controlled to enter the second warning state.

[0144] In an optional embodiment of the present application, the acquisition module 41 is further used to: when it is detected that the target object is outside the preset target area, obtain the third vibration intensity data and the vibration duration. The fusion judgment module 43 is also used to: when it is detected that the third vibration intensity data reaches the vibration scene intensity threshold, and the vibration duration reaches the preset vibration duration threshold, control the vehicle to enter the third warning state.

[0145] Figure 5 A schematic diagram of the structure of the vehicle abnormality monitoring data processing device provided in the embodiment of the present application Figure 2 .

[0146] In an optional embodiment of the present application, a vehicle abnormality monitoring data processing device is also provided. Figure 4The difference between the device embodiment shown is that the vehicle abnormality monitoring data processing device provided by this embodiment also includes: an Ethernet transmission control module 44, which is specifically used to: when it is detected that the speed of obtaining the visual monitoring data of the target object is lower than the preset transmission speed, control the wide-angle visual module WAM to receive the visual monitoring data of the target object collected by multiple cameras; control the wide-angle visual module WAM to send the visual monitoring data to the driving recorder DVR via Ethernet; control the driving recorder DVR to send the visual monitoring data to the vehicle body gateway module VGM via Ethernet; control the vehicle body gateway module VGM to complete the video stream transmission operation of the visual monitoring data via Ethernet to obtain the final visual monitoring data, and transfer to the step of performing target detection processing on the visual monitoring data using the target detection model.

[0147] The vehicle abnormality monitoring data processing device provided in this embodiment can be used to execute the technical solution in the above-mentioned method embodiment 2. Its implementation principle and technical effects are similar and will not be repeated in this embodiment.

[0148] Figure 6 A schematic diagram of the hardware structure of the vehicle abnormality monitoring data processing device provided in the embodiment of the present application is shown in FIG. Figure 6 As shown, the device includes:

[0149] The surround view camera 601 is used to collect the location data around the vehicle and the radar monitoring duration.

[0150] Radar 602 is used to collect radar position data and radar monitoring duration around the vehicle.

[0151] The vibration sensor 603 is used to collect first vibration position data, first vibration intensity data, second vibration position data, second vibration intensity data, third vibration position data and vibration duration of the vehicle.

[0152] one or more processors 604;

[0153] The storage device 605 is used to store one or more programs; when the one or more programs are executed by one or more processors 604, the various steps involved in the above method embodiment are implemented. For details, please refer to the relevant description in the above method embodiment.

[0154] Optionally, the storage device 605 may be independent or integrated with the processor 604 .

[0155] When the storage device 605 is independently provided, the apparatus further includes a bus 606 for connecting the storage device 605 and the processor 604 .

[0156] An embodiment of the present application also provides a vehicle, comprising a vehicle body and the vehicle abnormality monitoring data processing device described in the above embodiment, wherein the vehicle abnormality monitoring data processing device is deployed on the vehicle body.

[0157] In this embodiment, deployment refers to fixing the surround view camera, radar, vibration sensor, storage device and processor in various places of the vehicle body by hardware installation. The vehicle body can be an old model that does not originally support the sentry mode, or a factory or modified vehicle that supports the sentry mode.

[0158] An embodiment of the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above-mentioned vehicle abnormality monitoring data processing method is implemented.

[0159] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above vehicle abnormality monitoring data processing method when executed by a processor.

[0160] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0161] The modules described above as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of this embodiment.

[0162] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each module may exist physically separately, or two or more modules may be integrated into one unit. The unit composed of the above modules may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0163] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the method of each embodiment of the present application.

[0164] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor.

[0165] The memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk, etc.

[0166] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0167] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0168] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic device or a main control device.

[0169] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0170] The present invention is intended to illustrate the technical solution of the present application rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solution from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A vehicle abnormality monitoring data processing method, characterized in that: include: Acquire image data of the target object; Performing target detection processing on the image data using a target detection model to detect whether the target object is located in a preset target area, and obtaining a target vibration perception threshold corresponding to the scene where the target object is located; When it is detected that the target object is located in the preset target area, the staying position data and the staying time of the target object in the preset target area are obtained; When it is detected that the stay time reaches a first stay threshold, first vibration position data and first vibration intensity data are acquired; When it is detected that the first vibration intensity data is greater than a target vibration perception threshold and the vibration position data matches the stay position data, the vehicle is controlled to enter a first warning state.

2. The method according to claim 1, characterized in that The target detection model is obtained through a model training process, and the model training process includes: Acquire at least one group of sample images, each group of sample images corresponding to a target object type and an abnormal event scene type; Performing a frame clipping operation on each group of sample images to obtain a group of positive sample data and a group of negative sample data; assigning a first weight value to the positive sample data and assigning a second weight value to the negative sample data; The positive sample data assigned with a first weight value and the negative sample data assigned with a second weight value are input into the initial detection model to perform a model training operation to obtain a target detection model.

3. The method according to claim 2, characterized in that Before inputting the positive sample data assigned with the first weight value and the negative sample data assigned with the second weight value into the initial detection model, the method further includes: The positive sample data assigned with the first weight value and the negative sample data assigned with the second weight value are subjected to sample interpolation processing.

4. The method according to claim 1, characterized in that: The controlling the vehicle to enter the first warning state comprises: The vehicle is controlled to perform warning light operation, send first target warning information to a user terminal, and collect monitoring image data of the target object in a preset target area.

5. The method according to claim 1, characterized in that Also includes: When it is detected that the image clarity of the image data of the target object is lower than the clarity threshold, radar monitoring position data and radar monitoring duration are obtained; When it is detected that the radar monitoring intrusion duration reaches a second dwell threshold, second vibration position data and second vibration intensity data are acquired; when it is detected that the radar monitoring position data matches the second vibration position data, the vehicle is controlled to enter a second warning state.

6. The method according to claim 1, characterized in that Also includes: When it is detected that the target object is outside the preset target area, third vibration intensity data and vibration duration are obtained; When it is detected that the third vibration intensity data reaches the vibration scene intensity threshold, and when the vibration duration reaches the preset vibration duration threshold, the vehicle is controlled to enter the third warning state.

7. A vehicle abnormality monitoring data processing device, characterized in that: include: An acquisition module, used for acquiring image data of a target object; A visual detection module, used to perform target detection processing on the image data using a target detection model to detect whether the target object is located in a preset target area, and obtain a target vibration perception threshold corresponding to the scene where the target object is located; The acquisition module is further used to acquire the stay position data and stay duration of the target object in the preset target area when it is detected that the target object is located in the preset target area; a fusion judgment module, configured to obtain first vibration position data and first vibration intensity data when it is detected that the stay time reaches a first stay threshold; The fusion judgment module is further used to control the vehicle to enter a first warning state when it is detected that the first vibration intensity data is greater than a target vibration perception threshold and the vibration position data matches the stop position data.

8. A vehicle abnormality monitoring data processing device, characterized in that: The device comprises: Surround view camera, Used to collect image data of targets around the vehicle; Radar, used to collect radar monitoring location data and radar monitoring duration around the vehicle; A vibration sensor, used to collect first vibration position data, first vibration intensity data, second vibration position data, second vibration intensity data, third vibration position data and vibration duration of the vehicle; one or more processors; A storage device for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-6.

9. A vehicle comprising a vehicle body and the vehicle abnormality monitoring data processing device according to claim 8, wherein the vehicle abnormality monitoring data processing device is deployed on the vehicle body.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When the processor executes the computer-executable instructions, the vehicle abnormality monitoring data processing method according to any one of claims 1 to 6 is implemented.

Citation Information

Patent Citations

  • Vehicle monitoring method, device and system and storage medium

    CN109118693A

  • Vehicle monitoring method and device and computer readable storage medium

    CN115880634A

  • Method and system for adjusting threshold value of vehicle active safety early warning system

    CN116215545A

  • Vehicle safety monitoring method, device and equipment and storage medium

    CN117068194A

  • Early warning control method and device, vehicle and storage medium

    CN117302104A

Cited By

  • Vehicle anti-theft detection method, vehicle and storage medium

    CN121180145A

  • Vehicle theft prevention detection method, vehicle, and storage medium

    CN121180145B

  • Sensing method, and electronic device, system, vehicle, storage medium and program product

    WO2026113967A1