A vehicle safety monitoring system and method

Abstract

This invention relates to the field of image processing technology and provides a vehicle safety monitoring system and method. The vehicle safety monitoring method includes: acquiring vibration data collected by multiple vibration sensors and determining the vibration direction based on the multiple vibration data; when the vibration direction is the side of the vehicle body, acquiring video monitoring data corresponding to multiple cameras; extracting abnormal pixels between the first and last frames of the video monitoring data; when the number of abnormal pixels exceeds a first threshold, determining the current monitoring direction as the collision location; when the number of abnormal pixels does not exceed the first threshold, calculating the offset distance of the abnormal pixels between the first and last frames; when the offset distance is greater than a second threshold and the angle between the offset direction and the current monitoring location is greater than a preset angle, determining the current monitoring location as the collision location. This invention improves the accuracy of collision recognition through the coordinated operation of vibration sensors and cameras.

Description

Technical Field

[0001] This invention belongs to the technical field of image processing, and in particular relates to a vehicle safety monitoring system and method. Background Technology

[0002] With the rapid development of the automotive industry, vehicle safety has become a major concern for more and more consumers. Traditional vehicle safety monitoring systems mainly rely on devices such as cameras, radar, and ultrasonic sensors to detect and identify the environment and potential hazards around the vehicle. However, existing safety monitoring technologies often suffer from problems such as untimely response and insufficient accuracy when dealing with sudden events such as vehicle impacts and vibrations.

[0003] Specifically, traditional vehicle safety monitoring systems typically rely on a single sensor (such as a vibration sensor) to determine the location of external vibrations, which often lacks sufficient accuracy to pinpoint the impact point or collision location in a timely manner. Furthermore, existing systems often depend on simple vibration data analysis, but in real-world scenarios, vibration sources may be subtle, and the vibration data may contain significant noise, leading to inaccurate determination of vibration location and consequently affecting the accuracy of collision detection.

[0004] In this context, employing multiple sensors (such as vibration sensors and cameras) in combination can more accurately determine the location of vibrations and utilize video surveillance data to further analyze the nature of the vibrations and potential collision scenarios, becoming a new method to improve vehicle safety. This method not only improves the accuracy of vibration location determination but also allows for further verification using video surveillance data when vibrations occur, thereby determining whether a collision event has occurred and promptly sending warnings to users, achieving a more reliable safety protection effect.

[0005] Therefore, how to utilize the synergistic effect of multiple sensors such as vibration sensors and cameras to improve the accuracy of vibration location identification in automotive safety monitoring systems has become an urgent problem to be solved in the field of automotive safety monitoring technology. Summary of the Invention

[0006] In view of this, embodiments of the present invention provide an automotive safety monitoring system and method to solve the technical problem of how to improve the accuracy of the automotive safety monitoring system in identifying the direction of vibration.

[0007] A first aspect of the present invention provides a vehicle safety monitoring system, the vehicle safety monitoring system including multiple vibration sensors located at different positions on the vehicle body, multiple cameras monitoring different directions, and a processor;

[0008] The vibration sensor is used to collect vibration data;

[0009] The multiple cameras in different monitoring positions are used to collect video monitoring data from different monitoring positions.

[0010] The processor is used to acquire vibration data collected by multiple vibration sensors and determine the vibration direction based on the multiple vibration data; the vibration direction includes the bottom direction, the top direction, or the side direction of the vehicle body.

[0011] The processor is used to acquire video monitoring data corresponding to multiple cameras when the vibration location is the side of the vehicle body; wherein, the video monitoring data includes the N frames before the vibration start time point;

[0012] The processor is used to extract abnormal pixels between the first and last frames of the video surveillance data.

[0013] When the number of abnormal pixels exceeds a first threshold, the processor will use the current monitoring location as the collision location and send the collision location to the user terminal.

[0014] The processor is used to calculate the offset distance of the abnormal pixel between the first frame image and the last frame image when the number of abnormal pixels does not exceed a first threshold; wherein, the first frame image refers to the starting frame in the video surveillance data, and the last frame image refers to the ending frame in the video surveillance data.

[0015] The processor is used to take the current monitoring location as the collision location and send the collision location to the user terminal when the offset distance is greater than the second threshold and the angle between the offset direction and the current monitoring location is greater than a preset angle.

[0016] A second aspect of this invention provides a vehicle safety monitoring method, which is applied to a vehicle safety monitoring system, and includes:

[0017] The vibration data collected by multiple vibration sensors is acquired, and the vibration direction is determined based on the multiple vibration data; the vibration direction includes the bottom direction, the top direction, or the side direction of the vehicle body.

[0018] When the vibration location is the side of the vehicle body, video monitoring data corresponding to multiple cameras is acquired; wherein, the video monitoring data includes the N frames before the vibration start time point;

[0019] Extract abnormal pixels between the first and last frames of the video surveillance data;

[0020] When the number of abnormal pixels exceeds a first threshold, the current monitoring location is taken as the collision location and sent to the user terminal.

[0021] When the number of abnormal pixels does not exceed a first threshold, the offset distance of the abnormal pixels between the first frame image and the last frame image is calculated; wherein, the first frame image refers to the starting frame in the video surveillance data, and the last frame image refers to the ending frame in the video surveillance data;

[0022] When the offset distance is greater than the second threshold and the angle between the offset direction and the current monitoring position is greater than the preset angle, the current monitoring position is taken as the collision position and the collision position is sent to the user terminal.

[0023] Furthermore, the step of acquiring vibration data collected by multiple vibration sensors and determining the vibration orientation based on the multiple vibration data includes:

[0024] Acquire vibration data collected by multiple vibration sensors; the vibration data includes X-axis vibration data, Y-axis vibration data and Z-axis vibration data;

[0025] If multiple Z-axis vibration data are greater than the third threshold, the vibration orientation is determined to be either the bottom orientation or the top orientation.

[0026] If multiple X-axis vibration data or multiple Y-axis vibration data are greater than a third threshold, the vibration orientation is determined to determine the side orientation of the vehicle body; wherein, the side orientation of the vehicle body includes the front orientation, rear orientation, left orientation, and right orientation.

[0027] Furthermore, the step of extracting abnormal pixels between the first and last frames of the video surveillance data includes:

[0028] Extract the dividing line between the vehicle image area and the non-vehicle image area in the video surveillance data;

[0029] Based on the dividing line between the vehicle image area and the non-vehicle image area, a first image area to be identified is cropped around the dividing line of the first frame image, and a second image area to be identified is cropped around the dividing line of the last frame image.

[0030] Calculate the pixel difference between each pixel in the first image region to be identified and the second image region to be identified, and identify pixels with a pixel difference greater than a fourth threshold as abnormal pixels.

[0031] Furthermore, the step of extracting the dividing line between the vehicle image region and the non-vehicle image region in the video surveillance data includes:

[0032] Obtain the vehicle body color information input by the user;

[0033] In the first image region to be identified in the first frame image, the current image region corresponding to the vehicle body color information is extracted; wherein, the number of pixels in the current image region is greater than a first preset number and the pixels are in a continuous relationship;

[0034] The current image region is taken as the vehicle body image region, and the remaining region in the first frame image is taken as the non-vehicle body image region; the remaining region refers to the image region in the first frame image other than the current image region.

[0035] The boundary line between the vehicle image area and the non-vehicle image area is used as the dividing line.

[0036] Further, the step of calculating the offset distance of the abnormal pixels between the first frame and the last frame when the number of abnormal pixels does not exceed the first threshold includes:

[0037] If the number of abnormal pixels does not exceed the first threshold, then an abnormal region composed of consecutive abnormal pixels is extracted; wherein the number of pixels in the abnormal region is greater than the second preset number.

[0038] In the last frame image, identify multiple current feature regions in the abnormal region and extract the first position of the multiple current feature regions;

[0039] In the video surveillance data, feature regions are tracked sequentially from the last frame image to the first frame image to obtain the second position corresponding to each of the current feature regions in the first frame image.

[0040] Calculate the positional movement distance between the first and second positions corresponding to each of the multiple current feature regions, and calculate the average value among the multiple positional movement distances;

[0041] The average value is used as the offset distance.

[0042] Further, the step of identifying multiple current feature regions in the abnormal region in the tail frame image and extracting the first positions of the multiple current feature regions includes:

[0043] The abnormal regions in the last frame image are binarized based on multiple preset grayscale values ​​to obtain multiple binarized images;

[0044] Identify connected regions in a binary image;

[0045] Calculate the grayscale stability index and texture stability index for each connected region;

[0046] The grayscale stability index and the texture stability index are weighted to obtain a weighted stability index.

[0047] Multiple weighted stability indices are sorted, and the connected regions corresponding to the top N weighted stability indices are taken as multiple current feature regions.

[0048] Furthermore, the step of calculating the grayscale stability index and texture stability index of each connected region includes:

[0049] Obtain the area of ​​each of the multiple connected regions corresponding to each of the multiple preset gray values, and substitute the area of ​​each of the multiple connected regions corresponding to the multiple preset gray values ​​into function one to obtain the gray stability index corresponding to each of the multiple connected regions.

[0050] The first function is:

[0051]

[0052] Where Δ(A(r)) represents the grayscale stability index, A(r) represents the area of ​​the current connected region under the preset grayscale value r, ΔI represents the change in adjacent preset grayscale values ​​among multiple preset grayscale values, A(r+ΔI) represents the area of ​​the current connected region under the preset grayscale value r+ΔI, and A(r-ΔI) represents the area of ​​the current connected region under the preset grayscale value r-ΔI.

[0053] Substitute the area of ​​each of the multiple connected regions corresponding to the multiple preset gray values ​​and the gray values ​​of the pixels in the connected regions into function 2 to obtain the texture stability index corresponding to each of the multiple connected regions.

[0054] The second function is:

[0055]

[0056] Among them, S texture (Y,t) represents the stability index of the connected region Y under the preset gray value r, I(p,r) represents the gray value of pixel p under the preset gray value r, I(p,r-ΔI) represents the gray value of pixel p under the preset gray value r-ΔI, |Y| represents the area of ​​the connected region Y, max(I(p,r),I(p,r-ΔI)) represents the maximum value of I(p,r) and I(p,r-ΔI), and p∈Y means that the connected region Y includes pixel p.

[0057] A third aspect of the present invention provides a vehicle safety monitoring device, comprising:

[0058] The first acquisition unit is used to acquire vibration data collected by multiple vibration sensors and determine the vibration direction based on the multiple vibration data; the vibration direction includes the bottom direction, the top direction, or the side direction of the vehicle body.

[0059] The second acquisition unit is used to acquire video monitoring data corresponding to multiple cameras when the vibration direction is the side of the vehicle body; wherein, the video monitoring data includes the N frames before the vibration start time point;

[0060] The extraction unit is used to extract abnormal pixels between the first frame and the last frame of the video surveillance data.

[0061] The first judgment unit is used to determine the current monitoring location as the collision location when the number of abnormal pixels exceeds the first threshold, and send the collision location to the user terminal.

[0062] The second judgment unit is used to calculate the offset distance of the abnormal pixel between the first frame image and the last frame image when the number of abnormal pixels does not exceed the first threshold; wherein, the first frame image refers to the starting frame in the video surveillance data, and the last frame image refers to the ending frame in the video surveillance data.

[0063] The sending unit is used to send the current monitoring location as the collision location to the user terminal when the offset distance is greater than the second threshold and the angle between the offset direction and the current monitoring location is greater than a preset angle.

[0064] A fourth aspect of the present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the vehicle safety monitoring method described in the first aspect.

[0065] A fifth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the vehicle safety monitoring method described in the first aspect.

[0066] The beneficial effects of this invention compared to existing technologies are as follows: By acquiring vibration data collected from multiple vibration sensors, the location of vibrations can be accurately determined, including the bottom, top, and side of the vehicle. Multi-point acquisition and analysis of vibration data significantly improves the accuracy of vibration source location, providing a reliable data foundation for subsequent monitoring and analysis. When the vibration location is determined to be the side of the vehicle, this invention combines video monitoring data from multiple cameras, especially the images of the N frames before the vibration start time, for precise analysis. By extracting abnormal pixels from the video, collision events can be efficiently identified. If the number of abnormal pixels exceeds a predetermined threshold (first threshold), a collision event can be directly identified, and the collision location can be promptly sent to the user terminal, greatly improving the accident response speed. When the number of abnormal pixels does not reach the first threshold, the system further calculates the offset distance of the abnormal pixels between the first and last frames. If the offset distance exceeds a second threshold, a collision event is determined. This mechanism effectively avoids false alarms of minor vibrations or non-collision events, ensuring high sensitivity and low false alarm rate in practical applications. By promptly sending the collision location to the user terminal, real-time monitoring and feedback of the vehicle's safety status are provided. In summary, this invention improves the accuracy, sensitivity, and reaction speed of collision recognition by working in tandem with a vibration sensor and a camera. Attached Figure Description

[0067] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0068] Figure 1 A schematic flowchart of a vehicle safety monitoring method provided by the present invention is shown;

[0069] Figure 2 A schematic diagram of a vehicle safety monitoring device according to an embodiment of the present invention is shown;

[0070] Figure 3 A schematic diagram of a terminal device provided in an embodiment of the present invention is shown. Detailed Implementation

[0071] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0072] This invention provides an automotive safety monitoring system and method to address the technical problem of how to improve the accuracy of vibration location identification in automotive safety monitoring systems.

[0073] First, this invention provides a vehicle safety monitoring system. The vehicle safety monitoring system includes multiple vibration sensors located at different positions on the vehicle body, multiple cameras monitoring different directions, and a processor;

[0074] Vibration sensors are used to collect vibration data;

[0075] Multiple cameras are used to collect video surveillance data from different monitoring locations.

[0076] The processor is used to acquire vibration data collected by multiple vibration sensors and determine the vibration direction based on the multiple vibration data; the vibration direction includes the bottom direction, the top direction, or the side direction of the vehicle body.

[0077] The processor is used to acquire video monitoring data from multiple cameras when the vibration is located at the side of the vehicle body; the video monitoring data includes the N frames preceding the vibration start time.

[0078] The processor is used to extract abnormal pixels between the first and last frames of video surveillance data;

[0079] When the number of abnormal pixels exceeds a first threshold, the processor will use the current monitoring location as the collision location and send the collision location to the user terminal.

[0080] The processor is used to calculate the offset distance of abnormal pixels between the first frame and the last frame when the number of abnormal pixels does not exceed a first threshold; where the first frame refers to the starting frame in the video surveillance data and the last frame refers to the ending frame in the video surveillance data.

[0081] The processor is used to determine the current monitoring location as the collision location when the offset distance is greater than the second threshold and the angle between the offset direction and the current monitoring location is greater than a preset angle, and then sends the collision location to the user terminal.

[0082] Secondly, this invention provides a method for monitoring vehicle safety. Please see below. Figure 1 , Figure 1A schematic flowchart of a vehicle safety monitoring method provided by the present invention is shown. Figure 1 As shown, the vehicle safety monitoring method may include the following steps:

[0083] Step 101: Acquire vibration data collected by multiple vibration sensors, and determine the vibration direction based on the multiple vibration data; the vibration direction includes the bottom direction, the top direction, or the side direction of the vehicle body;

[0084] Vibration sensors can detect the direction and intensity of vibrations, thus determining the approximate direction of the vibration source. Based on this approximate direction, a camera is then used to pinpoint the specific origin of the vibration. Vibration locations include, but are not limited to, the bottom, top, or side of the vehicle body. The specific method for detecting the vibration location is as follows:

[0085] Specifically, step 101 includes steps 1011 to 1013:

[0086] Step 1011: Acquire vibration data collected by multiple vibration sensors; the vibration data includes X-axis vibration data, Y-axis vibration data and Z-axis vibration data;

[0087] X-axis vibration data represents lateral vibration along the vehicle body. Y-axis vibration data represents longitudinal vibration along the vehicle body. Z-axis vibration data represents vertical vibration along the vehicle body.

[0088] Step 1012: If multiple Z-axis vibration data are greater than the third threshold, then determine the vibration orientation as the bottom orientation or the top orientation;

[0089] Z-axis vibration data reflects the vertical vibration of the vehicle. If the Z-axis vibration values ​​detected by multiple vibration sensors exceed a preset third threshold, it means that the vehicle may have experienced a severe vertical impact. Typically, this vibration may be related to a collision or earthquake that occurred on the roof of the vehicle. The combined detection by multiple vibration sensors can avoid errors or noise caused by a single vibration sensor.

[0090] Step 1013: If multiple X-axis vibration data or multiple Y-axis vibration data are greater than the third threshold, then determine the vibration orientation to determine the side orientation of the vehicle body; wherein, the side orientation of the vehicle body includes the front orientation, the rear orientation, the left orientation, and the right orientation.

[0091] X-axis and Y-axis vibration data reflect the vehicle's lateral and longitudinal vibrations, respectively. If the vibration data in these directions exceeds a third threshold, it indicates that the vehicle may have experienced a side impact or collision. For example, the front or rear of the vehicle may have been struck by an external force, or the side of the vehicle may have been impacted. Based on this vibration data, the system determines that the vibration occurred on the side of the vehicle body, thus identifying the vibration location as the side orientation of the vehicle body. The side orientation of the vehicle body includes, but is not limited to, the front, rear, left, and right sides.

[0092] In this embodiment, the location of the vibration is determined by analyzing vibration data acquired from multiple vibration sensors and combining the vibration intensity in different directions. When the vibration data indicates that the vehicle has experienced an impact in the vertical direction, the system determines that the vibration occurred under the vehicle or on the roof; if the vibration comes from the lateral or longitudinal direction of the vehicle body, it is determined to be from the side of the vehicle body. This provides a reference direction for subsequent analysis.

[0093] Step 102: When the vibration location is the side of the vehicle body, acquire video monitoring data corresponding to multiple cameras; wherein, the video monitoring data includes the N frames before the vibration start time point;

[0094] When the vibration location is at the side of the vehicle body, video monitoring data from multiple cameras is acquired. If the vibration location is determined to be at the side of the vehicle body (the side of the vehicle body includes, but is not limited to, the front, left, right, and rear sides), the system will further utilize cameras mounted on the side of the vehicle to acquire relevant video data. The video data captured by the cameras includes the N frames preceding the vibration, where N is a predetermined value. These preceding frames help analyze the vehicle situation before and after the collision, especially the image changes related to the vibration, to accurately locate the vibration location.

[0095] Step 103: Extract abnormal pixels between the first and last frames of the video surveillance data;

[0096] The first and last frames are extracted from the first N frames. Then, by comparing the changes between these two frames, "abnormal pixels" are identified. Abnormal pixels represent image changes caused by an object approaching the vehicle. These anomalies are key clues to the occurrence of a collision.

[0097] The logic for extracting abnormal pixels is as follows:

[0098] Specifically, step 103 includes steps 1031 to 1033:

[0099] Step 1031: Extract the dividing line between the vehicle image area and the non-vehicle image area in the video surveillance data;

[0100] In video images, the vehicle body image area and non-vehicle body image areas are usually clearly distinguishable. The vehicle body image area refers to the area of ​​the vehicle itself, while the non-vehicle body image area includes the background, surrounding objects, or environment. The specific method for extracting the segmentation lines is as follows:

[0101] Specifically, step 1031 includes steps A1 to A4:

[0102] Step A1: Obtain the vehicle body color information input by the user;

[0103] The system needs to obtain the vehicle's body color information. This body color information is crucial data for subsequent image processing because images of the vehicle body area typically exhibit color characteristics that are significantly different from the background (non-body areas). Users can help the system identify the vehicle body area by inputting the vehicle's exterior color (e.g., red, blue, black, etc.).

[0104] Step A2: In the first image region to be identified in the first frame image, extract the current image region corresponding to the vehicle body color information; wherein, the number of pixels in the current image region is greater than a first preset number and the pixels are in a continuous relationship;

[0105] The region is formed by pixels located using the vehicle body color information. The number of pixels in this region must be greater than a preset first threshold; that is, the number of pixels in the region needs to be sufficiently large to be considered a valid part of the vehicle body. Furthermore, the system requires these pixels to be continuous, meaning they need to be connected together to form a region, rather than being scattered pixels. Continuous pixels indicate that this region has a certain spatial consistency, conforming to the characteristics of a vehicle body area.

[0106] Step A3: Use the current image region as the vehicle body image region, and use the remaining region in the first frame image as the non-vehicle body image region; the remaining region refers to the image region in the first frame image other than the current image region;

[0107] After identifying the current image region, the system marks it as the vehicle body image region; that is, this area is considered the vehicle's body. Next, the remaining portion of the first frame image, excluding the current image region, will be marked as the non-vehicle body image region. This region represents the environment, background, or other objects surrounding the vehicle, and is typically a different color from the vehicle body.

[0108] Step A4: Use the boundary line between the vehicle image area and the non-vehicle image area as the dividing line.

[0109] In an image, a clear boundary or critical line forms between the vehicle body image area and the non-vehicle body image area. This critical line is the dividing line, which marks the boundary between the vehicle body and the non-vehicle body parts.

[0110] In this embodiment, by acquiring the vehicle body color information input by the user, the system can accurately identify the image region matching the vehicle body color. This method not only reduces dependence on complex environmental backgrounds but also works stably in various environments, ensuring the accuracy of vehicle body region recognition. In the preset region of the first frame image, the current image region corresponding to the vehicle body color information is extracted, requiring that the number of pixels in this region is greater than a first preset number, and that the pixels are continuous. This requirement effectively filters out regions that match the vehicle body characteristics, avoiding interference from noise or irrelevant parts that may appear in the image, thereby ensuring the accurate positioning of the vehicle body region. Based on the division of the extracted current image region (vehicle body image region) and the remaining region (non-vehicle body image region), the system can clearly distinguish between the vehicle body and non-vehicle body parts, and use the boundary line between them as a dividing line. The extraction of this dividing line ensures a clear boundary between the vehicle body region and the background, providing a clear basis for subsequent image processing and anomaly detection. By setting the condition that the number of pixels is greater than a first preset number and that they are continuous, the system can effectively filter out irregular or discontinuous regions in the image, avoiding misjudgments caused by factors such as changes in lighting and vehicle body reflection. These constraints enhance the system's stability in complex environments, enabling it to adapt to different scenarios and lighting conditions.

[0111] Step 1032: Based on the dividing line between the vehicle image area and the non-vehicle image area, a first image area to be identified is cropped in the area around the dividing line of the first frame image, and a second image area to be identified is cropped in the area around the dividing line of the last frame image.

[0112] After determining the dividing line, the system will extract a certain area from the first frame (the starting frame of the video) and the last frame (the ending frame of the video) based on the position of the dividing line for analysis.

[0113] In the first frame, the region cropped based on the position of the dividing line is called the first image region to be identified. This region includes image information of the vehicle body and the surrounding background. Similarly, in the last frame, a region is also cropped based on the position of the dividing line, called the second image region to be identified. This region corresponds to the first image region to be identified and is used for comparison between the preceding and following frames. The aforementioned surrounding region is defined based on the center point of the dividing line, and the corresponding image region is cropped accordingly.

[0114] Step 1033: Calculate the pixel difference between each pixel in the first image region to be identified and the second image region to be identified, and identify pixels with pixel differences greater than the fourth threshold as abnormal pixels.

[0115] The difference between each pixel in the first and second image regions to be identified is calculated. If the color, brightness, or other characteristic values ​​of a pixel in the two image regions change significantly (i.e., the difference is large), then this pixel is considered an "abnormal pixel". The fourth threshold is a preset difference threshold. When the difference between pixels is greater than this threshold, it indicates that the change of the pixel exceeds the normal fluctuation range.

[0116] In this embodiment, by extracting the dividing line between the vehicle body image area and the non-vehicle body image area, the present invention can effectively distinguish the vehicle body from the surrounding environment, providing a clear boundary for subsequent image analysis. This dividing line extraction technique ensures accurate division of the image area, avoids the influence of interference factors, and thus improves the accuracy of subsequent abnormal pixel identification. Based on the dividing line, the system extracts the first and second image regions to be identified from the areas surrounding the dividing line in the first and last frame images, respectively. This method reduces unnecessary computation by centrally processing pixel data in key areas, enhancing the ability to identify potential anomalies. Furthermore, the comparison between the two key areas further improves the sensitivity of anomaly detection. After calculating the pixel difference between each pixel in the first and second image regions to be identified, the present invention filters abnormal pixels by setting a fourth threshold. Pixels with a pixel difference greater than the fourth threshold are judged as abnormal. This method effectively captures significant differences in image changes and can accurately identify abnormal changes caused by events such as object contact. Setting an appropriate threshold (the fourth threshold) can effectively avoid misjudgments caused by factors such as changes in ambient light and minor changes in object position, thereby ensuring the accuracy and reliability of the system. The application of this threshold gives the system a high degree of anti-interference capability, enabling it to work stably in complex dynamic environments.

[0117] Step 104: When the number of abnormal pixels exceeds the first threshold, the current monitoring location is taken as the collision location, and the collision location is sent to the user terminal.

[0118] Due to the physical principle of "near objects appear larger, far objects appear smaller" (objects closer to the camera occupy a larger area in the image, while objects farther away occupy a smaller area), when the number of abnormal pixels exceeds a first threshold, it can be assumed that an impact occurred near the camera, causing vibration. When the number of abnormal pixels does not exceed the first threshold, the impact object occupies a smaller area in the image, and confusion may occur between the impact object and background objects (e.g., environmental objects passing near the vehicle will also intersect with the vehicle, leading to misjudgment). Therefore, to improve detection accuracy, precise identification of abnormal pixels is necessary.

[0119] Step 105: When the number of abnormal pixels does not exceed the first threshold, calculate the offset distance of the abnormal pixels between the first frame image and the last frame image; wherein, the first frame image refers to the starting frame in the video surveillance data, and the last frame image refers to the ending frame in the video surveillance data.

[0120] Offset distance refers to the displacement of an abnormal pixel from the first frame to the last frame, and is usually related to the motion of an object after a collision (such as a vehicle veering off course). The specific calculation logic for offset distance is as follows:

[0121] Specifically, step 105 includes steps 1051 to 1055:

[0122] Step 1051: If the number of abnormal pixels does not exceed the first threshold, then extract the abnormal region composed of consecutive abnormal pixels; wherein the number of pixels in the abnormal region is greater than the second preset number;

[0123] These anomalous pixels are aggregated into an anomalous region. This anomalous region consists of multiple consecutive anomalous pixels, meaning that these pixels are spatially adjacent or connected. This helps the system identify a more obvious area of ​​change, rather than a single discrete noise point. The number of pixels in the anomalous region must exceed a second preset number. This requirement ensures that the changes in the anomalous region are of sufficient scale or significance, preventing the system from mistaking very small, potentially erroneous areas as valid anomalous regions.

[0124] In other words, further analysis will only proceed when the changes in the abnormal region are sufficiently significant.

[0125] Step 1052: Identify multiple current feature regions in the abnormal region in the tail frame image, and extract the first position of the multiple current feature regions;

[0126] In the last frame (the last frame of the video), the system further extracts multiple current feature regions based on the identified anomaly regions. The first position refers to the coordinates or specific location of each current feature region in the last frame. This position is the basis for subsequent tracking and comparison. The identification method for the current feature region is as follows:

[0127] Specifically, step 1052 includes steps B1 to B5:

[0128] Step B1: Binarize the abnormal regions in the last frame image based on multiple preset grayscale values ​​to obtain multiple binarized images;

[0129] Multiple preset grayscale values ​​are consecutive grayscale values ​​with a fixed difference, used to analyze the changes in the binarized image of anomaly regions under different grayscale values. Binarization converts each pixel in an image into two states (usually black and white) to simplify image analysis. In this step, the system uses multiple preset grayscale values ​​to process the anomaly region in the last frame image. Each preset grayscale value corresponds to a different binarization threshold. The system divides the image into two parts based on these thresholds: one part with grayscale values ​​greater than the threshold, and another part with grayscale values ​​less than the threshold. Through different grayscale values, the system obtains multiple binarized images, each representing a different degree of binarization of the anomaly region, highlighting features within different grayscale ranges.

[0130] Step B2: Identify connected regions in the binarized image;

[0131] For each binarized image, the system performs connected component identification. In a binarized image, black and white pixels may form multiple consecutive regions. The system detects these connected components, which are spatially adjacent regions composed of the same color (black or white).

[0132] A connected component is a region in an image where pixels have the same or similar values, and these pixels are interconnected. Specifically, in a binary image (e.g., a black and white image), white pixels have a value of 1, and black pixels have a value of 0. A connected component consists of all pixels with a value of 1 that are spatially connected, either horizontally, vertically, or diagonally. There are typically two ways to achieve connectivity:

[0133] 4. Connectivity: Only pixels that are horizontally and vertically adjacent are considered connected.

[0134] 8 Connectivity: Allows pixels to be connected in the horizontal, vertical, and diagonal directions. Any adjacent pixels are considered connected.

[0135] Step B3: Calculate the grayscale stability index and texture stability index for each connected region;

[0136] The grayscale stability index indicates the stability of pixel grayscale values ​​within a connected region. A region with a lower grayscale stability index means that the region does not change much in time or space, i.e., the pixel grayscale values ​​are relatively stable.

[0137] Texture stability metrics represent the stability of texture features within a connected region. The main benefit of local texture stability metrics lies in improving feature detection performance, especially against complex backgrounds, by measuring the stability of texture as grayscale changes within an image region.

[0138] Specifically, step B3 includes steps B31 to B32:

[0139] Step B31: Obtain the area of ​​each of the multiple connected regions corresponding to the multiple preset gray values, and substitute the area of ​​each of the multiple connected regions corresponding to the multiple preset gray values ​​into function one to obtain the gray stability index corresponding to each of the multiple connected regions.

[0140] The first function is:

[0141]

[0142] Where Δ(A(r)) represents the grayscale stability index, A(r) represents the area of ​​the current connected region under the preset grayscale value r, ΔI represents the change in adjacent preset grayscale values ​​among multiple preset grayscale values, A(r+ΔI) represents the area of ​​the current connected region under the preset grayscale value r+ΔI, and A(r-ΔI) represents the area of ​​the current connected region under the preset grayscale value r-ΔI.

[0143] Function 1 calculates the area under a preset grayscale value r and small variations above and below that preset grayscale value (i.e., r+ΔI and r-ΔI). This variation indicates whether the region remains stable when the grayscale level changes.

[0144] This represents the rate of change in area. If the rate is small, it means that the area remains relatively stable under changes in gray level. The smaller, The larger the area, the higher the stability of the region. If the area of ​​a region changes very little across multiple gray levels, it means that the shape of the region remains stable under different thresholds, and such a region is considered "stable". If the area of ​​a region changes significantly, it means that it is very sensitive to changes in gray levels, and then the region is considered unstable.

[0145] The core idea of ​​Function 1 is to measure the "stability" of a region by comparing the area changes of the region under different grayscale thresholds. Regions with high stability are not easily deformed when the grayscale level changes, and can be effectively used as image features for subsequent processing.

[0146] Step B32: Substitute the area of ​​each of the multiple connected regions corresponding to the multiple preset gray values ​​and the gray values ​​of the pixels in the connected regions into function 2 to obtain the texture stability index corresponding to each of the multiple connected regions.

[0147] The second function is:

[0148]

[0149] Among them, S texture (Y,t) represents the stability index of the connected region Y under the preset gray value r, I(p,r) represents the gray value of pixel p under the preset gray value r, I(p,r-ΔI) represents the gray value of pixel p under the preset gray value r-ΔI, |Y| represents the area of ​​the connected region Y, max(I(p,r),I(p,r-ΔI)) represents the maximum value of I(p,r) and I(p,r-ΔI), and p∈Y means that the connected region Y includes pixel p.

[0150] For each pixel p in region Y, calculate its grayscale change magnitude between two consecutive grayscale levels (or thresholds): |I(p,r) - I(p,r - ΔI)|. This difference reflects the degree of change of the pixel at different grayscale levels. However, simple grayscale differences do not reflect the stability of the texture. Therefore, a normalization process is used to normalize the change magnitude of each pixel to its relative change at the current and previous grayscale levels: This standardization method ensures that even if the grayscale value itself is large or small, the calculated variation can be compared on a uniform scale.

[0151] Texture stability reflects the impact of grayscale changes on the image texture structure. To measure texture stability, an inverse metric is used: This inverse metric ensures that smaller grayscale changes (i.e., smaller texture changes) result in higher stability values. If a region shows little variation across different grayscale levels, its texture features are relatively stable, and the corresponding stability metric will be higher.

[0152] Finally, the average value of all pixels within region Y is used to obtain a measure of local texture stability for the entire region. In this way, texture stability not only considers the stability of individual pixels, but also reflects the stability of the entire region by averaging.

[0153] The core of Function 2 is the standardization of grayscale changes, aiming to eliminate the interference of grayscale differences on texture evaluation and thus more accurately measure texture stability. By averaging the texture stability of all pixels within a region, a holistic measure reflecting the smoothness of texture feature changes in the region can be obtained. Standardized change amplitude and inverse metrics are used to ensure that smoother texture changes receive higher stability scores, while regions with drastic changes are deemed unstable. This texture stability measure captures the continuity of texture features within a region as grayscale changes, thus more accurately identifying true feature regions in complex textured backgrounds, and is less susceptible to interference from background noise or texture variations.

[0154] As an optional embodiment of the present invention, the weighted processing of grayscale stability index and texture stability index can be used as the final stability index, or either grayscale stability index or texture stability index can be used as the final stability index.

[0155] Step B4: Perform weighted processing on the grayscale stability index and texture stability index to obtain a weighted stability index;

[0156] The grayscale stability index and texture stability index are weighted, with different weights assigned to each index based on different situations or priorities. The purpose of this step is to adjust the importance of the two indices according to actual needs, so as to finally obtain a weighted stability index that comprehensively reflects the stability of the region.

[0157] Step B5: Sort the multiple weighted stability indices and take the connected regions corresponding to the top N weighted stability indices as the multiple current feature regions.

[0158] After calculating the weighted stability indices for all connected regions, the system sorts these indices. The sorted indices represent the stability level of each connected region. The system selects the top N connected regions, which have the strongest weighted stability and represent the most stable and significant feature regions in the last frame image. These N connected regions are selected as the current feature regions and will be the focus of subsequent analysis for feature region tracking.

[0159] In this embodiment, by binarizing the abnormal regions in the last frame image based on multiple preset grayscale values, multiple binarized images are obtained. The system can then analyze image changes in depth from different grayscale levels. This multi-dimensional grayscale analysis helps capture subtle changes in the image from different perspectives. After identifying connected regions in the binarized images, the system can treat interconnected pixel blocks in the image as independent regions for processing. The identification of connected regions enables the system to extract meaningful feature regions from the image, avoiding misjudgments caused by noise or small objects. For each connected region, the system calculates its grayscale stability index and texture stability index. This index calculation method can measure the stability of grayscale changes and texture structure within the region, thereby reflecting whether the region has strong feature representation. Grayscale stability mainly focuses on the uniformity of region brightness, while texture stability reflects the persistence of region shape and structure. These indices help to evaluate abnormal regions in the image more deeply. After weighting the grayscale stability index and texture stability index, a comprehensive weighted stability index is obtained. In this way, the system can comprehensively consider the brightness balance and texture variation patterns of the image, and more comprehensively evaluate the importance of regional features. By ranking multiple weighted stability indices, the system can select the most characteristic regions from high to low weighted stability, and take the connected regions corresponding to the top N weighted stability indices as the current feature regions. In this way, the system can ensure that the regions that best represent abnormal behavior or object movement are selected, reducing the interference of irrelevant or insignificantly changing regions. This invention, through a series of processing methods such as multi-grayscale binarization, connected region recognition, calculation and weighted ranking of grayscale stability and texture stability indices, can accurately identify multiple current feature regions in the last frame image.

[0160] Step 1053: In the video surveillance data, feature regions are tracked sequentially from the last frame image to the first frame image to obtain the second position corresponding to each of the current feature regions in the first frame image;

[0161] The system will sequentially trace the current feature regions, starting from the last frame and moving forward (towards the first frame). The purpose of this feature region tracing is to pinpoint their location in the first frame, thereby understanding the movement trajectory of these anomalous regions in the video. Through feature tracing, the system can ultimately determine the second position of each current feature region in the first frame, i.e., their specific location within the first frame. Feature region tracing can employ feature tracing algorithms such as optical flow; the feature tracing used in this embodiment is existing technology and will not be elaborated upon further.

[0162] Step 1054: Calculate the positional movement distance between the first and second positions corresponding to each of the multiple current feature regions, and calculate the average value among the multiple positional movement distances;

[0163] The system has obtained the first position of each current feature region in the last frame and the second position in the first frame. Using these two positions, the displacement distance of each feature region can be calculated, i.e., the distance these regions have moved from the last frame to the first frame. Then, the system calculates the average displacement distance of all these feature regions. This average value represents the combined offset of multiple feature regions, indicating the average displacement of the entire anomalous region between image frames.

[0164] Step 1055: Use the average value as the offset distance.

[0165] Offset distance can be used to analyze the direction and magnitude of an object's (e.g., a vehicle) movement in a video, serving as the basis for subsequent motion analysis.

[0166] In this embodiment, when the number of abnormal pixels does not exceed a first threshold, the present invention first extracts an abnormal region composed of consecutive abnormal pixels to ensure that only regions with significant changes are considered. By setting the number of pixels in the abnormal region to be greater than a second preset number, low-intensity, scattered anomalies are effectively filtered out, thereby reducing the possibility of false alarms and improving the sensitivity and accuracy of the system. By identifying multiple current feature regions in the abnormal region in the last frame image and further extracting the first position of these feature regions, the system can accurately determine the initial position of each feature region. This process lays the foundation for subsequent feature region tracking and helps to capture the motion trajectory of objects in the image. In video surveillance data, the system sequentially tracks feature regions from the last frame image to the first frame image, obtaining the second position corresponding to multiple current feature regions in the first frame image. By calculating the movement distance of multiple feature regions between the last frame image and the first frame image, and further calculating the average of these movement distances, the system can accurately measure the displacement of the abnormal region, thereby obtaining the offset distance. This offset distance provides an important basis for determining whether an object has collided or exhibited other abnormal behavior. The calculation of offset distance provides a quantitative analysis of object motion. If the object's displacement in the image exceeds a set threshold (such as a second threshold), it can be further identified as a possible collision or abnormal event. This process improves the accuracy of abnormal event identification and reduces the interference of environmental factors (such as slight vibrations or non-collision situations) on the monitoring system. This technical solution can accurately capture object displacement even when the number of abnormal pixels is small, thereby enhancing the judgment capability and response speed of the vehicle safety monitoring system.

[0167] Step 106: When the offset distance is greater than the second threshold and the angle between the offset direction and the current monitoring position is greater than the preset angle, the current monitoring position is taken as the collision position and the collision position is sent to the user terminal.

[0168] Since the direction of motion of environmental objects passing by the vehicle is almost parallel to the monitoring position (side of the vehicle), while the direction of motion of the colliding object is usually at a certain angle to the monitoring position and has a certain displacement, when the offset distance is greater than the second threshold and the angle between the offset direction and the current monitoring position is greater than the preset angle, it can be considered that a colliding object has occurred (i.e., a collision event has occurred). Then, the current monitoring position is taken as the collision position and the collision position is sent to the user terminal for timely response.

[0169] In this embodiment, by acquiring vibration data collected from multiple vibration sensors, the location of the vibration can be accurately determined, including the bottom, top, and side of the vehicle. Multi-point acquisition and analysis of vibration data significantly improves the accuracy of vibration source location, providing a reliable data foundation for subsequent monitoring and analysis. When the vibration location is determined to be the side of the vehicle, this invention combines video monitoring data from multiple cameras, especially the images of the N frames before the vibration start time, for precise analysis. By extracting abnormal pixels from the video, collision events can be efficiently identified. If the number of abnormal pixels exceeds a predetermined threshold (first threshold), a collision event can be directly identified, and the collision location can be promptly sent to the user terminal, greatly improving the accident response speed. When the number of abnormal pixels does not reach the first threshold, the system further calculates the offset distance of the abnormal pixels between the first and last frames. If the offset distance exceeds a second threshold, a collision event is determined. This mechanism effectively avoids false alarms for minor vibrations or non-collision events, ensuring high sensitivity and low false alarm rate in practical applications. By promptly sending the collision location to the user terminal, real-time monitoring and feedback of the vehicle's safety status are provided. In summary, this invention improves the accuracy, sensitivity, and reaction speed of collision recognition by working in tandem with a vibration sensor and a camera.

[0170] like Figure 2 This invention provides a vehicle safety monitoring device; please refer to [link / reference]. Figure 2 , Figure 2 A schematic diagram of a vehicle safety monitoring device provided by the present invention is shown, such as... Figure 2 The vehicle safety monitoring device shown includes:

[0171] The first acquisition unit 21 is used to acquire vibration data collected by multiple vibration sensors and determine the vibration direction based on the multiple vibration data; the vibration direction includes the bottom direction, the top direction, or the side direction of the vehicle body.

[0172] The second acquisition unit 22 is used to acquire video monitoring data corresponding to multiple cameras when the vibration direction is the side of the vehicle body; wherein, the video monitoring data includes the N frames before the vibration start time point;

[0173] Extraction unit 23 is used to extract abnormal pixels between the first frame and the last frame in the video surveillance data;

[0174] The first judgment unit 24 is used to determine the current monitoring location as the collision location when the number of abnormal pixels exceeds the first threshold, and send the collision location to the user terminal.

[0175] The second judgment unit 25 is used to calculate the offset distance between the abnormal pixel and the first frame image and the last frame image when the number of abnormal pixels does not exceed the first threshold; wherein, the first frame image refers to the starting frame in the video surveillance data, and the last frame image refers to the ending frame in the video surveillance data.

[0176] The sending unit 26 is used to send the current monitoring position as the collision position to the user terminal when the offset distance is greater than the second threshold and the angle between the offset direction and the current monitoring position is greater than a preset angle.

[0177] This invention provides a vehicle safety monitoring device that, by acquiring vibration data from multiple vibration sensors, can accurately determine the location of vibrations, including the bottom, top, and sides of the vehicle. Multi-point acquisition and analysis of vibration data significantly improves the accuracy of vibration source location, providing a reliable data foundation for subsequent monitoring and analysis. When the vibration location is determined to be the side of the vehicle, this invention combines video monitoring data from multiple cameras, especially the images of the N frames preceding the vibration start time, for precise analysis. By extracting abnormal pixels from the video, collision events can be efficiently identified. If the number of abnormal pixels exceeds a predetermined threshold (first threshold), a collision event can be directly identified, and the collision location can be promptly sent to the user terminal, greatly improving the accident response speed. When the number of abnormal pixels does not reach the first threshold, the system further calculates the offset distance of the abnormal pixels between the first and last frames. If the offset distance exceeds a second threshold, a collision event is determined. This mechanism effectively avoids false alarms for minor vibrations or non-collision events, ensuring high sensitivity and a low false alarm rate in practical applications. By promptly sending the collision location to the user terminal, real-time monitoring and feedback of the vehicle's safety status are provided. In summary, this invention improves the accuracy, sensitivity, and reaction speed of collision recognition by working in tandem with a vibration sensor and a camera.

[0178] Figure 3 This is a schematic diagram of a terminal device provided in an embodiment of the present invention. Figure 3As shown, a terminal device 3 in this embodiment includes: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30, such as a car safety monitoring program. When the processor 30 executes the computer program 32, it implements the steps described in the various embodiments of the car safety monitoring method above, for example... Figure 1 Steps 101 to 106 are shown. Alternatively, when the processor 30 executes the computer program 32, it implements the functions of each unit in the above-described device embodiments, for example... Figure 2 The function of the unit shown.

[0179] For example, the computer program 32 can be divided into one or more units, which are stored in the memory 31 and executed by the processor 30 to complete the present invention. The one or more units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 32 in the terminal device 3. For example, the specific functions of each unit of the computer program 32 can be divided as follows:

[0180] The first acquisition unit is used to acquire vibration data collected by multiple vibration sensors and determine the vibration direction based on the multiple vibration data; the vibration direction includes the bottom direction, the top direction, or the side direction of the vehicle body.

[0181] The second acquisition unit is used to acquire video monitoring data corresponding to multiple cameras when the vibration direction is the side of the vehicle body; wherein, the video monitoring data includes the N frames before the vibration start time point;

[0182] The extraction unit is used to extract abnormal pixels between the first frame and the last frame of the video surveillance data.

[0183] The first judgment unit is used to determine the current monitoring location as the collision location when the number of abnormal pixels exceeds the first threshold, and send the collision location to the user terminal.

[0184] The second judgment unit is used to calculate the offset distance of the abnormal pixel between the first frame image and the last frame image when the number of abnormal pixels does not exceed the first threshold; wherein, the first frame image refers to the starting frame in the video surveillance data, and the last frame image refers to the ending frame in the video surveillance data.

[0185] The sending unit is used to send the current monitoring location as the collision location to the user terminal when the offset distance is greater than the second threshold and the angle between the offset direction and the current monitoring location is greater than a preset angle.

[0186] The terminal device includes, but is not limited to, a processor 30 and a memory 31. Those skilled in the art will understand that... Figure 3 This is merely an example of a terminal device 3 and does not constitute a limitation on a terminal device 3. It may include more or fewer components than shown, or combine certain components, or different components. For example, the terminal device may also include input / output devices, network access devices, buses, etc.

[0187] The processor 30 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0188] The memory 31 can be an internal storage unit of the terminal device 3, such as a hard disk or memory of the terminal device 3. The memory 31 can also be an external storage device of the terminal device 3, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the terminal device 3. Furthermore, the memory 31 can include both internal and external storage units of the terminal device 3. The memory 31 is used to store the computer program and other programs and data required by the roaming control device. The memory 31 can also be used to temporarily store data that has been output or will be output.

[0189] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0190] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0191] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0192] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0193] This invention provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the above-described method embodiments.

[0194] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can at least include: a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, such as a USB flash drive, a portable hard drive, a magnetic disk, or an optical disk.

[0195] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0196] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0197] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0198] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units.

[0199] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0200] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0201] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once [the described condition or event] is detected," or "in response to detection of [the described condition or event]."

[0202] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0203] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0204] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A vehicle safety monitoring system, characterized in that, The vehicle safety monitoring system includes multiple vibration sensors located at different positions on the vehicle body, multiple cameras monitoring different directions, and a processor. The vibration sensor is used to collect vibration data; The multiple cameras in different monitoring positions are used to collect video monitoring data from different monitoring positions. The processor is used to acquire vibration data collected by multiple vibration sensors and determine the vibration orientation based on the multiple vibration data. The vibration location includes the bottom, top, or side of the vehicle body; The processor is used to acquire video monitoring data corresponding to multiple cameras when the vibration location is the side of the vehicle body; wherein, the video monitoring data includes the N frames before the vibration start time point; The processor is used to extract abnormal pixels between the first and last frames of the video surveillance data. When the number of abnormal pixels exceeds a first threshold, the processor will use the current monitoring location as the collision location and send the collision location to the user terminal. The processor is used to calculate the offset distance of the abnormal pixel between the first frame image and the last frame image when the number of abnormal pixels does not exceed a first threshold; wherein, the first frame image refers to the starting frame in the video surveillance data, and the last frame image refers to the ending frame in the video surveillance data. The processor is used to take the current monitoring location as the collision location and send the collision location to the user terminal when the offset distance is greater than the second threshold and the angle between the offset direction and the current monitoring location is greater than a preset angle.

2. A vehicle safety monitoring method, characterized in that, The vehicle safety monitoring method is applied to a vehicle safety monitoring system, and the vehicle safety monitoring method includes: The vibration data collected by multiple vibration sensors is acquired, and the vibration direction is determined based on the multiple vibration data; the vibration direction includes the bottom direction, the top direction, or the side direction of the vehicle body. When the vibration location is the side of the vehicle body, video monitoring data corresponding to multiple cameras is acquired; wherein, the video monitoring data includes the N frames before the vibration start time point; Extract abnormal pixels between the first and last frames of the video surveillance data; When the number of abnormal pixels exceeds a first threshold, the current monitoring location is taken as the collision location and sent to the user terminal. When the number of abnormal pixels does not exceed a first threshold, the offset distance of the abnormal pixels between the first frame image and the last frame image is calculated; wherein, the first frame image refers to the starting frame in the video surveillance data, and the last frame image refers to the ending frame in the video surveillance data; When the offset distance is greater than the second threshold and the angle between the offset direction and the current monitoring position is greater than the preset angle, the current monitoring position is taken as the collision position and the collision position is sent to the user terminal.

3. The vehicle safety monitoring method as described in claim 2, characterized in that, The step of acquiring vibration data collected by multiple vibration sensors and determining the vibration location based on the multiple vibration data includes: Acquire vibration data collected by multiple vibration sensors; the vibration data includes X-axis vibration data, Y-axis vibration data and Z-axis vibration data; If multiple Z-axis vibration data are greater than the third threshold, the vibration orientation is determined to be either the bottom orientation or the top orientation. If multiple X-axis vibration data or multiple Y-axis vibration data are greater than a third threshold, the vibration orientation is determined to determine the side orientation of the vehicle body; wherein, the side orientation of the vehicle body includes the front orientation, rear orientation, left orientation, and right orientation.

4. The vehicle safety monitoring method as described in claim 2, characterized in that, The step of extracting abnormal pixels between the first and last frames of the video surveillance data includes: Extract the dividing line between the vehicle image area and the non-vehicle image area in the video surveillance data; Based on the dividing line between the vehicle image area and the non-vehicle image area, a first image area to be identified is cropped around the dividing line of the first frame image, and a second image area to be identified is cropped around the dividing line of the last frame image. Calculate the pixel difference between each pixel in the first image region to be identified and the second image region to be identified, and identify pixels with a pixel difference greater than a fourth threshold as abnormal pixels.

5. The vehicle safety monitoring method as described in claim 4, characterized in that, The step of extracting the dividing line between the vehicle image region and the non-vehicle image region in the video surveillance data includes: Obtain the vehicle body color information input by the user; In the first image region to be identified in the first frame image, the current image region corresponding to the vehicle body color information is extracted; wherein, the number of pixels in the current image region is greater than a first preset number and the pixels are in a continuous relationship; The current image region is taken as the vehicle body image region, and the remaining region in the first frame image is taken as the non-vehicle body image region; the remaining region refers to the image region in the first frame image other than the current image region. The boundary line between the vehicle image area and the non-vehicle image area is used as the dividing line.

6. The vehicle safety monitoring method as described in claim 2, characterized in that, The step of calculating the offset distance of the abnormal pixels between the first frame and the last frame when the number of abnormal pixels does not exceed the first threshold includes: If the number of abnormal pixels does not exceed the first threshold, then an abnormal region composed of consecutive abnormal pixels is extracted; wherein the number of pixels in the abnormal region is greater than the second preset number. In the last frame image, identify multiple current feature regions in the abnormal region and extract the first position of the multiple current feature regions; In the video surveillance data, feature regions are tracked sequentially from the last frame image to the first frame image to obtain the second position corresponding to each of the current feature regions in the first frame image. Calculate the positional movement distance between the first and second positions corresponding to each of the multiple current feature regions, and calculate the average value among the multiple positional movement distances; The average value is used as the offset distance.

7. The vehicle safety monitoring method as described in claim 6, characterized in that, The step of identifying multiple current feature regions in the abnormal region in the tail frame image and extracting the first position of the multiple current feature regions includes: The abnormal regions in the last frame image are binarized based on multiple preset grayscale values ​​to obtain multiple binarized images; Identify connected regions in a binary image; Calculate the grayscale stability index and texture stability index for each connected region; The grayscale stability index and the texture stability index are weighted to obtain a weighted stability index. Multiple weighted stability indices are sorted, and the connected regions corresponding to the top N weighted stability indices are taken as multiple current feature regions.

8. The vehicle safety monitoring method as described in claim 7, characterized in that, The steps for calculating the grayscale stability index and texture stability index of each connected region include: Obtain the area of ​​each of the multiple connected regions corresponding to each of the multiple preset gray values, and substitute the area of ​​each of the multiple connected regions corresponding to the multiple preset gray values ​​into function one to obtain the gray stability index corresponding to each of the multiple connected regions. The first function is: in, This represents the grayscale stability index. This represents the area of ​​the currently connected region at a preset grayscale value r. This represents the amount of change between adjacent preset grayscale values ​​among multiple preset grayscale values. This indicates that at the preset grayscale value r The area of ​​the current connected region. This indicates that at the preset grayscale value r The area of ​​the current connected region; Substitute the area of ​​each of the multiple connected regions corresponding to the multiple preset gray values ​​and the gray values ​​of the pixels in the connected regions into function 2 to obtain the texture stability index corresponding to each of the multiple connected regions. The second function is: in, Represents connected regions Stability index under a preset grayscale value r Represents pixels The grayscale value at the preset grayscale value r Represents pixels At the preset grayscale value r The grayscale value below, Represents connected regions area, express and The maximum value in, Represents connected regions Including pixels .

9. A terminal device, characterized in that, The terminal device includes: a memory, a processor, and a vehicle safety monitoring program stored in the memory and executable on the processor, the vehicle safety monitoring program being configured to implement the steps of the vehicle safety monitoring method as described in any one of claims 2 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps in the vehicle safety monitoring method as described in any one of claims 2 to 8.

Images