Automobile safety monitoring system and method

Abstract

The invention is suitable for the technical field of image processing, and provides an automobile safety monitoring system and method.The automobile safety monitoring method comprises the steps that vibration data collected by a plurality of vibration sensors are obtained, and the vibration direction is determined according to the multiple pieces of vibration data; when the vibration direction is the side direction of the vehicle body, video monitoring data corresponding to a plurality of cameras are obtained; extracting abnormal pixel points between a first frame image and a tail frame image in the video monitoring data; when the number of the abnormal pixel points exceeds a first threshold value, taking the current monitoring direction as a collision position; when the number of the abnormal pixel points does not exceed a first threshold value, the offset distance of the abnormal pixel points between the first frame image and the tail frame image is calculated; when the offset distance is larger than a second threshold value and the included angle between the offset direction and the current monitoring direction is larger than a preset angle, the current monitoring direction serves as the collision position. According to the invention, through cooperative work of the vibration sensor and the camera, the accuracy of collision identification is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and particularly relates to an automotive safety monitoring system and method. Background Art

[0002] With the rapid development of the automotive industry, automotive safety has become the focus of increasing attention from consumers. Traditional automotive safety monitoring systems mainly rely on devices such as cameras, radars, and ultrasonic sensors to detect and identify the environment around the vehicle and potential hazards. However, existing safety monitoring technologies often have problems such as untimely response and insufficient accuracy when dealing with emergencies such as the vehicle being impacted or vibrated by the outside world.

[0003] Specifically, when a traditional vehicle safety monitoring system detects external vibrations of the vehicle, it usually can only rely on a single sensor (such as a vibration sensor) to determine the location where the vibration occurs, and often cannot provide sufficient accuracy to timely determine the impact point or collision location. In addition, existing systems mostly rely on simple vibration data analysis, but in actual situations, the vibration source may not be obvious, and the vibration data may contain a large amount of noise, resulting in inaccurate judgment of the vibration direction by the system, thereby affecting the accuracy of collision detection.

[0004] Against this background, using multiple sensors (such as vibration sensors and cameras) to work together can more accurately determine the direction of vibration, and use video monitoring data to further analyze the nature of the vibration and potential collision situations, becoming a new method to improve automotive safety. This method can not only improve the accuracy of vibration direction judgment, but also be further verified through video monitoring data when the vibration occurs, so as to determine whether a collision event has occurred and send a warning to the user in a timely manner, achieving a more reliable safety protection effect.

[0005] Therefore, how to utilize the synergistic effect of multiple sensors such as vibration sensors and cameras to improve the accurate identification ability of the automotive safety monitoring system for the vibration direction has become an urgent problem to be solved in the field of automotive safety monitoring technology. Summary of the Invention

[0006] In view of this, embodiments of the present invention provide an automotive safety monitoring system and method to solve the technical problem of how to improve the accurate identification ability of the automotive safety monitoring system for the vibration direction.

[0007] The first aspect of the embodiments of the present invention provides an automotive safety monitoring system, where the automotive safety monitoring system includes multiple vibration sensors located at different body positions, cameras for multiple monitoring directions, and a processor;

[0008] The vibration sensor is used to collect vibration data;

[0009] The cameras at multiple monitoring orientations are used to collect video monitoring data at different monitoring orientations respectively;

[0010] The processor is used to obtain the vibration data collected by multiple vibration sensors, and determine the vibration orientation according to the multiple vibration data; the vibration orientation includes a bottom orientation, a top orientation or a side orientation of the vehicle body;

[0011] The processor is used to obtain the video monitoring data corresponding to multiple cameras when the vibration orientation is the side orientation of the vehicle body; wherein, the video monitoring data includes the first N frames before the vibration start time point;

[0012] The processor is used to extract the abnormal pixel points between the first frame image and the last frame image in the video monitoring data;

[0013] When the number of the abnormal pixel points exceeds the first threshold, the processor is used to take the current monitoring orientation as the collision position and send the collision position to the user terminal;

[0014] When the number of the abnormal pixel points does not exceed the first threshold, the processor is used to calculate the offset distance of the abnormal pixel points between the first frame image and the last frame image; wherein, the first frame image refers to the starting frame in the video monitoring data, and the last frame image refers to the ending frame in the video monitoring data;

[0015] When the offset distance is greater than the second threshold and the included angle between the offset direction and the current monitoring orientation is greater than the preset angle, the processor is used to take the current monitoring orientation as the collision position and send the collision position to the user terminal.

[0016] A second aspect of the embodiments of the present invention provides an automobile safety monitoring method, which is applied to an automobile safety monitoring system, and the automobile safety monitoring method includes:

[0017] Obtain the vibration data collected by multiple vibration sensors, and determine the vibration orientation according to the multiple vibration data; the vibration orientation includes a bottom orientation, a top orientation or a side orientation of the vehicle body;

[0018] When the vibration orientation is the side orientation of the vehicle body, obtain the video monitoring data corresponding to multiple cameras; wherein, the video monitoring data includes the first N frames before the vibration start time point;

[0019] Extract the abnormal pixel points between the first frame image and the last frame image in the video monitoring data;

[0020] When the number of the abnormal pixel points exceeds the first threshold, take the current monitoring orientation as the collision position and send the collision position to the user terminal;

[0021] When the number of the abnormal pixel points does not exceed a first threshold, calculate an offset distance of the abnormal pixel points between a first frame image and a last frame image; wherein, the first frame image refers to a starting frame in the video surveillance data, and the last frame image refers to an ending frame in the video surveillance data;

[0022] When the offset distance is greater than a second threshold and an included angle between an offset direction and a current surveillance orientation is greater than a preset angle, use the current surveillance orientation as a collision position, and send the collision position to a user terminal.

[0023] Further, the step of obtaining vibration data collected by a plurality of the vibration sensors and determining a vibration orientation according to the plurality of vibration data includes:

[0024] Obtain vibration data collected by a plurality of the vibration sensors; the vibration data includes X-axis vibration data, Y-axis vibration data, and Z-axis vibration data;

[0025] If a plurality of the Z-axis vibration data is greater than a third threshold, determine that the vibration orientation is a bottom orientation or a top orientation;

[0026] If a plurality of the X-axis vibration data or a plurality of the Y-axis vibration data is greater than the third threshold, determine that the vibration orientation is a side orientation of the vehicle body; wherein, the side orientation of the vehicle body includes a front side orientation, a rear side orientation, a left side orientation, and a right side orientation.

[0027] Further, the step of extracting abnormal pixel points between a first frame image and a last frame image in the video surveillance data includes:

[0028] Extract a dividing line between a vehicle body image area and a non-vehicle body image area in the video surveillance data;

[0029] Based on the dividing line between the vehicle body image area and the non-vehicle body image area, intercept a first image area to be recognized in an area around the dividing line of the first frame image, and intercept a second image area to be recognized in an area around the dividing line of the last frame image;

[0030] Calculate a pixel difference between each pixel point in the first image area to be recognized and the second image area to be recognized, and use a pixel point with a pixel difference greater than a fourth threshold as an abnormal pixel point.

[0031] Further, the step of extracting the dividing line between the vehicle body image area and the non-vehicle body image area in the video surveillance data includes:

[0032] Obtain vehicle body color information input by a user;

[0033] In the first image region to be recognized of the first frame image, extract the current image region corresponding to the body color information; wherein, the number of pixels in the current image region is greater than a first preset number and the pixel points are in a continuous relationship;

[0034] Take the current image region as the body image region, and take the remaining region in the first frame image as the non-body image region; the remaining region refers to the image region in the first frame image except the current image region;

[0035] Take the critical line between the body image region and the non-body image region as the segmentation line.

[0036] Further, the step of calculating the offset distance of the abnormal pixel points between the first frame image and the last frame image when the number of the abnormal pixel points does not exceed a first threshold includes:

[0037] If the number of the abnormal pixel points does not exceed the first threshold, extract the abnormal region composed of continuous abnormal pixel points; wherein, the number of pixel points in the abnormal region is greater than a second preset number;

[0038] Identify a plurality of current feature regions in the abnormal region in the last frame image, and extract the first positions of the plurality of current feature regions;

[0039] Perform feature region tracking in the video surveillance data from the last frame image to the first frame image in sequence to obtain the second positions corresponding to the plurality of current feature regions in the first frame image;

[0040] Calculate the position movement distances between the first positions and the second positions corresponding to the plurality of current feature regions respectively, and calculate the average value between the plurality of position movement distances;

[0041] Take the average value as the offset distance.

[0042] Further, the step of identifying a plurality of current feature regions in the abnormal region in the last frame image and extracting the first positions of the plurality of current feature regions includes:

[0043] Perform binarization processing on the abnormal region in the last frame image respectively based on a plurality of preset gray values to obtain a plurality of binarized images;

[0044] Identify the connected regions in the binarized images;

[0045] Calculate the gray stability index and the texture stability index of each connected region;

[0046] Perform weighted processing on the gray stability index and the texture stability index to obtain a weighted stability index;

[0047] Sort multiple weighted stability indicators, and use the connected regions corresponding to the top N weighted stability indicators as the multiple current feature regions.

[0048] Further, the step of calculating the gray stability index and texture stability index of each connected region includes:

[0049] Obtain the areas of multiple connected regions corresponding to multiple preset gray values respectively, and substitute the areas of the multiple connected regions corresponding to the multiple preset gray values into Function 1 to obtain the gray stability indicators corresponding to the multiple connected regions respectively;

[0050] Function 1 is:

[0051]

[0052] Where, Δ(A(r)) represents the gray stability index, A(r) represents the area of the current connected region at the preset gray value r, ΔI represents the change amount between adjacent preset gray values among the multiple preset gray values, A(r + ΔI) represents the area of the current connected region at the preset gray value r + ΔI, and A(r - ΔI) represents the area of the current connected region at the preset gray value r - ΔI;

[0053] Substitute the areas of the multiple connected regions corresponding to the multiple preset gray values and the gray values of the pixel points in the connected regions into Function 2 to obtain the texture stability indicators corresponding to the multiple connected regions respectively;

[0054] Function 2 is:

[0055]

[0056] Where, S texture (Y, t) represents the stability index of the connected region Y at the preset gray value r, I(p, r) represents the gray value of the pixel point p at the preset gray value r, I(p, r - ΔI) represents the gray value of the pixel point p at the preset gray value r - ΔI, |Y| represents the area of the connected region Y, max(I(p, r), I(p, r - ΔI)) represents the maximum value of I(p, r) and I(p, r - ΔI), and p ∈ Y means the connected region Y includes the pixel point p.

[0057] The third aspect of the embodiments of the present invention provides an automotive safety monitoring device, including:

[0058] A first acquisition unit, configured to acquire vibration data collected by multiple vibration sensors, and determine the vibration direction according to the multiple vibration data; the vibration direction includes a bottom direction, a top direction or a vehicle body side direction;

[0059] A second acquisition unit, configured to acquire video monitoring data corresponding to multiple cameras when the vibration orientation is the side orientation of the vehicle body; wherein, the video monitoring data includes the first N frames before the vibration start time point;

[0060] An extraction unit, configured to extract abnormal pixel points between the first frame image and the last frame image in the video monitoring data;

[0061] A first determination unit, configured to use the current monitoring orientation as the collision position and send the collision position to the user terminal when the number of the abnormal pixel points exceeds a first threshold;

[0062] A second determination unit, configured to calculate an offset distance of the abnormal pixel points between the first frame image and the last frame image when the number of the abnormal pixel points does not exceed the first threshold; wherein, the first frame image refers to the starting frame in the video monitoring data, and the last frame image refers to the ending frame in the video monitoring data;

[0063] A sending unit, configured to use the current monitoring orientation as the collision position and send the collision position to the user terminal when the offset distance is greater than a second threshold and the included angle between the offset direction and the current monitoring orientation is greater than a preset angle.

[0064] A fourth aspect of the embodiments of the present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the vehicle safety monitoring method described in the first aspect when executing the computer program.

[0065] A fifth aspect of the embodiments of the present invention provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and the computer program implements the steps of the vehicle safety monitoring method described in the first aspect when being executed by a processor.

[0066] The beneficial effects of the embodiments of the present invention compared with the prior art are as follows: By acquiring the vibration data collected by multiple vibration sensors, the azimuth where the vibration occurs can be accurately determined, including the bottom, top, and side positions of the vehicle body. The multi-point collection and analysis of vibration data greatly improve the positioning accuracy of the vibration source, providing a reliable data basis for subsequent monitoring and analysis. When the vibration azimuth is determined to be the side of the vehicle body, the present invention combines the video monitoring data of multiple cameras, especially the images of the N frames before the starting time point of the vibration, for precise analysis. By extracting the abnormal pixel points in the video, a collision event can be efficiently identified. If the number of abnormal pixel points exceeds a predetermined threshold (the first threshold), it can be directly determined as a collision event, and thus the collision position is sent to the user terminal in a timely manner, greatly improving the accident response speed. When the number of abnormal pixel points does not reach the first threshold, the system further calculates the offset distance of the abnormal pixel points between the first frame and the last frame images. If the offset distance exceeds the second threshold, it is determined as a collision event. This mechanism effectively avoids false alarms for minor vibrations or non-collision events, ensuring high sensitivity and low false alarm rate of the system in practical applications. By sending the collision position to the user terminal in a timely manner, real-time monitoring and feedback of the vehicle safety status are provided. In summary, the present invention improves the accuracy, sensitivity, and response speed of collision recognition through the collaborative work of vibration sensors and cameras. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for use in the embodiments or the description of related technologies. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0068] Figure 1 FIG. shows a schematic flowchart of an automobile safety monitoring method provided by the present invention;

[0069] Figure 2 FIG. shows a schematic diagram of an automobile safety monitoring device provided by an embodiment of the present invention;

[0070] Figure 3 FIG. shows a schematic diagram of a terminal device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0071] In the following description, specific details such as specific system structures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present invention. However, those skilled in the art should clearly understand that the present invention can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present invention.

[0072] Embodiments of the present invention provide an automotive safety monitoring system and method to solve the technical problem of how to improve the accurate recognition ability of the automotive safety monitoring system for the vibration direction.

[0073] First of all, the present invention provides an automotive safety monitoring system. The automotive safety monitoring system includes a plurality of vibration sensors located at different body positions, a plurality of cameras for monitoring different directions, and a processor;

[0074] The vibration sensors are used to collect vibration data;

[0075] The plurality of cameras for monitoring different directions are used to respectively collect video monitoring data of different monitoring directions;

[0076] The processor is used to obtain the vibration data collected by the plurality of vibration sensors and determine the vibration direction according to the plurality of vibration data; the vibration direction includes the bottom direction, the top direction, or the side direction of the vehicle body;

[0077] The processor is used to obtain the video monitoring data corresponding to the plurality of cameras when the vibration direction is the side direction of the vehicle body; wherein, the video monitoring data includes the first N frames before the vibration start time point;

[0078] The processor is used to extract the abnormal pixel points between the first frame image and the last frame image in the video monitoring data;

[0079] The processor is used to, when the number of abnormal pixel points exceeds the first threshold, take the current monitoring direction as the collision position and send the collision position to the user terminal;

[0080] The processor is used to, when the number of abnormal pixel points does not exceed the first threshold, calculate the offset distance of the abnormal pixel points between the first frame image and the last frame image; wherein, the first frame image refers to the starting frame in the video monitoring data, and the last frame image refers to the ending frame in the video monitoring data;

[0081] The processor is used to, when the offset distance is greater than the second threshold and the included angle between the offset direction and the current monitoring direction is greater than the preset angle, take the current monitoring direction as the collision position and send the collision position to the user terminal.

[0082] Secondly, the present invention provides an automotive safety monitoring method. Please refer to Figure 1 , Figure 1The figure shows a schematic flow chart of an automotive safety monitoring method provided by the present invention. As Figure 1 shown, the automotive safety monitoring method may include the following steps:

[0083] Step 101: Obtain vibration data collected by a plurality of vibration sensors, and determine a vibration orientation based on the plurality of vibration data; the vibration orientation includes a bottom orientation, a top orientation, or a vehicle body side orientation;

[0084] The vibration sensor can sense the direction and intensity of the vibration, thereby determining the approximate direction of the vibration source. Furthermore, based on the approximate direction of the vibration source, the camera is mobilized to locate the specific vibration origin point. The vibration orientation includes but is not limited to the bottom orientation, the top orientation, or the vehicle body side orientation. The specific method for detecting the vibration orientation is as follows:

[0085] Specifically, step 101 specifically includes steps 1011 to 1013:

[0086] Step 1011: Obtain vibration data collected by the plurality of vibration sensors; the vibration data includes X-axis vibration data, Y-axis vibration data, and Z-axis vibration data;

[0087] The X-axis vibration data represents the lateral vibration along the vehicle body. The Y-axis vibration data represents the longitudinal vibration along the vehicle body. The Z-axis vibration data represents the vibration in the up and down direction of the vehicle body.

[0088] Step 1012: If the plurality of Z-axis vibration data is greater than a third threshold, determine that the vibration orientation is the bottom orientation or the top orientation;

[0089] The Z-axis vibration data reflects the up and down vibration of the vehicle. If the Z-axis vibration values detected by a plurality of vibration sensors exceed a preset third threshold, this means that the vehicle may have experienced a severe impact in the up and down direction. Usually, this kind of vibration may be related to a collision or earthquake occurring at the top of the vehicle. Among them, the joint detection of a plurality of vibration sensors can avoid errors or noises caused by a single vibration sensor.

[0090] Step 1013: If the plurality of X-axis vibration data or the plurality of Y-axis vibration data is greater than the third threshold, determine that the vibration orientation is the vehicle body side orientation; wherein, the vehicle body side orientation includes a front side orientation, a rear side orientation, a left side orientation, and a right side orientation.

[0091] The X-axis and Y-axis vibration data reflect the lateral and longitudinal vibrations of the vehicle, respectively. If the vibration data in these directions exceeds the third threshold, it means that the vehicle may have encountered an impact or collision from the side. For example, the front or rear of the vehicle may be hit by an external force, or the side of the vehicle may have been hit. Based on these vibration data, the system determines that the vibration occurs on the side of the vehicle body, and therefore determines the vibration direction to be the side direction of the vehicle body. The side direction of the vehicle body includes but is not limited to the front side direction, the rear side direction, the left side direction, and the right side direction.

[0092] In this embodiment, the vibration data obtained from multiple vibration sensors are analyzed and combined with the vibration intensity in different directions to determine the direction of the vibration. When the vibration data indicates that the vehicle has encountered an impact in the up and down directions, the system will determine that the vibration occurred at the bottom or top of the vehicle; if the vibration comes from the lateral or longitudinal direction of the vehicle body, it will be determined to be the side of the vehicle body. In this way, a reference direction is provided for subsequent analysis.

[0093] Step 102: when the vibration direction is the side direction of the vehicle body, obtaining video monitoring data corresponding to multiple cameras; wherein the video monitoring data includes the first N frames of the vibration starting time point;

[0094] When the vibration direction is the side direction of the vehicle body, the video monitoring data corresponding to multiple cameras are obtained. If the vibration direction is determined to be the side of the vehicle body (the side of the vehicle body includes but is not limited to the front, left, right and rear sides), the system will further use the camera installed on the side of the vehicle to obtain relevant video data. The video data captured by the camera includes the first N frames of the vibration, where N is a predetermined value. These front frames help analyze the vehicle conditions before and after the collision, especially the image changes related to the vibration, to accurately locate the vibration direction.

[0095] Step 103: extracting abnormal pixel points between the first frame image and the last frame image in the video surveillance data;

[0096] Extract the first and last frames from the first N frames, and then find "abnormal pixels" by comparing the changes between the two frames. Abnormal pixels represent the image changes caused by the object approaching the car body. These abnormal points are the key clues to the collision.

[0097] Among them, the extraction logic of abnormal pixels is as follows:

[0098] Specifically, step 103 specifically includes steps 1031 to 1033:

[0099] Step 1031: extracting a dividing line between a vehicle body image area and a non-vehicle body image area in the video surveillance data;

[0100] In a video image, the vehicle body image area and the non-vehicle body image area are usually clearly distinguishable. The vehicle body image area refers to the area of the vehicle itself, and the non-vehicle body image area includes the background, surrounding objects, or the environment. Among them, the specific extraction method of the dividing line is as follows:

[0101] Specifically, step 1031 specifically includes steps A1 to A4:

[0102] Step A1: Obtain the vehicle body color information input by the user;

[0103] The system needs to obtain the vehicle body color information. The vehicle body color information is key data for subsequent image processing because the image of the vehicle body area usually exhibits obvious color characteristics different from the background (non-vehicle body area). The user can help the system identify the vehicle body area by inputting the exterior color of the vehicle (such as red, blue, black, etc.).

[0104] Step A2: In the first image area to be recognized in the first frame image, extract the current image area corresponding to the vehicle body color information; wherein, the number of pixels in the current image area is greater than a first preset number and the pixel points are in a continuous relationship;

[0105] The area composed of pixel points found through the vehicle body color information. The number of pixels in this area must be greater than a preset first preset number, that is, the number of pixels in this area needs to be large enough to be considered as an effective part of the vehicle body. In addition, the system requires that these pixel points must be continuous, that is, these pixel points need to be connected together to form an area, rather than scattered pixel points. The continuous pixel points indicate that this part of the area has a certain spatial consistency, which conforms to the characteristics of the vehicle body area.

[0106] Step A3: Take the current image area as the vehicle body image area, and take the remaining area in the first frame image as the non-vehicle body image area; the remaining area refers to the image area in the first frame image except the current image area;

[0107] After determining the current image area, the system marks it as the vehicle body image area, that is, this part of the area is considered to be the vehicle body of the vehicle. Next, the rest of the first frame image except the current image area will be marked as the non-vehicle body image area. This part of the area represents the environment, background, or other objects around the vehicle, usually with a different color from the vehicle body.

[0108] Step A4: Take the critical line between the vehicle body image area and the non-vehicle body image area as the dividing line.

[0109] In an image, an obvious boundary or critical line is formed between the vehicle body image area and the non-vehicle body image area. This critical line is the segmentation line, which marks the boundary between the vehicle body and the non-vehicle body parts.

[0110] In this embodiment, by obtaining the vehicle body color information input by the user, the system can accurately identify the image area that matches the vehicle body color. This method not only reduces the dependence on complex environmental backgrounds but also can work stably in various environments, ensuring the recognition accuracy of the vehicle body area. In the preset area of the first frame image, the current image area corresponding to the vehicle body color information is extracted, and it is required that the number of pixels in this area is greater than the first preset number and the pixel points are in a continuous relationship. This requirement effectively filters out the areas that meet the vehicle body characteristics, avoiding the interference of possible noises or irrelevant parts in the image, thereby ensuring the precise positioning of the vehicle body area. According to the division of the extracted current image area (vehicle body image area) and the remaining area (non-vehicle body image area), the system can clearly distinguish between the vehicle body and the non-vehicle body parts and use the critical line between them as the segmentation line. The extraction of this segmentation line ensures a clear boundary between the vehicle body area and the background, providing a clear basis for subsequent image processing and anomaly detection. By setting the conditions that the number of pixels is greater than the first preset number and has a continuous relationship, the system can effectively filter out irregular or discontinuous areas in the image, avoiding misjudgment caused by factors such as light changes and vehicle body reflections. This condition restriction enhances the stability of the system in complex environments and can adapt to different scenarios and lighting conditions.

[0111] Step 1032: Based on the segmentation line between the vehicle body image area and the non-vehicle body image area, intercept the first image area to be recognized in the area around the segmentation line of the first frame image, and intercept the second image area to be recognized in the area around the segmentation line of the last frame image;

[0112] After determining the segmentation line, the system will intercept a certain area based on the position of the segmentation line in the first frame (the starting frame of the video) and the last frame (the ending frame of the video) images for analysis.

[0113] In the first frame image, the intercepted area based on the position of the segmentation line is called the first image area to be recognized. This area contains the image information of the vehicle body part and the surrounding background. Similarly, in the last frame image, an area will also be intercepted according to the position of the segmentation line, which is called the second image area to be recognized. This area corresponds to the first image area to be recognized and is used for comparison between the front and back frames. Among them, the above-mentioned surrounding area is set based on the center point of the segmentation line, and the corresponding image area is intercepted.

[0114] Step 1033: Calculate the pixel difference between each pixel point in the first image area to be recognized and the second image area to be recognized, and use the pixel points with a pixel difference greater than the fourth threshold as abnormal pixel points.

[0115] By comparing each pixel in the first image region to be recognized and the second image region to be recognized, the difference between them is calculated. If there is a significant change (i.e., a large difference) in the color, brightness, or other characteristic values of a certain pixel point in the two image regions, then this pixel is regarded as an "abnormal pixel point". The fourth threshold is a preset difference threshold. When the difference value of the pixel point is greater than this threshold, it indicates that the change of this pixel point exceeds the normal fluctuation range.

[0116] In this embodiment, by extracting the dividing line between the vehicle body image region and the non-vehicle body image region, the present invention can effectively distinguish the vehicle body part from the surrounding environment, providing a clear boundary for subsequent image analysis. The extraction technology of this dividing line ensures the accurate division of the image region, avoids the influence of interference factors, and thus improves the accuracy of subsequent abnormal pixel point recognition. Based on the dividing line, the system respectively intercepts the first image region to be recognized and the second image region to be recognized in the regions around the dividing line of the first frame image and the last frame image. This method reduces unnecessary calculation amount by centrally processing the pixel data of the key regions, enhances the recognition ability of potential abnormalities. In addition, the comparison between the two key regions further improves the sensitivity of abnormal detection. After calculating the pixel difference between each pixel point in the first image region to be recognized and the second image region to be recognized, the present invention screens abnormal pixel points by setting the fourth threshold. The pixel points with pixel differences greater than the fourth threshold are determined to be abnormal. This method effectively captures the significant differences in image changes and can accurately identify the abnormal changes caused by events such as object contact. Setting an appropriate threshold (the fourth threshold) can effectively avoid misjudgment caused by factors such as environmental light changes and minor changes in object positions, thus ensuring the accuracy and reliability of the system. The application of this threshold enables the system to have a high anti-interference ability and can work stably in a complex dynamic environment.

[0117] Step 104: When the number of the abnormal pixel points exceeds the first threshold, take the current monitoring direction as the collision position and send the collision position to the user terminal;

[0118] Due to the physical law of "objects appear larger when close and smaller when far away" for cameras on the left or right side of the vehicle body (when the impact object is close to the camera, the occupied area of the impact object in the image is larger; when the impact object is far from the camera, the occupied area of the impact object in the image is smaller). Therefore, when the number of abnormal pixel points exceeds the first threshold, it can be considered that the impact object generates an impact at a position close to the camera, thereby causing vibration. When the number of abnormal pixel points does not exceed the first threshold, the occupied area of the impact object in the image is smaller, and there may be confusion between the impact object and the background object (for example: when environmental objects pass near the vehicle body, they will also intersect with the vehicle body, resulting in misjudgment). Therefore, in order to improve the detection accuracy, it is necessary to accurately identify abnormal pixel points.

[0119] Step 105: When the number of the abnormal pixel points does not exceed the first threshold, calculate the offset distance of the abnormal pixel points between the first frame image and the last frame image; wherein, the first frame image refers to the starting frame in the video surveillance data, and the last frame image refers to the ending frame in the video surveillance data;

[0120] The offset distance refers to the displacement of the abnormal pixel points from the first frame to the last frame, which is usually related to the movement of the object after the collision (such as vehicle offset). Among them, the specific calculation logic of the offset distance is as follows:

[0121] Specifically, step 105 specifically includes steps 1051 to 1055:

[0122] Step 1051: If the number of the abnormal pixel points does not exceed the first threshold, extract the abnormal area composed of continuous abnormal pixel points; wherein, the number of pixel points in the abnormal area is greater than the second preset number;

[0123] These abnormal pixel points are aggregated into an abnormal area. This abnormal area is composed of multiple continuous abnormal pixel points, which means that these pixel points are adjacent or connected in space. This can help the system identify a more obvious change area rather than individual discrete noise points. The number of pixel points in the abnormal area must exceed a preset second preset number. This requirement ensures that the change in the abnormal area has sufficient scale or significance, avoiding the system regarding a very small area that may be an error as a valid abnormal area.

[0124] In other words, only when the change in the abnormal area is significant enough will the subsequent analysis continue.

[0125] Step 1052: Identify multiple current feature areas in the abnormal area in the last frame image, and extract the first positions of the multiple current feature areas;

[0126] In the tail frame image (i.e., the last frame of the video), the system further extracts multiple current feature regions from the already identified abnormal regions according to the coordinates or specific positions of each current feature region in the tail frame image. This position is the basis for subsequent tracking and comparison. Among them, the current feature regions are identified as follows:

[0127] Specifically, step 1052 specifically includes steps B1 to B5:

[0128] Step B1: Perform binarization processing on the abnormal region in the tail frame image based on multiple preset gray values respectively to obtain multiple binarized images;

[0129] The multiple preset gray values are multiple consecutive gray values with a fixed difference to analyze the changes in the binarized images of the abnormal region under different gray values. Binarization processing is to convert each pixel point in the image into two states (usually black and white) to simplify image analysis. In this step, the system uses multiple preset gray values to process the abnormal region in the tail frame image. Each preset gray value corresponds to a different binarization threshold, and the system divides the image into two parts according to these thresholds: one is the part with a gray value greater than the threshold, and the other is the part with a gray value less than the threshold. Through different gray values, the system obtains multiple binarized images, and each image represents a different degree of binarization result of the abnormal region, which can highlight the features in different gray value ranges.

[0130] Step B2: Identify the connected regions in the binarized images;

[0131] For each binarized image, the system performs connected region identification. In the binarized image, black and white pixel points may form multiple continuous regions. The system detects these connected regions, that is, regions that are adjacent in space and composed of the same color (black or white).

[0132] Connected regions refer to regions in the image where the pixel values are the same or similar, and the pixel points in these regions are connected to each other. Specifically, assume that in a binary image (for example, a black and white image), the white pixel value is 1 and the black pixel value is 0. Connected regions refer to all pixel points with a value of 1, and these pixel points are connected in space and can be connected horizontally, vertically, or diagonally. There are usually two connection methods:

[0133] 4 - connectivity: Only horizontally and vertically adjacent pixels are considered connected.

[0134] 8 - connectivity: Allows pixels to be connected in the horizontal, vertical, and diagonal directions, and any adjacent pixels are considered connected.

[0135] Step B3: Calculate the gray - scale stability index and texture stability index of each connected region;

[0136] The gray - scale stability index represents the stability of the pixel gray - scale values within the connected region. A region with a smaller gray - scale stability index means that the region does not change much in time or space, that is, the pixel gray - scale values are relatively stable.

[0137] The texture stability index represents the stability of the texture features within the connected region. The role of the local texture stability measure is mainly reflected in improving the performance of feature detection by measuring the stability of the texture within the image region as the gray - scale changes, especially in complex backgrounds.

[0138] Specifically, step B3 specifically includes steps B31 to B32:

[0139] Step B31: Obtain the areas of multiple connected regions corresponding to multiple preset gray - scale values respectively, and substitute the areas of the multiple connected regions corresponding to multiple preset gray - scale values into Function 1 to obtain the gray - scale stability indexes corresponding to the multiple connected regions respectively;

[0140] Function 1 is:

[0141]

[0142] Among them, Δ(A(r)) represents the gray - scale stability index, A(r) represents the area of the current connected region at the preset gray - scale value r, ΔI represents the change amount between adjacent preset gray - scale values among the multiple preset gray - scale values, A(r + ΔI) represents the area of the current connected region at the preset gray - scale value r + ΔI, and A(r - ΔI) represents the area of the current connected region at the preset gray - scale value r - ΔI;

[0143] Function 1 calculates the areas at the preset gray - scale value r and the slight changes above and below this preset gray - scale value (i.e., r + ΔI and r - ΔI). This change indicates whether the region remains stable when the gray - scale changes.

[0144] It represents the ratio of the area change. If this ratio is small, it means that the region remains relatively stable under gray - scale changes. The smaller, the larger. Therefore, the stability of this region is higher. If the area of the region changes little at multiple gray - scale levels, it indicates that the shape of the region remains stable at different thresholds, and such a region is "stable". If the area of the region changes greatly, it means that it is sensitive to gray - scale changes, and then this region is considered unstable.

[0145] The core idea of Function 1 is to measure the "stability" of the region by comparing the area changes of the region at different gray - scale thresholds. Regions with high stability are not easily deformed when the gray - scale changes and can be effectively used as image features for subsequent processing.

[0146] Step B32: Substitute the areas of multiple connected regions corresponding to multiple preset gray values and the gray values of the pixel points in the connected regions into Function 2 to obtain the texture stability indexes corresponding to the multiple connected regions respectively;

[0147] The Function 2 is as follows:

[0148]

[0149] where S texture (Y, t) represents the stability index of the connected region Y at the preset gray value r, I(p, r) represents the gray value of the pixel point p at the preset gray value r, I(p, r - ΔI) represents the gray value of the pixel point p at the preset gray value r - ΔI, |Y| represents the area of the connected region Y, max(I(p, r), I(p, r - ΔI)) represents the maximum value of I(p, r) and I(p, r - ΔI), and p ∈ Y means that the connected region Y includes the pixel point p.

[0150] For each pixel p in the region Y, calculate the gray value change amplitude at two consecutive gray levels (or thresholds): that is, |I(p, r) - I(p, r - ΔI)|. This difference reflects the change degree of the pixel at different gray levels. However, the simple gray difference cannot reflect the texture stability. Therefore, a normalization process is used to normalize the change amplitude of each pixel to its relative change at the current and previous gray levels: This normalization method ensures that even if the gray value itself is large or small, the calculated change amplitudes can be compared on a unified scale.

[0151] The stability of the texture reflects the influence of the gray level change on the image texture structure. To measure the texture stability, a reverse metric is adopted: This reverse metric ensures that a smaller gray change (i.e., the case of smaller texture change) will obtain a higher stability value. If a region has a smaller change at different gray levels, it means that its texture features are more stable, and the corresponding stability metric will be higher.

[0152] Finally, average all the pixels in the region Y to obtain the local texture stability metric of the entire region: In this way, the texture stability not only considers the stability of individual pixels but also reflects the stability of the entire region by taking the average.

[0153] The core of Function 2 is the standardization of gray-level changes, aiming to eliminate the interference of the differences in gray levels themselves on texture evaluation, so as to more accurately measure texture stability. By averaging the texture stability of all pixels within a region, a holistic measure reflecting the smoothness of the change in regional texture features can be obtained. The standardized change amplitude and reverse measure are used to ensure that smoother texture changes will receive higher stability scores, while regions with drastic changes will be judged as unstable. This texture stability measure can capture the continuity of texture features within a region with respect to gray-level changes. Therefore, in a background with complex textures, it can more accurately identify the true feature regions and is not easily affected by background noise or texture interference.

[0154] As an alternative embodiment of the present invention, the weighted processing of the gray-scale stability index and the texture stability index can be used as the final stability index, or either the gray-scale stability index or the texture stability index can be used as the final stability index.

[0155] Step B4: Perform weighted processing on the gray-scale stability index and the texture stability index to obtain a weighted stability index;

[0156] Perform weighted processing on the gray-scale stability index and the texture stability index, and assign different weights to these two indexes according to different situations or priorities. The purpose of this step is to adjust the importance of the two indexes according to actual needs, so as to finally obtain a weighted stability index that comprehensively reflects the regional stability.

[0157] Step B5: Sort the multiple weighted stability indexes, and use the connected regions corresponding to the top N weighted stability indexes as the multiple current feature regions.

[0158] After calculating the weighted stability indexes of all connected regions, the system will sort these indexes. The sorted indexes represent the stability degree of each connected region. The system will select the top N connected regions, whose weighted stability is the strongest, representing the most stable and prominent feature regions in the last frame image. The N connected regions will be selected as the current feature regions, which will be used as the key regions for subsequent analysis and for feature region tracking.

[0159] In this embodiment, by performing binarization processing on the abnormal regions in the tail frame image based on multiple preset gray values respectively, multiple binarized images are obtained. The system can deeply analyze the changes in the image from different gray levels. This multi-dimensional gray analysis helps to capture the subtle changes in the image from different perspectives. After identifying the connected regions in the binarized image, the system can regard the connected pixel blocks in the image as an independent region for processing. The identification of connected regions enables the system to extract meaningful feature regions from the image, avoiding misjudgment caused by noise or small objects. For each connected region, the system calculates its gray stability index and texture stability index. This index calculation method can measure the stability of the gray change inside the region and the stability of the texture structure, so as to reflect whether the region has strong feature performance. Gray stability mainly focuses on the uniformity of the region brightness, while texture stability reflects the persistence of the region shape and structure. These indexes help to conduct a deeper evaluation of the abnormal regions in the image. After weighting the gray stability index and the texture stability index, a comprehensive weighted stability index is obtained. In this way, the system can comprehensively consider the brightness balance of the image and the change law of the texture, and more comprehensively evaluate the importance of the region features. By sorting multiple weighted stability indexes, the system can select the most characteristic regions according to the weighted stability from high to low, and take the connected regions corresponding to the top N weighted stability indexes as the current feature regions. Through this method, the system can ensure to select the regions that can best represent the abnormal behavior or object movement, reducing the interference of irrelevant regions or insignificantly changed regions. The present invention can accurately identify multiple current feature regions in the tail frame image through a series of processing methods such as multi-gray value binarization processing, connected region identification, calculation of gray stability and texture stability indexes, and weighted sorting.

[0160] Step 1053: Trace the feature regions in the video surveillance data sequentially from the tail frame image to the head frame image to obtain the second positions corresponding to each of the multiple current feature regions in the head frame image;

[0161] The system will start from the tail frame image and trace the current feature regions forward (towards the head frame image). The purpose of this feature region tracing is to trace back to the positions in the head frame image, so as to understand the movement trajectories of these abnormal regions in the video. Through feature tracing, the system can finally determine the second positions corresponding to each current feature region in the head frame image, that is, their specific positions in the head frame image. Among them, the feature region tracing can adopt feature tracing algorithms such as the optical flow method. The feature tracing adopted in this embodiment is prior art and will not be elaborated here.

[0162] Step 1054: Calculate the position movement distances between the first positions and the second positions corresponding to each of the multiple current feature regions, and calculate the average value among the multiple position movement distances;

[0163] The system has obtained the first position of each current feature region in the tail frame image and the second position in the head frame image. Through these two positions, the position movement distance of each feature region can be calculated, that is, the movement distance of these regions from the tail frame to the head frame. Then, the system calculates the average value of the position movement distances of all these feature regions. This average value is the comprehensive representation of the offset conditions of multiple feature regions and represents the average displacement of the entire abnormal region between image frames.

[0164] Step 1055: Use the average value as the offset distance.

[0165] The offset distance can be used to analyze the direction and amplitude of the movement of an object (such as a vehicle) in the video and serve as the basis for subsequent motion analysis.

[0166] In this embodiment, when the number of abnormal pixel points does not exceed the first threshold, the present invention first extracts the abnormal region composed of continuous abnormal pixel points to ensure that only the regions with significant changes are concerned. By setting the number of pixel points in the abnormal region to be greater than the second preset number, low-intensity and scattered abnormalities are effectively filtered out, thereby reducing the possibility of false alarms and improving the sensitivity and accuracy of the system. By identifying multiple current feature regions in the abnormal region in the tail frame image and further extracting the first positions of these feature regions, the system can accurately calibrate the initial positions of each feature region. This process lays the foundation for subsequent tracking of feature regions and helps to capture the motion trajectory of the object in the image. In the video surveillance data, the system sequentially tracks the feature regions from the tail frame image to the head frame image and obtains the second positions corresponding to multiple current feature regions in the head frame image. By calculating the movement distances of multiple feature regions between the tail frame image and the head frame image and further calculating the average value of these movement distances, the system can accurately measure the displacement of the abnormal region and thus obtain the offset distance. This offset distance provides an important basis for judging whether a collision or other abnormal behavior has occurred to the object. The calculation of the offset distance provides a quantitative analysis of the object's motion. If the displacement of the object in the image exceeds a set threshold (such as the second threshold), it can be further judged as a possible collision or abnormal event. This processing process improves the accuracy of abnormal event judgment and reduces the interference of environmental factors (such as slight vibrations or non-collision situations) on the monitoring system. This technical solution can still accurately capture the object displacement when the number of abnormal pixel points is small, thereby enhancing the judgment ability and response speed of the vehicle safety monitoring system.

[0167] Step 106: When the offset distance is greater than the second threshold and the included angle between the offset direction and the current monitoring orientation is greater than the preset angle, use the current monitoring orientation as the collision position and send the collision position to the user terminal.

[0168] Since the moving direction of the environmental object is almost parallel to the monitoring orientation (the side direction of the vehicle body) when passing by the vehicle body, while the moving direction of the impacting object usually forms a certain angle with the monitoring orientation and has a certain displacement, when the offset distance is greater than the second threshold and the angle between the offset direction and the current monitoring orientation is greater than the preset angle, it can be considered that an impacting object appears (i.e., a collision event occurs). Then, the current monitoring orientation is used as the collision position and sent to the user terminal for timely response.

[0169] In this embodiment, by acquiring the vibration data collected by multiple vibration sensors, the orientation where the vibration occurs can be accurately determined, including the bottom, top, and side orientations of the vehicle body. The multi-point acquisition and analysis of the vibration data greatly improve the positioning accuracy of the vibration source, providing a reliable data basis for subsequent monitoring and analysis. When the vibration orientation is determined to be the side of the vehicle body, the present invention combines the video monitoring data of multiple cameras, especially the images of the first N frames before the vibration start time point, for precise analysis. By extracting the abnormal pixel points in the video, the collision event can be efficiently identified. If the number of abnormal pixel points exceeds a predetermined threshold (the first threshold), it can be directly determined as a collision event, and thus the collision position is sent to the user terminal in a timely manner, greatly improving the accident response speed. When the number of abnormal pixel points does not reach the first threshold, the system further calculates the offset distance of the abnormal pixel points between the first frame and the last frame images. If the offset distance exceeds the second threshold, it is determined as a collision event. This mechanism effectively avoids false alarms of minor vibrations or non-collision events, ensuring high sensitivity and low false alarm rate of the system in practical applications. By sending the collision position to the user terminal in a timely manner, real-time monitoring and feedback of the vehicle safety status are provided. In summary, the present invention improves the accuracy, sensitivity, and reaction speed of collision recognition through the collaborative work of vibration sensors and cameras.

[0170] As Figure 2 The present invention provides an automotive safety monitoring device. Please refer to Figure 2 , Figure 2 which shows a schematic diagram of an automotive safety monitoring device provided by the present invention. As Figure 2 shown, an automotive safety monitoring device includes:

[0171] A first acquisition unit 21, configured to acquire the vibration data collected by multiple vibration sensors and determine the vibration orientation according to the multiple vibration data; the vibration orientation includes the bottom orientation, the top orientation, or the side orientation of the vehicle body;

[0172] A second acquisition unit 22, configured to acquire the video monitoring data corresponding to multiple cameras when the vibration orientation is the side orientation of the vehicle body; wherein, the video monitoring data includes the first N frames before the vibration start time point;

[0173] An extraction unit 23 is configured to extract abnormal pixel points between the first frame image and the last frame image in the video surveillance data;

[0174] A first determination unit 24 is configured to, when the number of the abnormal pixel points exceeds a first threshold, take the current surveillance orientation as a collision position and send the collision position to a user terminal;

[0175] A second determination unit 25 is configured to, when the number of the abnormal pixel points does not exceed the first threshold, calculate an offset distance of the abnormal pixel points between the first frame image and the last frame image; wherein, the first frame image refers to a starting frame in the video surveillance data, and the last frame image refers to an ending frame in the video surveillance data;

[0176] A sending unit 26 is configured to, when the offset distance is greater than a second threshold and an included angle between the offset direction and the current surveillance orientation is greater than a preset angle, take the current surveillance orientation as a collision position and send the collision position to a user terminal.

[0177] An automotive safety monitoring device provided by the present invention can accurately determine the orientation where a vibration occurs, including the bottom, top, and side body orientations, by acquiring vibration data collected by multiple vibration sensors. The multi-point acquisition and analysis of vibration data greatly improve the positioning accuracy of the vibration source position, providing a reliable data basis for subsequent monitoring and analysis. When the vibration orientation is determined to be the side of the vehicle body, the present invention combines the video surveillance data of multiple cameras, especially the images of the first N frames before the vibration start time point, for precise analysis. By extracting abnormal pixel points in the video, a collision event can be efficiently identified. If the number of abnormal pixel points exceeds a predetermined threshold (the first threshold), a collision event can be directly determined, and thus the collision position is sent to the user terminal in a timely manner, greatly improving the accident response speed. When the number of abnormal pixel points does not reach the first threshold, the system further calculates the offset distance of the abnormal pixel points between the first frame and the last frame images. If the offset distance exceeds the second threshold, a collision event is determined. This mechanism effectively avoids false alarms of minor vibrations or non-collision events, ensuring high sensitivity and low false alarm rate of the system in practical applications. By sending the collision position to the user terminal in a timely manner, real-time monitoring and feedback of the automotive safety status are provided. In summary, the present invention improves the accuracy, sensitivity, and response speed of collision recognition through the collaborative work of vibration sensors and cameras.

[0178] Figure 3 is a schematic diagram of a terminal device provided by an embodiment of the present invention. As Figure 3As shown in the figure, a terminal device 3 of this embodiment includes: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30, such as an automotive safety monitoring program. When the processor 30 executes the computer program 32, the steps in each of the above embodiments of the automotive safety monitoring method are implemented, such as Figure 1 the steps 101 to 106 shown in the figure. Alternatively, when the processor 30 executes the computer program 32, the functions of each unit in each of the above device embodiments are implemented, such as Figure 2 the functions of the units shown in the figure.

[0179] Exemplarily, the computer program 32 may be divided into one or more units. The one or more units are stored in the memory 31 and executed by the processor 30 to complete the present invention. The one or more units may be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program 32 in the terminal device 3. For example, the specific functions of the computer program 32 divided into each unit are as follows:

[0180] A first acquisition unit, configured to acquire vibration data collected by a plurality of vibration sensors, and determine a vibration orientation according to the plurality of vibration data; the vibration orientation includes a bottom orientation, a top orientation, or a vehicle body side orientation;

[0181] A second acquisition unit, configured to acquire video monitoring data corresponding to a plurality of cameras when the vibration orientation is the vehicle body side orientation; wherein, the video monitoring data includes the first N frames before the vibration start time point;

[0182] An extraction unit, configured to extract abnormal pixel points between the first frame image and the last frame image in the video monitoring data;

[0183] A first judgment unit, configured to use the current monitoring orientation as a collision position and send the collision position to the user terminal when the number of the abnormal pixel points exceeds a first threshold;

[0184] A second judgment unit, configured to calculate an offset distance of the abnormal pixel points between the first frame image and the last frame image when the number of the abnormal pixel points does not exceed the first threshold; wherein, the first frame image refers to the starting frame in the video monitoring data, and the last frame image refers to the ending frame in the video monitoring data;

[0185] A sending unit, configured to use the current monitoring orientation as a collision position and send the collision position to the user terminal when the offset distance is greater than a second threshold and the included angle between the offset direction and the current monitoring orientation is greater than a preset angle.

[0186] The terminal device includes, but is not limited to, a processor 30 and a memory 31. Those skilled in the art can understand that Figure 3 This is merely an example of a terminal device 3 and does not constitute a limitation on a terminal device 3. It may include more or fewer components than those shown in the figure, or combine certain components, or different components. For example, the terminal device may further include an input / output device, a network access device, a bus, etc.

[0187] The processor 30 may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0188] The memory 31 may be an internal storage unit of the terminal device 3, such as a hard disk or memory of a terminal device 3. The memory 31 may also be an external storage device of the terminal device 3, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the terminal device 3. Further, the memory 31 may also include both an internal storage unit and an external storage device of the terminal device 3. The memory 31 is used to store the computer program and other programs and data required by the roaming control device. The memory 31 may also be used to temporarily store data that has been output or is to be output.

[0189] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0190] It should be noted that the information interaction, execution process, etc. between the above devices / units, due to being based on the same concept as the method embodiments of the present invention, for their specific functions and the technical effects brought, reference may be specifically made to the method embodiment part, and details are not described herein again.

[0191] Those skilled in the art can clearly understand that, for the convenience and conciseness of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of the present invention. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiment and will not be repeated here.

[0192] An embodiment of the present invention also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the foregoing method embodiments can be implemented.

[0193] An embodiment of the present invention provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal is caused to execute the steps in the foregoing method embodiments.

[0194] If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above method embodiments of the present invention, a computer program can be used to instruct relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps in the foregoing method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can at least include: recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disc, etc.

[0195] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0196] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0197] In the embodiments provided by the present invention, it should be understood that the disclosed apparatus / network device and method can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the couplings or direct couplings or communication connections shown or discussed with each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical or other forms.

[0198] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units. They can be located in one place or distributed to multiple network units.

[0199] It should be understood that when used in the specification of the present invention and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their combinations.

[0200] It should also be understood that the term "and / or" used in the specification of the present invention and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0201] As used in the specification of the present invention and the appended claims, the term "if" can be interpreted as "when", "once", "in response to determining" or "in response to detecting" according to the context. Similarly, the phrase "if it is determined" or "if [the described condition or event] is detected" can be interpreted as meaning "once it is determined", "in response to determining", "once [the described condition or event] is detected" or "in response to detecting [the described condition or event]" according to the context.

[0202] In addition, in the description of the specification and the appended claims of the present invention, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be construed as indicating or implying relative importance.

[0203] The reference to "an embodiment" or "some embodiments" in the description of the present invention means that a specific feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present invention. Thus, the statements "in an embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in another way. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in another way.

[0204] The above-described embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the same; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention and should all be included within the protection scope of the present invention.

Claims

1. An automobile safety monitoring system, characterized in that: The automobile safety monitoring system includes a plurality of vibration sensors located at different positions of the automobile body, a plurality of cameras at monitoring positions, and a processor; The vibration sensor is used to collect vibration data; The cameras at the multiple monitoring positions are used to respectively collect video monitoring data at different monitoring positions; The processor is used to obtain vibration data collected by multiple vibration sensors, and determine the vibration direction according to the multiple vibration data; The vibration orientation includes a bottom orientation, a top orientation or a side orientation of the vehicle body; The processor is used to obtain video monitoring data corresponding to multiple cameras when the vibration direction is the side direction of the vehicle body; wherein the video monitoring data includes the first N frames of the vibration starting time point; The processor is used to extract abnormal pixel points between the first frame image and the last frame image in the video monitoring data; The processor is used for taking the current monitoring position as the collision position and sending the collision position to the user terminal when the number of the abnormal pixel points exceeds a first threshold; The processor is used to calculate the offset distance of the abnormal pixel points between the first frame image and the last frame image when the number of the abnormal pixel points does not exceed the first threshold; wherein the first frame image refers to the starting frame in the video surveillance data, and the last frame image refers to the ending frame in the video surveillance data; The processor is configured to use the current monitoring position as a collision position and send the collision position to a user terminal when the offset distance is greater than a second threshold and an angle between the offset direction and the current monitoring position is greater than a preset angle.

2. A method for monitoring vehicle safety, characterized in that: The automobile safety monitoring method is applied to an automobile safety monitoring system, and the automobile safety monitoring method comprises: Acquire vibration data collected by multiple vibration sensors, and determine the vibration direction according to the multiple vibration data; the vibration direction includes the bottom direction, the top direction or the side direction of the vehicle body; When the vibration direction is the side direction of the vehicle body, video monitoring data corresponding to multiple cameras is obtained; wherein the video monitoring data includes the first N frames of the vibration starting time point; Extracting abnormal pixel points between the first frame image and the last frame image in the video surveillance data; When the number of abnormal pixel points exceeds a first threshold, taking the current monitoring position as a collision position, and sending the collision position to a user terminal; When the number of abnormal pixel points does not exceed the first threshold, calculating the offset distance of the abnormal pixel points between the first frame image and the last frame image; wherein the first frame image refers to the starting frame in the video surveillance data, and the last frame image refers to the ending frame in the video surveillance data; When the offset distance is greater than the second threshold and the angle between the offset direction and the current monitoring position is greater than a preset angle, the current monitoring position is used as the collision position, and the collision position is sent to the user terminal.

3. The automobile safety monitoring method according to claim 2, characterized in that: The step of obtaining vibration data collected by the plurality of vibration sensors and determining the vibration direction according to the plurality of vibration data comprises: Acquire vibration data collected by the plurality of vibration sensors; the vibration data includes X-axis vibration data, Y-axis vibration data and Z-axis vibration data; If a plurality of the Z-axis vibration data are greater than a third threshold, determining that the vibration orientation is a bottom orientation or a top orientation; If the plurality of X-axis vibration data or the plurality of Y-axis vibration data are greater than a third threshold, the vibration orientation is determined to determine the side orientation of the vehicle body; wherein the side orientation of the vehicle body includes the front orientation, the rear orientation, the left orientation and the right orientation.

4. The automobile safety monitoring method according to claim 2, characterized in that: The step of extracting abnormal pixel points between the first frame image and the last frame image in the video surveillance data comprises: Extracting a dividing line between a vehicle body image area and a non-vehicle body image area in the video surveillance data; Based on the dividing line between the vehicle body image area and the non-vehicle body image area, a first image area to be identified is intercepted in an area around the dividing line of the first frame image, and a second image area to be identified is intercepted in an area around the dividing line of the last frame image; Calculate the pixel difference between each pixel in the first image area to be identified and the second image area to be identified, and take the pixel whose pixel difference is greater than a fourth threshold as an abnormal pixel.

5. The automobile safety monitoring method according to claim 4, characterized in that: The step of extracting a dividing line between a vehicle body image area and a non-vehicle body image area in the video monitoring data comprises: Get the vehicle body color information input by the user; Extracting a current image region corresponding to the vehicle body color information in the first image region to be identified in the first frame image; wherein the number of pixels in the current image region is greater than a first preset number and the pixel points are in a continuous relationship; The current image area is used as the vehicle body image area, and the remaining area in the first frame image is used as the non-vehicle body image area; the remaining area refers to the image area in the first frame image other than the current image area; A critical line between the vehicle body image area and the non-vehicle body image area is used as the dividing line.

6. The automobile safety monitoring method according to claim 2, characterized in that: When the number of abnormal pixel points does not exceed the first threshold, the step of calculating the offset distance between the first frame image and the last frame image of the abnormal pixel points comprises: If the number of abnormal pixels does not exceed the first threshold, extracting an abnormal area consisting of continuous abnormal pixels; wherein the number of pixels in the abnormal area is greater than a second preset number; Identify multiple current feature regions in the abnormal region in the tail frame image, and extract first positions of the multiple current feature regions; Tracking the feature regions from the last frame image to the first frame image in the video surveillance data, and obtaining second positions corresponding to the plurality of current feature regions in the first frame image; Calculating the position movement distance between the first position and the second position corresponding to each of the multiple current feature regions, and calculating the average value of the multiple position movement distances; The average value is taken as the offset distance.

7. The automobile safety monitoring method according to claim 6, characterized in that: The step of identifying a plurality of current feature regions in the abnormal region in the tail frame image and extracting the first positions of the plurality of current feature regions comprises: Binarization is performed on the abnormal regions in the tail frame image based on a plurality of preset grayscale values ​​to obtain a plurality of binary images; Identify connected regions in a binary image; Calculate the grayscale stability index and texture stability index of each connected region; The grayscale stability index and the texture stability index are weighted to obtain a weighted stability index; The multiple weighted stability indicators are sorted, and the connected areas corresponding to the first N weighted stability indicators are used as the multiple current feature areas.

8. The automobile safety monitoring method according to claim 7, characterized in that: The step of calculating the grayscale stability index and the texture stability index of each connected region comprises: Acquire the areas of the multiple connected regions corresponding to the multiple preset grayscale values, and substitute the areas of the multiple connected regions corresponding to the multiple preset grayscale values ​​into function one to obtain the grayscale stability index corresponding to the multiple connected regions; The function 1 is: Wherein, Δ(A(r)) represents the grayscale stability index, A(r) represents the area of ​​the current connected region under the preset grayscale value r, ΔI represents the change amount of adjacent preset grayscale values ​​among multiple preset grayscale values, A(r+ΔI) represents the area of ​​the current connected region under the preset grayscale value r+ΔI, and A(r-ΔI) represents the area of ​​the current connected region under the preset grayscale value r-ΔI; Substituting the areas of the multiple connected regions corresponding to the multiple preset grayscale values ​​and the grayscale values ​​of the pixels in the connected regions into function 2, obtaining the texture stability index corresponding to the multiple connected regions; The second function is: Among them, S texture (Y, t) represents the stability index of the connected area Y at the preset grayscale value r, I(p, r) represents the grayscale value of the pixel p at the preset grayscale value r, I(p, r-ΔI) represents the grayscale value of the pixel p at the preset grayscale value r-ΔI, |Y| represents the area of ​​the connected area Y, max(I(p, r), I(p, r-ΔI)) represents the maximum value between I(p, r) and I(p, r-ΔI), and p∈Y represents that the connected area Y includes the pixel p.

9. A terminal device, characterized in that: The terminal device includes: a memory, a processor, and a vehicle safety monitoring program stored in the memory and executable on the processor, wherein the vehicle safety monitoring program is configured to implement the steps in the vehicle safety monitoring method as described in any one of claims 2 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps in the vehicle safety monitoring method according to any one of claims 2 to 8 are implemented.

Images