Vehicle safety monitoring method and device, storage medium and electronic equipment

Abstract

The invention discloses a vehicle safety monitoring method and device, a storage medium and electronic equipment. The method comprises the following steps: when a target vehicle is in a non-driving state, acquiring an image frame sequence acquired by an acquisition device on the target vehicle; wherein the image frame sequence comprises at least two image frames; determining difference information between the at least two image frames, and determining position information of the at least one target object according to the image frame sequence under the condition that the image frame sequence is determined to contain the at least one target object according to the difference information; furthermore, if the position information is located in the key area of the target vehicle, alarm processing is carried out according to the duration of the at least one target object located in the key area. Through the technical scheme provided by the invention, the key area of the target vehicle can be focused, the environmental interference is filtered as much as possible, and the accuracy of monitoring the target vehicle is improved, so that an alarm is given in time under the condition that the target vehicle has a safety risk.

Description

Technical Field

[0001] The present application belongs to the field of intelligent vehicle technology, and in particular relates to a vehicle safety monitoring method, device, storage medium and electronic equipment. Background Art

[0002] At present, vehicle safety monitoring technology has become one of the core technologies of intelligent vehicles. Its main solution is to detect and warn of abnormal dynamics around the vehicle through the coordinated work of cameras and sensors.

[0003] However, conventional visual detection algorithms require a large amount of processor resources and easily affect the normal operation of other vehicle functions, making the algorithm unable to be widely used in smart vehicles of different specifications and levels; at the same time, conventional algorithms are easily interfered with by changes in lighting, weather conditions and other non-threatening movements, making it difficult to distinguish between real threats and noise, resulting in a high false alarm rate in the detection results. Summary of the Invention

[0004] The embodiments of the present application provide a vehicle safety monitoring method, device, storage medium and electronic device, which can focus on the key areas of the target vehicle, filter out environmental interference as much as possible, improve the accuracy of monitoring the target vehicle, and thus provide timely alarms when there are safety risks to the target vehicle.

[0005] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by practice of the present application.

[0006] According to a first aspect of an embodiment of the present application, a vehicle safety monitoring method is provided, comprising:

[0007] When the target vehicle is in a non-driving state, acquiring an image frame sequence acquired by an acquisition device on the target vehicle; wherein the image frame sequence includes at least two image frames;

[0008] determining difference information between at least two image frames;

[0009] In a case where it is determined according to the difference information that the image frame sequence includes at least one target object, determining position information of the at least one target object according to the image frame sequence;

[0010] If the location information indicates that the target vehicle is located in a key area, an alarm process is performed based on the length of time that at least one target object is located in the key area.

[0011] In some embodiments of the present application, based on the aforementioned scheme, the difference information between at least two image frames is determined, including: for any two consecutive image frames in the image frame sequence, calculating the absolute difference between associated pixel points, and using the absolute difference as the difference information between the two consecutive image frames; wherein the associated pixel points are two pixel points with the same coordinates.

[0012] In some embodiments of the present application, based on the aforementioned solution, determining that the image frame sequence contains at least one target object according to the difference information includes:

[0013] For any two consecutive image frames in the image frame sequence, at least one target outline is generated based on associated pixel points whose absolute difference is greater than a preset pixel threshold, and at least one target object contained in the two consecutive image frames is determined from the at least one target outline;

[0014] A fusion process is performed on at least one target object contained in each image frame to obtain at least one target object contained in an image frame sequence.

[0015] In some embodiments of the present application, based on the aforementioned scheme, before generating at least one target contour according to the associated pixel points whose absolute difference is greater than the preset pixel threshold, it also includes: determining the preset pixel threshold under the corresponding monitoring mode according to the light intensity of the current position of the target vehicle; wherein, there are at least two monitoring modes related to the light intensity for the target vehicle, and the preset pixel thresholds under different monitoring modes are different, and the preset pixel threshold is inversely correlated with the light intensity.

[0016] In some embodiments of the present application, based on the aforementioned solution, alarm processing is performed according to the duration that at least one target object is located in a key area, including:

[0017] When at least one target object is located in the key area for a time period shorter than a preset time threshold, controlling the light equipment of the target vehicle to flash;

[0018] When the time duration of at least one target object being located in the key area is not less than a preset time duration threshold, the horn of the target vehicle is controlled to sound the horn, and an image frame corresponding to the at least one target object is saved.

[0019] In some embodiments of the present application, based on the aforementioned solution, before performing alarm processing based on the duration for which at least one target object is located in the critical area, the method further includes:

[0020] Get the image acquisition frame rate of the acquisition device;

[0021] A duration for which the at least one target object is located in the key area is determined according to the number of image frames corresponding to the at least one target object and the image acquisition frame rate.

[0022] In some embodiments of the present application, based on the aforementioned solution, obtaining the image acquisition frame rate of the acquisition device includes: determining the image acquisition frame rate of the acquisition device according to the current time and the current position of the target vehicle.

[0023] According to a second aspect of an embodiment of the present application, a vehicle safety monitoring device is provided, comprising:

[0024] An acquisition module is used to acquire an image frame sequence acquired by an acquisition device on the target vehicle when the target vehicle is in a non-driving state; wherein the image frame sequence includes at least two image frames;

[0025] A first determining module, configured to determine difference information between at least two image frames;

[0026] a second determining module, configured to determine position information of the at least one target object according to the image frame sequence, when it is determined according to the difference information that the image frame sequence includes the at least one target object;

[0027] The alarm module is used to perform alarm processing according to the length of time that at least one target object is located in the key area when the location information indicates that the target vehicle is located in the key area.

[0028] According to a third aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which computer program instructions are stored. When the computer program instructions are loaded and executed by a processor, the steps of the method as described in any one of the first aspects above are implemented.

[0029] According to a fourth aspect of an embodiment of the present application, an electronic device is provided, comprising a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of any one of the methods in the first aspect above are implemented.

[0030] According to a fifth aspect of an embodiment of the present application, a computer program product is provided, comprising a computer program. When the computer program is executed by a processor, the computer program implements the steps of any one of the methods in the first aspect above.

[0031] In the present application, when the target vehicle is in a non-driving state, an image frame sequence acquired by an acquisition device on the target vehicle is obtained; wherein the image frame sequence includes at least two image frames; and then the difference information between the at least two image frames is determined. When it is determined based on the difference information that the image frame sequence contains at least one target object, the position information of the at least one target object is determined based on the image frame sequence; further, if the position information is located in a critical area of ​​the target vehicle, an alarm is processed based on the length of time that the at least one target object is located in the critical area. The technical solution provided by the present application can focus on the critical areas of the target vehicle during the safety monitoring of a parked target vehicle, thereby filtering out environmental interference as much as possible, improving the accuracy of safety monitoring, and then issuing a timely alarm when there is a safety risk to the target vehicle.

[0032] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings are incorporated into and constitute a part of the specification, illustrating embodiments consistent with the present application and, together with the specification, explaining the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and those skilled in the art can derive other drawings based on these drawings without inventive effort. In the drawings:

[0034] Figure 1 A schematic diagram showing a scenario in which the vehicle safety monitoring method according to an embodiment of the present application can be applied;

[0035] Figure 2 A flow chart of a vehicle safety monitoring method in an embodiment of the present application is shown;

[0036] Figure 3 A detailed flow chart of determining whether an image frame sequence contains a target object in an embodiment of the present application is shown;

[0037] Figure 4 A detailed flowchart of alarm processing according to duration in an embodiment of the present application is shown;

[0038] Figure 5 A detailed flow chart of determining the duration of time in a key area in an embodiment of the present application is shown;

[0039] Figure 6 Another flow chart of the vehicle safety monitoring method in an embodiment of the present application is shown;

[0040] Figure 7 A block diagram of a vehicle safety monitoring device in an embodiment of the present application is shown;

[0041] Figure 8 A schematic structural diagram of an electronic device in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0043] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner.In the following description, many specific details are provided so as to provide a full understanding of the embodiments of the present application. However, it will be appreciated by those skilled in the art that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps etc. can be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0044] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0045] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0046] In order to make those skilled in the art better understand this application, first combine Figure 1 A brief description of the application scenarios involved in this application is given.

[0047] See also Figure 1 , which shows a scenario schematic diagram of the vehicle safety monitoring method of the embodiment of the present application can be applied.

[0048] In the present application, the vehicle safety monitoring method can be independently executed by the vehicle-side controller of the target vehicle, reusing the existing acquisition equipment on the target vehicle to collect image frame sequences, and then determining the difference information through a lightweight algorithm, and judging whether there is a target object located in the key area of ​​the target vehicle based on the difference information. This method does not require new hardware and does not require the configuration of GPU (Graphics Processing Unit) or NPU (Neural Processing Unit) acceleration chips. It is widely applicable to different types of vehicle models and can ensure the real-time and accuracy of safety monitoring of the target vehicle.

[0049] Specifically, when the target vehicle is in a non-driving state, the vehicle-side controller 101 obtains an image frame sequence including at least two image frames obtained by the acquisition device on the target vehicle, and then determines the difference information between the at least two image frames. When it is determined that the image frame sequence contains at least one target object based on the difference information, the position information of the at least one target object is determined based on the image frame sequence. Furthermore, based on the position information, it is determined whether the at least one target object is located in a critical area of ​​the target vehicle. If so, an alarm is processed according to the length of time that the at least one target object is located in the critical area. Finally, the vehicle-side controller 101 uploads the processing results to the server 102 for storage in the corresponding data storage system.

[0050] Optionally, the vehicle safety monitoring method can be executed by the server 102, and the vehicle-side controller 101 uploads the image frame sequence obtained by the acquisition device on the target vehicle to the server 102, so that the server 102 obtains the image frame sequence including at least two image frames, and then determines the difference information between the at least two image frames. When it is determined that at least one target object is included in the image frame sequence according to the difference information, the position information of the at least one target object is determined according to the image frame sequence. Furthermore, based on the position information, it is judged whether the at least one target object is located in the critical area of ​​the target vehicle. If so, an alarm processing is performed according to the length of time that the at least one target object is located in the critical area. Finally, the server 102 sends relevant instructions for alarm processing to the vehicle-side controller 101 to control the target vehicle to flash lights or sound the horn.

[0051] Optionally, the vehicle-side controller 101 communicates with the server 102 to collaboratively execute the vehicle safety monitoring method of the embodiment of the present application.

[0052] The server 102 may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.

[0053] In an exemplary embodiment, referring to Figure 2 , shows a flow chart of the vehicle safety monitoring method in an embodiment of the present application, which is described in detail as follows:

[0054] Step 201 : When a target vehicle is in a non-driving state, a sequence of image frames acquired by an acquisition device on the target vehicle is acquired.

[0055] The continuous image frames form an image frame sequence, and the image frame sequence includes at least two image frames.

[0056] The target vehicle is the vehicle currently requiring safety monitoring. When the target vehicle is not in motion, such as when the engine is turned off, the acquisition device on the target vehicle continuously captures image frames at a preset image acquisition frame rate to obtain an image frame sequence. Generally, when the target vehicle is turned off, the power consumption for safety monitoring of the target vehicle is reduced from 5-10W to 0.3-0.5W. In extreme environments, the battery life can even be extended by 3 times. This application can protect the vehicle battery as much as possible without affecting the normal startup of the target vehicle or the normal operation of other functions.

[0057] The target vehicle can be equipped with multiple acquisition devices, such as a dashcam, a reversing camera, and a surround-view camera. This embodiment does not limit the type or number of acquisition devices. These acquisition devices are used to capture images of the environment surrounding the target vehicle. By combining the image frames captured by multiple acquisition devices, it is possible to determine whether the target vehicle is at risk of theft or vandalism.

[0058] Step 202: Determine difference information between at least two image frames.

[0059] The difference information between the image frames is used to represent the pixel changes between the image frames.

[0060] For a sequence of image frames acquired by the same acquisition device, difference information between each image frame is determined. Specifically, when the image frame sequence includes a large number of image frames, the image frames may be grouped. For example, every two consecutive image frames or every three consecutive image frames may be grouped together. Difference information between the image frames in the same group is then determined. Ultimately, the difference information between each group is combined to obtain difference information between all image frames in the sequence.

[0061] For example, an inter-frame differencing method is used to determine the difference information between all image frames in the image frame sequence, thereby distinguishing the static background and moving objects in the image frame sequence and determining the motion region corresponding to each group of image frames. Inter-frame differencing methods include at least two-frame differencing methods and three-frame differencing methods, which are not limited in this embodiment.

[0062] Step 203 : When it is determined according to the difference information that the image frame sequence includes at least one target object, position information of the at least one target object is determined according to the image frame sequence.

[0063] Based on the difference information, a determination is made as to whether the image frame sequence contains at least one target object. If so, the location information of the at least one target object is determined based on the image frame sequence. If not, a new image frame sequence is acquired and the above steps are repeated. Exemplarily, based on the difference information, a search is performed for target contours with significant pixel changes in the image frame sequence. The target contours in the image frame sequence are then traversed to determine whether the target contour corresponds to the at least one target object.

[0064] When pixels in an area change, this area is characterized by motion, also known as a target outline. When traversing target outlines in a sequence of image frames, one or more target outlines may be encountered. The same target outline corresponds to the same target object. A target outline can be the outline of a person, a vehicle, a fallen leaf, or a flying bird. Target objects do not include interfering objects like fallen leaves and flying birds; they only refer to objects that pose a risk to the target vehicle.

[0065] Furthermore, based on the position of at least one target object in the image scene, the position of at least one target object in the real scene is determined, such as the specific position of at least one target object in the front, rear, door or window of the target vehicle, to obtain the position information of at least one target object.

[0066] For example, critical and non-critical areas can be defined based on the vehicle's physical structure. Critical areas include, but are not limited to, the areas corresponding to doors and windows. Using rectangles or polygons, the critical and non-critical areas in the image scene can be precisely defined. Consequently, the critical and non-critical areas in the real scene can be obtained.

[0067] Based on the location information of the at least one target object, determine whether the at least one target object is located in the critical area of ​​the target vehicle. If so, perform an alarm process; if not, continue to acquire a new image frame sequence and repeat the above steps.

[0068] Step 204 : When the location information indicates that the target vehicle is located in a key area, an alarm process is performed based on the duration that the at least one target object is located in the key area.

[0069] When at least one target object is located in a critical area of ​​a target vehicle, it may pose a risk to the target vehicle. In order to avoid false alarms and further improve the accuracy of alarm processing, it is necessary to determine the corresponding alarm processing measures based on the length of time that at least one target object is located in the critical area, so as to achieve a graded response for safety monitoring of the target vehicle.

[0070] Optionally, for each target object, the timestamp of the first appearance of the target object in the image frame of the key area and the timestamp of the last appearance of the target object in the image frame of the key area are determined, and the length of time the target object is in the key area is calculated based on the timestamp. If the duration is short, only a warning is given to the target object. If the duration is long, the situation is also reported to the management platform or the terminal device held by the vehicle owner.

[0071] In the present application, when the target vehicle is in a non-driving state, an image frame sequence acquired by an acquisition device on the target vehicle is obtained; wherein the image frame sequence includes at least two image frames; and then the difference information between the at least two image frames is determined. When it is determined based on the difference information that the image frame sequence contains at least one target object, the position information of the at least one target object is determined based on the image frame sequence; further, if the position information is located in a critical area of ​​the target vehicle, an alarm is processed based on the length of time that the at least one target object is located in the critical area. The technical solution provided by the present application can focus on the critical areas of the target vehicle during the safety monitoring of a parked target vehicle, thereby filtering out environmental interference as much as possible, improving the accuracy of safety monitoring, and then issuing a timely alarm when there is a safety risk to the target vehicle.

[0072] In an exemplary embodiment, determining difference information between at least two image frames specifically includes: for any two consecutive image frames in an image frame sequence, calculating the absolute difference between associated pixel points, and using the absolute difference as the difference information between the two consecutive image frames; wherein the associated pixel points are two pixel points with the same coordinates.

[0073] For example, image frames 01 and 02 are two consecutive image frames in the same image frame sequence. Since these two image frames are related to the target vehicle and captured from the same angle, the associated pixels in the two image frames can be directly compared. The absolute value of the difference between the pixel values ​​of the associated pixels is calculated, which is the absolute difference between the associated pixels. Then, the absolute difference between all associated pixels is combined to obtain the difference information between image frames 01 and 02.

[0074] Optionally, for any two consecutive image frames, the absolute difference of pixel values ​​between the frames is calculated to obtain difference information between the frames.

[0075] Based on the above embodiments, in an exemplary embodiment, see Figure 3 , shows a detailed flowchart of determining whether an image frame sequence contains a target object in an embodiment of the present application, specifically including:

[0076] Step 301: for any two consecutive image frames in an image frame sequence, generate at least one target contour based on associated pixel points whose absolute difference is greater than a preset pixel threshold, and determine at least one target object contained in the two consecutive image frames from the at least one target contour.

[0077] For example, after calculating the absolute difference between the associated pixel points in image frame 01 and image frame 02, it is determined whether the absolute difference is greater than a preset pixel threshold, and then the associated pixel points whose absolute difference is greater than the preset pixel threshold are divided into the corresponding area of ​​the moving object, and the associated pixel points whose absolute difference is not greater than the preset pixel threshold are divided into the corresponding area of ​​the static background. At least one target contour in image frame 01 and image frame 02 can be obtained, and further, at least one target object can be obtained based on the analysis of at least one target contour.

[0078] Optionally, the target outlines are filtered to remove interfering outlines, such as fallen leaves and flying birds, while retaining the outlines of objects that could pose a risk to the target vehicle, thereby obtaining the target object, such as a person or a car. This filtering can be performed by calculating the centroid displacement of the target outlines. For example, if the centroid displacement of the target outlines exceeds 10 pixels in five consecutive image frames, the corresponding target object is retained. This can eliminate transient interference such as flying birds.

[0079] Optionally, the preset pixel threshold in the corresponding monitoring mode is determined based on the light intensity at the current position of the target vehicle; wherein, there are at least two monitoring modes related to light intensity for the target vehicle, and the preset pixel thresholds in different monitoring modes are different, and the preset pixel threshold is inversely correlated with the light intensity.

[0080] The preset pixel threshold is a dynamically changing threshold that varies depending on the corresponding light intensity. Generally, higher light intensity results in a lower preset pixel threshold, and vice versa. By dynamically adjusting the preset pixel threshold, the adaptive threshold algorithm can dynamically adjust the sensitivity of detecting target outlines and objects when comparing related pixels, minimizing the effects of low-light noise and overexposure in strong light.

[0081] For example, the light intensity during the day is much higher than the light intensity at night. Therefore, a daytime monitoring mode related to light intensity and a nighttime monitoring mode related to light intensity can be determined. The preset pixel threshold for the daytime monitoring mode is 15, while the preset pixel threshold for the nighttime monitoring mode is 30. That is, for image frames 01 and 02 captured in the daytime monitoring mode, the associated pixels with an absolute difference greater than 15 in image frames 01 and 02 are divided into regions corresponding to moving objects, and the target object corresponding to the target outline is obtained through analysis. For image frames 03 and 04 captured in the nighttime monitoring mode, the associated pixels with an absolute difference greater than 30 in image frames 03 and 04 are divided into regions corresponding to moving objects, and the target object corresponding to the target outline is obtained through analysis.

[0082] Monitoring modes are not limited to the daytime and nighttime monitoring modes described above. For example, the light intensity on a sunny day is generally 30,000-300,000 lux, the light intensity on a cloudy day is generally 3,000-10,000 lux, the light intensity at sunrise and sunset is generally around 300 lux, the light intensity at night is generally 0.001-0.02 lux, and the light intensity of street lights at night is generally around 0.1 lux. Multiple monitoring modes can be determined based on the range of light intensities, each corresponding to a preset pixel threshold, and the preset pixel thresholds corresponding to different monitoring modes vary in size.

[0083] Optionally, the light intensity at the current position of the target vehicle is detected by a sensor configured on the target vehicle, and the corresponding monitoring mode is activated according to the light intensity. After obtaining the image frame sequence, the target contour and target object are determined based on the absolute difference of the associated pixel points, and the preset pixel threshold under the monitoring mode is selected.

[0084] Step 302: Perform fusion processing on at least one target object contained in each image frame to obtain at least one target object contained in the image frame sequence.

[0085] As the target object moves, the coordinates of associated pixels whose absolute difference is greater than a preset pixel threshold also change. This requires fusion processing of the same target object in the image frame sequence. For example, target objects corresponding to overlapping target outlines are determined to be the same target object. For example, in an image frame sequence comprising 20 consecutive image frames, it is determined that the first 12 frames contain the same target object A, and the last 10 frames contain the same target object B. Ultimately, target objects A and B are obtained from the image frame sequence.

[0086] Optionally, adaptive histogram equalization is used to preprocess all image frames in the image frame sequence to enhance contrast and reduce the impact of illumination. Morphological filtering is then introduced after calculating the frame-to-frame difference to remove noise points and small holes and improve the target contour. Finally, time series analysis is combined with the results of multi-frame difference to obtain the target object contained in the image frame sequence.

[0087] In the present application, batch processing is performed on multiple image frames included in an image frame sequence, and the pixel value changes of pixel points with the same coordinates in two consecutive image frames are compared, that is, the absolute difference of the associated pixel points, so as to detect the target contour and target object in the image frame sequence, thereby ensuring the reliability of the detection results; at the same time, an adaptive threshold algorithm is introduced to dynamically adjust the preset pixel threshold based on the light intensity to avoid misjudgment caused by low-light noise and strong light overexposure as much as possible, thereby improving the accuracy of determining the target object. In actual applications, the method provided in the embodiment of the present application can reduce the false alarm rate to less than 5%.

[0088] In an exemplary embodiment, see Figure 4 , shows a detailed flowchart of alarm processing based on duration in an embodiment of the present application, specifically including:

[0089] Step 401: Determine whether the time duration that at least one target object is located in the key area of ​​the target vehicle is less than a preset time duration threshold.

[0090] If yes, go to step 402; if no, go to step 403.

[0091] Step 402: Control the lighting equipment of the target vehicle to flash.

[0092] Step 403: Control the horn of the target vehicle to honk, and save at least one image frame corresponding to the target object.

[0093] That is, when the time duration that at least one target object is located in the key area is less than a preset time threshold, the lighting equipment of the target vehicle is controlled to flash; when the time duration that at least one target object is located in the key area is not less than the preset time threshold, the horn of the target vehicle is controlled to honk, and the image frame corresponding to at least one target object is saved.

[0094] Specifically, a preset time threshold, such as 10 seconds, is set. If the target object is within the critical area for less than 10 seconds, it is identified as a Level 1 threat. If the target object briefly approaches, the hazard lights can flash three times to alert the target object. If the target object is within the critical area for at least 10 seconds, it is identified as a Level 2 threat. If the target object continues to linger, the horn can be honk for 0.5 seconds and recording can be initiated. Recording can be initiated by using a specific video sensor or by directly combining image frames corresponding to the target object into a new image frame sequence.

[0095] Optionally, the target vehicle can also be equipped with a vibration sensor. Based on the above response measures, if the vibration sensor detects that the target vehicle has been violently damaged, it will be identified as a level 3 threat, and recording will be started and uploaded. For example, the video will be sent to a management platform or the terminal device held by the vehicle owner for timely review by the management personnel or the vehicle owner. In actual application, the corresponding App (application) client can be installed in advance on the terminal device held by the vehicle owner. If the vehicle-side controller detects that the target vehicle has been violently damaged, the video will be pushed to the client to issue an alert to the vehicle owner.

[0096] Optionally, the battery voltage or temperature of the target vehicle can be detected in real time, and vehicle safety monitoring can be automatically stopped when the preset conditions are met, to prevent the vehicle from being unable to start normally due to battery exhaustion.

[0097] In this application, a hierarchical response mechanism is adopted. When a level one threat is triggered, the power consumption is about 0.1W, and when a level two threat is triggered, the power consumption is about 0.5W. This not only saves energy consumption and optimizes resource allocation, but also accurately reflects the risk level and improves the intelligence and rationality of the response measures.

[0098] In an exemplary embodiment, see Figure 5 , shows a detailed flowchart of determining the duration of being in a key area in an embodiment of the present application, specifically including:

[0099] Step 501: Obtain the image acquisition frame rate of the acquisition device.

[0100] When the target vehicle is not in a driving state, such as when the target vehicle is turned off, a lower image acquisition frame rate is used, such as 1 frame / second, and the sleep ratio is 90%.

[0101] Optionally, the image acquisition frame rate of the acquisition device is determined based on the current time and the current position of the target vehicle. In practical applications, the probability of the target vehicle being at risk at night is much greater than the probability of being at risk during the day. The image acquisition frame rate of the acquisition device can be determined based on the current time so that the image acquisition frame rate at night is greater than the image acquisition frame rate during the day. The probability of the target vehicle being at risk when parked on the roadside is much greater than the probability of being at risk when parked in a parking lot. The current position of the target vehicle can be used to detect whether the target vehicle is parked in a safe location. If so, a lower image acquisition frame rate can be used; if not, a higher image acquisition frame rate can be used.

[0102] Optionally, corresponding weights are assigned to the current time and the current position, and the risk probability is calculated by weighted calculation, and then a matching image acquisition frame rate is selected.

[0103] Step 502 : Determine the duration that the at least one target object is located in the key area according to the number of image frames corresponding to the at least one target object and the image acquisition frame rate.

[0104] Specifically, for any target object, in the process of determining the duration that the target object is located in the critical area of ​​the target vehicle, the image frames in which the target object appears can be filtered out from the image frame sequence, and the number of filtered out image frames can be used as the number of image frames corresponding to at least one target object. The product of the number of image frames corresponding to at least one target object and the image acquisition frame rate can then be determined as the duration that at least one target object is located in the critical area.

[0105] Optionally, for any target object, image frames in which the target object is located in a key area of ​​the target vehicle are filtered out from the image frame sequence, and the number of filtered image frames is used as the number of image frames corresponding to the target object. The product of the number of image frames corresponding to the target object and the image acquisition frame rate is then determined as the duration that the target object is located in the key area.

[0106] In this application, the duration that the target object is located in the key area of ​​the target vehicle is accurately determined, providing a data basis for the hierarchical response mechanism. In addition, the image acquisition frame rate is dynamically adjusted according to the current time and current position of the target vehicle, which can save energy power consumption, optimize resource allocation, improve the coverage time of the vehicle safety monitoring method, and ensure the endurance of the target vehicle.

[0107] In order to enable those skilled in the art to better understand the present application as a whole, the application process of the present application scheme will be briefly described below using a specific embodiment:

[0108] See also Figure 6 , shows another flow chart of the vehicle safety monitoring method in an embodiment of the present application, which specifically includes:

[0109] Step 601: When a target vehicle is in a non-driving state, an image frame sequence acquired by an acquisition device on the target vehicle is acquired.

[0110] Step 602: Determine a preset pixel threshold corresponding to the monitoring mode according to the light intensity at the current position of the target vehicle.

[0111] There are at least two monitoring modes for the target vehicle that are related to light intensity, and the preset pixel thresholds in different monitoring modes are different, and the preset pixel thresholds are inversely correlated with the light intensity.

[0112] Step 603 : For any two consecutive image frames in the image frame sequence, calculate the absolute difference between the associated pixels.

[0113] The associated pixel points are two pixel points with the same coordinates.

[0114] Step 604 : Determine whether there are any associated pixels whose absolute difference is greater than a preset pixel threshold.

[0115] If yes, execute step 605; if no, return to execute step 601.

[0116] Step 605: Generate at least one target contour based on associated pixel points whose absolute difference values ​​are greater than a preset pixel threshold.

[0117] Step 606 : Determine whether at least one target object included in two consecutive image frames exists in at least one target outline.

[0118] If yes, execute step 607; if no, return to execute step 601.

[0119] Step 607 : performing fusion processing on the at least one target object contained in each image frame to obtain the at least one target object contained in the image frame sequence.

[0120] Step 608: Determine position information of at least one target object according to the image frame sequence.

[0121] Step 609 , determining whether the location information is located in a key area of ​​the target vehicle.

[0122] If yes, execute step 610 ; if no, return to execute step 601 .

[0123] Step 610 : Determine the duration for which the at least one target object is located in the key area according to the number of image frames corresponding to the at least one target object and the image acquisition frame rate of the acquisition device.

[0124] Step 611: Determine whether the time duration that at least one target object is located in the key area is less than a preset time duration threshold.

[0125] If yes, go to step 612; if no, go to step 613.

[0126] Step 612: Control the lighting equipment of the target vehicle to flash.

[0127] Step 613: Control the horn of the target vehicle to honk, and save at least one image frame corresponding to the target object.

[0128] In this application, during the process of security monitoring of a parked target vehicle, it is possible to focus on the key areas of the target vehicle, thereby filtering out environmental interference as much as possible, improving the accuracy of security monitoring, and then issuing timely alarms when there is a safety risk to the target vehicle.

[0129] The following describes an embodiment of the device of the present application, which can be used to implement the vehicle safety monitoring method in the above embodiment of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the vehicle safety monitoring method in the above embodiment of the present application.

[0130] See also Figure 7 , shows a block diagram of a vehicle safety monitoring device 700 in an embodiment of the present application, specifically including:

[0131] The acquisition module 701 is used to acquire an image frame sequence acquired by an acquisition device on the target vehicle when the target vehicle is in a non-driving state; wherein the image frame sequence includes at least two image frames.

[0132] The first determining module 702 is configured to determine difference information between at least two image frames.

[0133] The second determining module 703 is configured to determine position information of the at least one target object according to the image frame sequence when it is determined according to the difference information that the image frame sequence includes at least one target object.

[0134] The alarm module 704 is configured to, when the location information indicates that the target vehicle is located in a key area, perform alarm processing based on the duration for which at least one target object is located in the key area.

[0135] In an exemplary embodiment, the above-mentioned first determination module 702 is specifically used to calculate the absolute difference between associated pixel points for any two consecutive image frames in the image frame sequence, and use the absolute difference as the difference information between the two consecutive image frames; wherein the associated pixel points are two pixel points with the same coordinates.

[0136] In an exemplary embodiment, the vehicle safety monitoring device 700 further includes:

[0137] The object determination module is used to generate at least one target contour based on associated pixel points whose absolute difference between any two consecutive image frames in the image frame sequence is greater than a preset pixel threshold, and to determine at least one target object contained in the two consecutive image frames from the at least one target contour.

[0138] The object fusion module is used to perform fusion processing on at least one target object contained in each image frame to obtain at least one target object contained in the image frame sequence.

[0139] In an exemplary embodiment, the vehicle safety monitoring device 700 further includes:

[0140] The threshold determination module is used to determine the preset pixel threshold in the corresponding monitoring mode based on the light intensity at the current position of the target vehicle; wherein the target vehicle has at least two monitoring modes related to light intensity, and the preset pixel thresholds in different monitoring modes are different, and the preset pixel threshold is inversely correlated with the light intensity.

[0141] In an exemplary embodiment, the alarm module 704 includes:

[0142] The first alarm unit is configured to control the lighting equipment of the target vehicle to flash when the duration for which at least one target object is located in the critical area is less than a preset duration threshold.

[0143] The second alarm unit is used to control the horn of the target vehicle to sound the horn and save the image frame corresponding to the at least one target object when the time length of at least one target object located in the key area is not less than a preset time length threshold.

[0144] In an exemplary embodiment, the vehicle safety monitoring device 700 further includes:

[0145] The frame rate acquisition module is used to obtain the image acquisition frame rate of the acquisition device.

[0146] The duration determination module is used to determine the duration that at least one target object is located in the key area according to the number of image frames corresponding to the at least one target object and the image acquisition frame rate.

[0147] In an exemplary embodiment, the frame rate acquisition module is specifically configured to determine the image acquisition frame rate of the acquisition device according to the current time and the current position of the target vehicle.

[0148] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium, in which computer program instructions are stored. When the computer program instructions are loaded and executed by a processor, the steps of the vehicle safety monitoring method as described above are implemented.

[0149] Based on the same inventive concept, the embodiment of the present application provides an electronic device, see Figure 8 , shows a schematic structural diagram of an electronic device in an embodiment of the present application, the electronic device includes one or more memories 804, one or more processors 802 and at least one computer program stored in the memory 804 and executable on the processor 802. When the processor 802 executes the computer program, the steps of the vehicle safety monitoring method described above are implemented.

[0150] The bus architecture (represented by bus 800) may include any number of interconnected buses and bridges, and bus 800 links various circuits including one or more processors represented by processor 802 and memory represented by memory 804. Bus 800 may also link various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not described further herein. Bus interface 805 provides an interface between bus 800 and receiver 801 and transmitter 803. Receiver 801 and transmitter 803 may be the same component, namely a transceiver, which provides a unit for communicating with various other devices over a transmission medium. Processor 802 is responsible for managing bus 800 and general processing, while memory 804 may be used to store data used by processor 802 when performing operations.

[0151] The functions described herein may be implemented in hardware, software executed by a processor, firmware, or any combination thereof. If implemented in software executed by a processor, the functions may be stored as one or more instructions or codes on or transmitted via a computer-readable medium. Other examples and implementations are within the scope and spirit of this application and the appended claims. For example, due to the nature of software, the functions described above may be implemented using software executed by a processor, hardware, firmware, hardwiring, or a combination of any of these. Furthermore, the functional units may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit.

[0152] Based on the same inventive concept, an embodiment of the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the steps of the above-mentioned vehicle safety monitoring method.

[0153] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0154] The units described as separate components may or may not be physically separate, and the components of the control device may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0155] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc. Various media that can store computer program instructions.

[0156] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of the claims of the present application.

Claims

1. A vehicle safety monitoring method, characterized in that: The method comprises: When the target vehicle is in a non-driving state, acquiring an image frame sequence acquired by an acquisition device on the target vehicle; wherein the image frame sequence includes at least two image frames; determining difference information between the at least two image frames; In a case where it is determined according to the difference information that the image frame sequence includes at least one target object, determining position information of the at least one target object according to the image frame sequence; If the location information indicates that the target vehicle is located in a key area, an alarm process is performed according to the length of time that the at least one target object is located in the key area.

2. The method according to claim 1, characterized in that The determining the difference information between the at least two image frames includes: For any two consecutive image frames in the image frame sequence, the absolute difference between the associated pixel points is calculated, and the absolute difference is used as the difference information between the two consecutive image frames; wherein the associated pixel points are two pixel points with the same coordinates.

3. The method according to claim 2, characterized in that Determining, based on the difference information, that the image frame sequence includes at least one target object includes: For any two consecutive image frames in the image frame sequence, generating at least one target outline based on associated pixel points whose absolute difference is greater than a preset pixel threshold, and determining at least one target object contained in the two consecutive image frames from the at least one target outline; A fusion process is performed on at least one target object contained in each image frame to obtain at least one target object contained in the image frame sequence.

4. The method according to claim 3, characterized in that Before generating at least one target contour based on the associated pixel points whose absolute difference values ​​are greater than a preset pixel threshold, the method further includes: According to the light intensity at the current position of the target vehicle, a preset pixel threshold under the corresponding monitoring mode is determined; wherein, the target vehicle has at least two monitoring modes related to light intensity, and the preset pixel thresholds under different monitoring modes are different, and the preset pixel threshold is inversely correlated with the light intensity.

5. The method according to claim 1, wherein The performing alarm processing according to the duration for which the at least one target object is located in the key area includes: When the duration for which the at least one target object is located in the key area is less than a preset duration threshold, controlling the lighting equipment of the target vehicle to flash; When the duration for which the at least one target object is located in the key area is not less than the preset duration threshold, the horn of the target vehicle is controlled to sound, and an image frame corresponding to the at least one target object is saved.

6. The method according to claim 5, characterized in that Before performing alarm processing based on the duration for which the at least one target object is located in the key area, the method further includes: Obtaining the image acquisition frame rate of the acquisition device; A time duration for which the at least one target object is located in the key area is determined according to the number of image frames corresponding to the at least one target object and the image acquisition frame rate.

7. The method according to claim 6, characterized in that The acquiring the image acquisition frame rate of the acquisition device includes: The image acquisition frame rate of the acquisition device is determined according to the current time and the current position of the target vehicle.

8. A vehicle safety monitoring device, characterized in that: The device comprises: an acquisition module, configured to acquire, when the target vehicle is in a non-driving state, an image frame sequence acquired by an acquisition device on the target vehicle; wherein the image frame sequence includes at least two image frames; A first determining module, configured to determine difference information between the at least two image frames; a second determining module, configured to determine position information of the at least one target object according to the image frame sequence, if it is determined according to the difference information that the image frame sequence includes at least one target object; The alarm module is used to perform alarm processing according to the length of time that the at least one target object is located in the key area when the location information indicates that the at least one target object is located in the key area.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer program instructions, which, when loaded and executed by a processor, implement the steps of the method according to any one of claims 1 to 7.

10. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program, and when the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images