A vehicle safety monitoring method and device, a vehicle and a storage medium

Abstract

The application provides a vehicle safety monitoring method and device, a vehicle and a storage medium, and relates to the technical field of vehicle safety. A current image collected by a vehicle-mounted visual sensor is acquired; target detection is performed on the current image to identify a current suspicious target in the current image that meets a preset condition; current position information of the current suspicious target in the current image is determined, and a current unique identifier corresponding to the current suspicious target is generated; from a comparison object list, a historical unique identifier identical to the current unique identifier is filtered, and historical position information corresponding to the historical unique identifier is extracted; based on the current position information and the historical position information, the current suspicious target is verified as a stationary object; and when the result of the stationary object verification indicates that the current suspicious target is a non-stationary object, an alarm operation is triggered. In a low-computing-power-cost manner, false alarms caused by stationary suspicious targets are eliminated, the effectiveness of the alarm is improved, the energy consumption of the vehicle is reduced, and the endurance of the vehicle is improved.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and in particular to a vehicle safety monitoring method, device, vehicle, and storage medium. Background Technology

[0002] With the increasing popularity of electric vehicles and the development of autonomous driving technology, vehicles are generally equipped with cameras and high-performance computing platforms. Some companies have leveraged these resources to launch Sentinel Mode for vehicle security monitoring. In Sentinel Mode, after the vehicle is turned off and locked, the onboard cameras monitor the surrounding environment in real time. Once a target (such as a person, vehicle, or other unidentified object) is detected within a preset distance threshold, an alarm is triggered to ensure the safety of the vehicle owner's property.

[0003] In practical applications, this method is highly prone to false alarms. For example, in open-air parking lots or on the roadside, there may be objects such as shared electric bikes, fire hydrants, or utility poles around the vehicle; because these objects are relatively close to the vehicle, even if the objects do not pose a threat, the vehicle alarm may still be triggered. On the one hand, car owners will receive a large number of invalid alarm notifications, disrupting their lives; on the other hand, frequent triggering of vehicle alarms increases vehicle energy consumption and affects the vehicle's range.

[0004] Therefore, there is a need to provide a vehicle safety monitoring method that reduces the possibility of false alarms, improves the effectiveness of alarms, and reduces vehicle energy consumption. Summary of the Invention

[0005] In view of this, this application provides a vehicle safety monitoring method, the method comprising: Acquire the current image captured by the vehicle's vision sensor; Target detection is performed on the current image to identify currently suspicious targets in the current image that meet preset conditions; Determine the current location information of the current suspicious target in the current image, and generate a current unique identifier corresponding to the current suspicious target; From a pre-built list of comparison objects, filter historical unique identifiers that are the same as the current unique identifier, and extract the historical location information corresponding to the historical unique identifier; wherein, the historical location information is the location information of the historical suspicious target corresponding to the historical unique identifier in historical images; Based on the current location information and the historical location information, a static object verification is performed on the current suspicious target. When the result of the static object verification indicates that the current suspicious target is a non-stationary target, an alarm operation is triggered.

[0006] Optionally, the current location information includes at least the current coordinates of a specified point in the current detection box, and the historical location information includes at least the historical coordinates of a specified point in a historical detection box; wherein, the current detection box is a detection box in the current image that covers the current suspicious target, and the historical detection box is a detection box in the historical image that covers the historical suspicious target; Correspondingly, the step of performing static object verification on the current suspicious target based on the current location information and the historical location information includes: Determine the Euclidean distance between the current coordinates and the historical coordinates; Determine the outer detection frame that covers the current detection frame and the historical detection frame, as well as the diagonal length of the outer detection frame; The ratio of the Euclidean distance to the diagonal length is used as the relative displacement ratio, and it is determined whether the relative displacement ratio is greater than a preset threshold. If the result is greater than the value, then the result of the static object verification is determined to be that the current suspected target is a non-static target; If the result is not greater than the value, then the result of the static object verification is determined to be that the current suspicious target is a stationary target.

[0007] Optionally, the current position information further includes the first height, first width, and first horizontal and vertical coordinates of the lower left vertex of the current detection box; the historical position information further includes the second height, second width, and second horizontal and vertical coordinates of the lower left vertex of the historical detection box; correspondingly, the step of determining the outer detection box covering the current detection box and the historical detection box includes: The smaller of the first and second x-coordinates is taken as the first target x-coordinate; the smaller of the first and second y-coordinates is taken as the first target y-coordinate. Add the first horizontal coordinate to the first width to obtain the third horizontal coordinate; add the first vertical coordinate to the first height to obtain the third vertical coordinate; add the second horizontal coordinate to the second width to obtain the fourth horizontal coordinate; add the second vertical coordinate to the second height to obtain the fourth vertical coordinate. The larger of the third and fourth horizontal coordinates is taken as the second target horizontal coordinate; the larger of the third and fourth vertical coordinates is taken as the second target vertical coordinate. The first target's x-coordinate and y-coordinate are used as the coordinates of the lower left vertex of the outer detection box, and the second target's x-coordinate and y-coordinate are used as the coordinates of the upper right vertex of the outer detection box. Based on the x-coordinate of the first target and the x-coordinate of the second target, the width of the outer detection frame is determined; based on the y-coordinate of the first target and the y-coordinate of the second target, the height of the outer detection frame is determined. The circumscribed detection box is constructed based on the coordinates of the lower left vertex, the upper right vertex, the width, and the height of the circumscribed detection box.

[0008] Optionally, the preset threshold is the threshold corresponding to the current unique identifier stored in a pre-constructed mapping relationship; correspondingly, after determining the result of the still life verification, the method further includes: Determine the total number of times the alarm operation is triggered by the currently suspicious target; Determine the threat level of the currently suspected target; Based on the total number of occurrences and the threat level, the preset threshold is adjusted to obtain an updated mapping relationship; wherein, both the total number of occurrences and the threat level are negatively correlated with the preset threshold.

[0009] Optionally, after generating the current unique identifier corresponding to the current suspicious target, the method further includes: Determine whether the list of comparison objects is empty; If it is empty, the alarm operation is triggered, and the current information pair consisting of the current unique identifier and the current location information is stored in the comparison object list; If not empty, then determine whether the historical unique identifier exists in the comparison object list; If it exists, proceed to the step of extracting the historical location information corresponding to the historical unique identifier, and replace the historical location information with the current location information; if it does not exist, trigger the alarm operation, and update the comparison object list based on the current information.

[0010] Optionally, the step of updating the comparison object list based on the current information includes: Determine whether the total number of information pairs in the comparison object list is less than a first preset number; If it is less than, then the current information pair is added to the comparison object list; If it is not less than, then determine the timestamp of each information pair; sort each information pair in descending order according to the time difference between the timestamp and the current time; remove the second preset number of information pairs that are ranked first, and add the current information pair to the comparison object list.

[0011] Optionally, the step of triggering the alarm operation includes: The threat level of the current suspicious target is assessed from specified dimensions; wherein the specified dimensions include at least one of the following: the distance between the current suspicious target and the vehicle, the duration of the current suspicious target's stay around the vehicle, the type of action of the current suspicious target, and the vehicle vibration amplitude determined based on a vibration sensor; When the threat level is mild, a first alarm operation is triggered; the first alarm operation includes at least storing video files within a preset time range before and after the first alarm operation is triggered, and performing a warning prompt operation through the vehicle-mounted prompting device; When the threat level is severe, a second alarm operation is triggered; the second alarm operation includes at least sending alarm information to the vehicle owner's mobile terminal.

[0012] This application also provides a vehicle safety monitoring device, the device comprising: The image acquisition module is used to acquire the current image collected by the vehicle-mounted vision sensor; The target recognition module is used to perform target detection on the current image to identify current suspicious targets in the current image that meet preset conditions; The current information determination module is used to determine the current location information of the current suspicious target in the current image, and generate a current unique identifier corresponding to the current suspicious target; The historical information determination module is used to filter historical unique identifiers that are the same as the current unique identifier from a pre-built comparison object list, and extract the historical location information corresponding to the historical unique identifier; wherein, the historical location information is the location information of the historical suspicious target corresponding to the historical unique identifier in the historical image; The alarm determination module is used to perform static object verification on the current suspicious target based on the current location information and the historical location information, and to trigger an alarm operation when the result of the static object verification indicates that the current suspicious target is a non-stationary target.

[0013] This application also provides a vehicle, including: Memory, used to store computer programs; A processor is used to implement the steps of any of the above-described vehicle safety monitoring methods when executing the computer program.

[0014] This application also provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described vehicle safety monitoring methods.

[0015] In summary, this application provides a vehicle safety monitoring method, device, vehicle, and storage medium, which acquires a current image collected by an onboard vision sensor; performs target detection on the current image to identify current suspicious targets that meet preset conditions in the current image; determines the current location information of the current suspicious target in the current image and generates a current unique identifier corresponding to the current suspicious target; filters historical unique identifiers that are the same as the current unique identifier from the comparison object list and extracts the historical location information corresponding to the historical unique identifier, wherein the historical location information is the location information of the historical suspicious target corresponding to the historical unique identifier in the historical image; performs static verification on the current suspicious target based on the current location information and the historical location information; and triggers an alarm operation when the result of the static verification indicates that the current suspicious target is a non-stationary target.

[0016] As can be seen, this application will only trigger the alarm operation when the target meets the preset conditions and is not a stationary target. This eliminates false alarms caused by stationary suspicious targets with extremely low computing power, improves the effectiveness of the alarm, avoids the interference of invalid alarms on the car owner, and reduces the vehicle's power consumption, thus ensuring the vehicle's range. Attached Figure Description

[0017] Figure 1 This is a first flowchart illustrating a vehicle safety monitoring method provided in this application; Figure 2 A schematic diagram of an external detection frame provided in this application; Figure 3 This is a schematic diagram of the second process of a vehicle safety monitoring method provided in this application; Figure 4 This application provides a structural schematic diagram of a vehicle safety monitoring device; Figure 5 This is a structural schematic diagram of a vehicle provided in this application. Detailed Implementation

[0018] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0019] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information.

[0020] Please refer to Figure 1 , Figure 1 This application provides a first flowchart illustrating a vehicle safety monitoring method, which includes: S101. Acquire the current image captured by the vehicle-mounted vision sensor.

[0021] The aforementioned vehicle-mounted vision sensors can be surround-view cameras deployed on the vehicle. This application does not impose any particular limitations on the number or deployment location of the surround-view cameras. For example, one or two surround-view cameras can be deployed on the left front, right front, left rear, and right rear sides of the vehicle to ensure full field of view coverage and achieve comprehensive monitoring of people, vehicles, or other unidentified objects around the vehicle.

[0022] When it is determined that vehicle safety monitoring is required, the current image collected by the aforementioned vehicle-mounted vision sensor is acquired. As an optional embodiment, the step of acquiring the current image collected by the vehicle-mounted vision sensor is initiated when the vehicle is detected to be turned off and locked, or when a control command triggered by the user to enter sentry mode is detected.

[0023] S102. Perform target detection on the current image to identify current suspicious targets in the current image that meet preset conditions.

[0024] In this application, a deep learning model (such as the YOLO model) can be used to detect targets in the current image and determine the type of the target (e.g., pedestrians, vehicles, non-motorized vehicles, or other unknown objects). Based on the current image and the image sequence containing the current image, the behavior and actions of the target are determined. Then, it is determined whether the detected target meets preset conditions, and targets that meet the preset conditions are designated as current suspected targets. The aforementioned preset conditions can be set according to actual needs, and this application does not impose any special limitations on them.

[0025] As an optional embodiment, the preset conditions include any one or more of the following: the target is not in a pre-set whitelist, the distance between the target and the vehicle is less than a pre-set distance threshold, the target's lingering time around the vehicle exceeds a pre-set duration threshold, and the target performs a predefined destructive action (including but not limited to touching or striking the vehicle). It should be noted that a deep learning model can be used to determine the target's category and appearance characteristics, and based on the target's category and appearance characteristics, it can be determined whether the target is in the whitelist. This application does not specifically limit the specific implementation methods for determining the aforementioned distance, lingering time, and action recognition.

[0026] S103. Determine the current location information of the current suspicious target in the current image, and generate the current unique identifier corresponding to the current suspicious target.

[0027] In this application, upon determining the presence of a suspicious target in the current image, an alarm is not triggered directly. Instead, a still-object verification is performed on the suspicious target. An alarm is only triggered if the still-object verification indicates that the suspicious target is not stationary. Therefore, in this application, an alarm is triggered only if the target meets both the preset conditions and is not stationary.

[0028] To achieve the above objectives, the current location information of the current suspicious target in the current image and the current unique identifier of the current suspicious target are first determined so that the historical location information of the current suspicious target can be extracted based on the current unique identifier. By comparing the historical location information and the current location information, it can be determined whether the current suspicious target is a stationary target.

[0029] The aforementioned current location information represents the position of the current suspicious target in the current image. For example, the smallest rectangle covering the current suspicious target in the current image is taken as the current detection box; the coordinates of specified points (such as the center point or each vertex) in the current detection box are taken as the current location information.

[0030] The aforementioned unique identifier is determined based on the appearance characteristics of the currently suspected target. This application does not specifically limit the particular method for calculating the unique identifier based on appearance characteristics. As an optional embodiment, low-level visual features of the target are extracted from different dimensions to obtain multi-dimensional feature maps (including but not limited to color histograms, gradient orientation histograms, and LBP maps (Local Binary Patterns)). The multi-dimensional feature maps are then fused, and the fused high-dimensional features are mapped to low-dimensional, compact binary codes using a hash function to obtain the target's unique identifier.

[0031] It is understandable that when targets identified from different images correspond to the same unique identifier, it indicates that the two targets are actually the same target; the difference in the positional information of the target in different images can characterize the range of motion of the target.

[0032] S104. From the pre-built comparison object list, filter the historical unique identifiers that are the same as the current unique identifier, and extract the historical location information corresponding to the historical unique identifier; wherein, the historical location information is the location information of the historical suspicious target corresponding to the historical unique identifier in the historical image.

[0033] In this application, a comparison object list is maintained. This list stores the unique identifiers and location information of suspicious targets that meet preset conditions during vehicle safety monitoring. The maintenance method for the comparison object list will be described in subsequent embodiments and will not be repeated here. Furthermore, this list is not cleared when the system stops or deinitializes, so that it can remember the information of suspicious targets that previously appeared around the vehicle upon the next startup.

[0034] As mentioned earlier, when targets identified from different images correspond to the same unique identifier, it indicates that these two targets are actually the same target. Based on this, from the comparison object list, historical unique identifiers that are identical to the current unique identifier are selected. The historical suspicious targets represented by these historical unique identifiers are actually the same target as the current suspicious target. Furthermore, from the comparison object list, the historical location information corresponding to the historical unique identifiers is extracted. This historical location information is equivalent to the past location information of the current suspicious target. The difference between the current location information and the historical location information can characterize the range of motion of the current suspicious target.

[0035] S105. Based on the current location information and historical location information, perform static object verification on the current suspicious target. When the result of the static object verification indicates that the current suspicious target is a non-stationary target, trigger an alarm operation.

[0036] Based on the above analysis, by comparing the current location information with historical location information, it can be determined whether the current suspicious target is a stationary target, that is, a static object verification is performed on the current suspicious target. The specific process of static object verification will be described in subsequent embodiments, and will not be repeated here. In summary, when the current suspicious target is determined to be a stationary target, no alarm operation is triggered to avoid false alarms; only when the current suspicious target is determined to be a non-stationary target is an alarm operation triggered to ensure the safety of the vehicle owner's property. Fire hydrants, utility poles, and other parked vehicles in open-air parking lots or roadsides can be effectively eliminated to avoid triggering vehicle alarms.

[0037] In summary, this application provides a vehicle safety monitoring method. First, it acquires the current image from an onboard vision sensor and performs target detection on the image to identify currently suspicious targets that meet preset conditions. It then determines the current location information of the suspicious target in the current image and generates a current unique identifier corresponding to the target. Next, it filters historical unique identifiers that match the current unique identifier from a comparison list and extracts the historical location information corresponding to each identifier. This historical location information represents the location of the historical suspicious target corresponding to the historical unique identifier in historical images. Based on the current and historical location information, it performs static object verification on the suspicious target. Finally, it triggers an alarm operation when the static object verification indicates that the suspicious target is not stationary. Therefore, this application only triggers an alarm operation when the target meets both preset conditions and is not stationary. This eliminates false alarms caused by stationary suspicious targets with extremely low computational cost, improving the effectiveness of the alarm, preventing the driver from being disturbed by invalid alarms, reducing vehicle power consumption, and ensuring the vehicle's range.

[0038] Based on the above embodiments: The process of still life verification will be explained in detail below.

[0039] As an optional embodiment, the current location information includes at least the current coordinates of the specified point in the current detection box, and the historical location information includes at least the historical coordinates of the specified point in the historical detection box; wherein, the current detection box is the detection box in the current image that covers the current suspicious target, and the historical detection box is the detection box in the current image that covers the historical suspicious target; Correspondingly, the steps for static object verification of the current suspicious target based on current and historical location information include: Determine the Euclidean distance between the current coordinates and the historical coordinates; Determine the outer detection box that covers the current detection box and the historical detection boxes, as well as the diagonal length of the outer detection box; The ratio of Euclidean distance to diagonal length is used as the relative displacement ratio, and it is determined whether the relative displacement ratio is greater than a preset threshold. If the value is greater than 1, then the result of the static object verification is determined to be that the current suspected target is a non-static target. If the result is not greater than the value, then the result of the static object verification is determined to be that the current suspicious target is a stationary target.

[0040] In this embodiment, the detection box that can cover the current suspicious target in the current image is taken as the current detection box; and a standard coordinate system is established in advance, and the current coordinates of the specified point (e.g., the center point) in the current detection box are taken as the current position information; similarly, the detection box that can cover the historical suspicious target in the historical image is taken as the historical detection box, and the historical coordinates of the specified point (e.g., the center point) in the historical detection box in the standard coordinate system are taken as the historical position information.

[0041] After determining the current and historical coordinates of the suspected target, the Euclidean distance between the current and historical coordinates is determined. Furthermore, this embodiment does not directly determine whether the suspected target is stationary by judging whether the Euclidean distance is greater than a preset threshold. Instead, it provides a discrimination method based on the relative displacement ratio, which verifies the motion amplitude of the suspected target in time sequence. This filters out suspected targets with only slight shaking or that are stationary, triggering vehicle alarms only for suspected targets that actually produce significant displacement, thereby reducing false alarms.

[0042] Specifically, in this embodiment, a detection frame that covers both the current and historical detection frames is designated as an outer detection frame, and the diagonal length of the outer detection frame is determined. Please refer to... Figure 2 , Figure 2 This is a schematic diagram of an external detection frame provided in this application. Figure 2 The red rectangle represents the current detection box, and point A represents the center point of the current detection box; the green rectangle represents the historical detection box, and point B represents the center point of the historical detection box; the black rectangle is the smallest bounding rectangle of the current detection box and the historical detection boxes, that is, the bounding detection box, and the black dashed line is the diagonal of the bounding detection box.

[0043] The ratio of the Euclidean distance between the current coordinates and the historical coordinates to the diagonal length is used as the relative displacement ratio. Since the relative displacement ratio is insensitive to small changes in the Euclidean distance, determining whether a suspected target is stationary by checking if the relative displacement ratio exceeds a preset threshold avoids interference from small target movements and minor vibrations of the visual sensor, thus enhancing the reliability of still object verification. Furthermore, the method of still object verification based on the relative displacement ratio is applicable to both large and small targets, unaffected by changes in target attitude, and possesses versatility.

[0044] The process of determining the external detection frame is explained below.

[0045] As an optional embodiment, the current position information further includes the first height, first width, and first x-coordinate and first y-coordinate of the lower left vertex of the current detection box; the historical position information further includes the second height, second width, and second x-coordinate and second y-coordinate of the lower left vertex of the historical detection box; correspondingly, the step of determining the outer detection box covering the current detection box and the historical detection box includes: The smaller of the first and second x-coordinates is taken as the first target x-coordinate; the smaller of the first and second y-coordinates is taken as the first target y-coordinate. Add the first horizontal coordinate to the first width to get the third horizontal coordinate; add the first vertical coordinate to the first height to get the third vertical coordinate; add the second horizontal coordinate to the second width to get the fourth horizontal coordinate; add the second vertical coordinate to the second height to get the fourth vertical coordinate. The larger of the third and fourth x-coordinates is taken as the x-coordinate of the second target; the larger of the third and fourth y-coordinates is taken as the y-coordinate of the second target. Use the x-coordinate and y-coordinate of the first target as the coordinates of the lower left vertex of the bounding box, and use the x-coordinate and y-coordinate of the second target as the coordinates of the upper right vertex of the bounding box. The width of the circumscribed detection box is determined based on the x-coordinate of the first target and the x-coordinate of the second target; the height of the circumscribed detection box is determined based on the y-coordinate of the first target and the y-coordinate of the second target. Construct the bounding box based on the coordinates of the lower left vertex, the upper right vertex, the width, and the height.

[0046] In this embodiment, the current location information also includes the first height of the current detection box. First width and the coordinates of the lower left vertex ,in The first horizontal coordinate mentioned above, The first vertical coordinate is mentioned above; historical location information also includes the second height of the historical detection box. Second width and the coordinates of the lower left vertex ,in The second horizontal axis mentioned above, This is the second ordinate mentioned above.

[0047] Therefore, in this embodiment, the smallest bounding rectangle that can cover both the current detection box and the historical detection boxes is used as the bounding detection box. Specifically, the smaller of the first and second horizontal coordinates is determined to obtain the first target horizontal coordinate. Determine the smaller of the first and second ordinates to obtain the first target ordinate. The points corresponding to the x-coordinate of the first target and the y-coordinate of the second target. This refers to the lower left vertex of the aforementioned outer detection frame.

[0048] Add the first horizontal coordinate to the first width to obtain the third horizontal coordinate. Add the first ordinate to the first height to obtain the third ordinate. The point corresponding to the third horizontal and third vertical coordinates is the top right vertex of the current detection box. Similarly, the fourth horizontal coordinate is obtained by adding the second horizontal coordinate to the second width. Add the second ordinate to the second height to obtain the fourth ordinate. The point corresponding to the fourth horizontal coordinate and the fourth vertical coordinate is the upper right corner of the historical detection box.

[0049] Based on this, the larger of the third and fourth x-coordinates will be used as the x-coordinate of the second target. The larger of the third and fourth ordinates will be used as the ordinate of the second target. The points corresponding to the x-coordinate and y-coordinate of the second target. This refers to the top-right vertex of the circumscribed bounding box. At this point, the coordinates of the bottom-left and top-right vertices of the circumscribed bounding box are known. The width of the circumscribed bounding box can be expressed as... The height of the outer detection frame can be expressed as The bounding box can be constructed based on the coordinates of the two vertices and the width and height of the bounding box.

[0050] In this embodiment, the width of the external detection frame can be expressed as: The height of the outer detection frame can be expressed as Its diagonal length If the Euclidean distance between the center point of the current detection box and the center point of the historical detection boxes is expressed as... The relative displacement ratio can then be expressed as: .like If so, the current suspicious target is determined to be a non-stationary target, and an alarm operation is triggered; if If the current suspicious target is determined to be a stationary target, no alarm operation will be triggered to avoid false alarms; among which, This is a preset threshold.

[0051] As an optional embodiment, the preset threshold is the threshold corresponding to the current unique identifier stored in the pre-built mapping relationship; correspondingly, after determining the result of the still life verification, the following steps are also included: Determine the total number of alarm operations triggered by the current suspicious target; Determine the threat level of the currently suspected target; Based on the total number of occurrences and the threat level, the preset thresholds are adjusted to obtain the updated mapping relationship; where both the total number of occurrences and the threat level are negatively correlated with the preset thresholds.

[0052] In this embodiment, a mapping relationship is pre-established, which is a correspondence between a unique identifier and a preset threshold, meaning that there is a one-to-one correspondence between a suspicious target and a preset threshold. Furthermore, the preset threshold is not a static, fixed value, but rather changes dynamically based on the total number of alarm operations triggered by the suspicious target and the threat level of the suspicious target.

[0053] Specifically, after determining the static verification results of the current suspicious target, the total number of times the current suspicious target triggers an alarm operation is determined; the higher the total number of times, the more likely the suspicious target is to pose a threat to the vehicle. Similarly, the threat level of the current suspicious target is also determined; the higher the threat level, the more likely the suspicious target is to pose a threat to the vehicle. Therefore, this embodiment adjusts a preset threshold based on the total number of times and the threat level, and both the total number of times and the threat level are negatively correlated with the preset threshold; that is, the higher the total number of times or the threat level, the lower the preset threshold, making it easier for the suspicious target to trigger an alarm operation.

[0054] Based on this, if the current suspicious target reappears, the adjusted preset threshold will be used to perform static verification on the current suspicious target, so as to improve the sensitivity to suspicious targets that frequently trigger alarm operations and have a high threat level, and maintain vehicle safety.

[0055] Furthermore, this embodiment does not specifically limit the method for determining the threat level of a current suspected target. As an optional embodiment, the threat level of a current suspected target is assessed from specified dimensions; wherein, the specified dimensions include at least one of the following: the distance between the current suspected target and the vehicle, the duration of the current suspected target's presence around the vehicle, the type of action of the current suspected target (e.g., determining the action type based on an image sequence containing the current image), and the vehicle vibration amplitude determined by a vibration sensor. The smaller the distance, the higher the threat level; the longer the presence, the higher the threat level; when the action type belongs to a pre-set threat action type (e.g., knocking on the vehicle), the higher the threat level; the greater the vehicle vibration amplitude, the higher the threat level.

[0056] As an optional embodiment, after generating the current unique identifier corresponding to the current suspicious target, the method further includes: Determine if the comparison object list is empty; If it is empty, an alarm operation is triggered, and the current information pair consisting of the current unique identifier and the current location information is stored in the comparison object list; If not empty, then determine whether a historical unique identifier exists in the comparison object list; If it exists, proceed to the step of extracting the historical location information corresponding to the historical unique identifier and replacing the historical location information with the current location information; if it does not exist, trigger an alarm operation and update the comparison object list based on the current information.

[0057] Please refer to Figure 3 , Figure 3 This is a schematic diagram of the second process of a vehicle safety monitoring method provided in this application. After generating the current unique identifier of the current suspicious target, it is first determined whether the comparison object list is empty. If the comparison object list is empty, there is no need to perform static verification on the current suspicious target; the alarm operation is triggered directly to avoid overlooking situations that pose a threat to the vehicle and to ensure vehicle safety. Furthermore, the unique identifier and current location information of the current suspicious target constitute a current information pair, which is stored in the comparison object list. This allows for static verification of the current suspicious target when it reappears, and a determination of whether to trigger an alarm operation based on the static verification, ensuring the effectiveness of the alarm.

[0058] If the comparison object list is not empty, determine whether a historical unique identifier exists in the list that is identical to the current unique identifier. If it exists, perform static verification on the current suspicious target based on the historical location information corresponding to the historical unique identifier, and replace the historical location information with the current location information to ensure that the latest location information of the suspicious target is always stored in the comparison object list, guaranteeing the correctness of the static verification. If it does not exist, there is no need to perform static verification on the current suspicious target; trigger the alarm directly to avoid overlooking situations that could threaten the vehicle and ensure vehicle safety. At the same time, update the comparison object list based on the current information so that when the current suspicious target reappears, static verification can be performed on the current suspicious target, and the triggering of the alarm can be determined based on the static verification, ensuring the effectiveness of the alarm.

[0059] In summary, in this embodiment, if there are no historical suspicious targets in the comparison object list, an alarm operation is directly triggered to ensure vehicle safety; and regardless of whether an alarm operation is triggered, the latest location information of the suspicious targets is always stored in the comparison object list to ensure the correctness of subsequent static object verification.

[0060] The process of updating the comparison object list is explained in detail below.

[0061] As an optional embodiment, the step of updating the comparison object list based on the current location includes: Determine whether the total number of information pairs in the comparison object list is less than a first preset number; If the value is less than the value, the current information pair is added to the comparison object list; If the difference is not less than the time difference, determine the timestamp of each information pair; sort the information pairs in descending order according to the time difference between the timestamp and the current time; remove the second preset number of information pairs that are ranked first, and add the current information pair to the comparison object list.

[0062] In this embodiment, the comparison object list has a maximum capacity limit, meaning it can store a maximum of a first preset number of information pairs. Updating the comparison object list based on the current information pair indicates that new information pairs need to be added. Therefore, it is first determined whether the total number of information pairs in the comparison object list is less than the first preset number. If it is less, the current information pair is directly added to the comparison object list. If it is not less, information pairs with longer storage times are removed, and then the current information pair is added to the comparison object list. Specifically, the timestamp of each information pair is determined; based on the time difference between the timestamp and the current time, the information pairs are sorted in descending order; the information pairs that appear earlier in the sorted list have been stored for longer. Therefore, a second preset number of information pairs that appear earlier in the sorted list are removed. Both the first and second preset numbers can be set according to actual needs, and this embodiment does not impose any particular limitation on them.

[0063] In summary, in this embodiment, the latest location information and unique identifier of the suspicious target are stored in the comparison object list to avoid errors in the results of static object verification; and only information pairs of recently appearing suspicious targets are stored to save storage resources.

[0064] As an optional embodiment, the steps for triggering an alarm operation include: The threat level of the current suspicious target is assessed from the specified dimensions; wherein the specified dimensions include at least one of the following: the distance between the current suspicious target and the vehicle, the duration of the current suspicious target's stay around the vehicle, the type of action of the current suspicious target, and the amplitude of vehicle vibration determined based on vibration sensors; When the threat level is mild, the first alarm operation is triggered; the first alarm operation includes at least storing video files within a preset time range before and after the first alarm operation is triggered, and performing a warning prompt operation through the vehicle-mounted prompting device; When the threat level is severe, a second alarm operation is triggered; the second alarm operation includes at least sending an alarm message to the vehicle owner's mobile terminal.

[0065] In this embodiment, the threat level of the current suspicious target is first assessed from the aforementioned specified dimensions in order to take differentiated alarm actions based on the threat level. The specified dimensions include at least one of the following: the distance between the current suspicious target and the vehicle, the duration of the current suspicious target's stay around the vehicle, the type of action of the current suspicious target, and the vehicle vibration amplitude determined based on vibration sensors. The specific dimensions can be selected according to actual needs, and this embodiment does not impose any particular limitations.

[0066] This embodiment does not specifically limit the rules for determining the threat level. Taking the distance between the current suspicious target and the vehicle as an example, if the distance is greater than a first distance threshold and less than a second distance threshold, the threat level is determined to be mild; if the distance is greater than or equal to the second distance threshold, the threat level is determined to be moderate. The same principle applies when the specified dimensions include the duration of stay and the amplitude of vehicle vibration, which will not be elaborated in this embodiment. As for the case where the specified dimensions include the action type of the current suspicious target, a first action type corresponding to mild (e.g., contacting the vehicle) and a second action type corresponding to severe (e.g., knocking on the vehicle) can be predefined; the threat level is determined based on the degree of conformity between the current suspicious target's action type and the first and second action types.

[0067] If the threat level is minor, the first alarm operation is triggered. The first alarm operation includes storing video files within a preset time range before and after the first alarm operation is triggered, providing crucial video evidence for subsequent liability determination, claims processing, or reporting to the police. The first alarm operation may also include issuing warnings via the vehicle's in-vehicle alert system. For example, controlling the vehicle's headlights to flash and displaying a "recording video" message on the central control screen to provide a warning and ensure vehicle safety.

[0068] If the threat level is severe, a second alarm operation will be triggered. The second alarm operation includes at least sending an alarm message to the vehicle owner's mobile terminal so that the vehicle owner can be informed of the relevant situation in a timely manner, maintain vehicle safety, and minimize the interference of frequent alarm messages to the vehicle owner.

[0069] In summary, this embodiment assesses the threat level of current suspicious targets from various specified dimensions and takes differentiated alarm actions based on the threat level. This ensures vehicle safety, saves vehicle energy consumption, reduces disruption to the driver's life, and provides a user-friendly alarm operation.

[0070] Please refer to Figure 4 , Figure 4 This application provides a schematic diagram of the structure of a vehicle safety monitoring device, which includes: Image acquisition module 401 is used to acquire the current image collected by the vehicle vision sensor; The target recognition module 402 is used to perform target detection on the current image to identify currently suspicious targets in the current image that meet preset conditions; The current information determination module 403 is used to determine the current location information of the current suspicious target in the current image and generate the current unique identifier corresponding to the current suspicious target; The historical information determination module 404 is used to filter historical unique identifiers that are the same as the current unique identifier from a pre-built comparison object list, and extract the historical location information corresponding to the historical unique identifier; wherein, the historical location information is the location information of the historical suspicious target corresponding to the historical unique identifier in the historical image; The alarm determination module 405 is used to perform static object verification on the current suspicious target based on the current location information and historical location information, and to trigger an alarm operation when the result of the static object verification indicates that the current suspicious target is a non-stationary target.

[0071] For a detailed description of the vehicle safety monitoring device provided in this application, please refer to the embodiments of the vehicle safety monitoring method described above; this application will not repeat the details here.

[0072] Based on the above embodiments: As an optional embodiment, the current location information includes at least the current coordinates of the specified point in the current detection box, and the historical location information includes at least the historical coordinates of the specified point in the historical detection box; wherein, the current detection box is the detection box in the current image that covers the current suspicious target, and the historical detection box is the detection box in the historical image that covers the historical suspicious target; Correspondingly, the alarm determination module 405 includes: The distance determination module is used to determine the Euclidean distance between the current coordinates and the historical coordinates; The diagonal length determination module is used to determine the outer detection box that covers the current detection box and the historical detection boxes, as well as the diagonal length of the outer detection box; The ratio determination module is used to take the ratio of Euclidean distance to diagonal length as the relative displacement ratio; The result determination module is used to determine whether the relative displacement ratio is greater than a preset threshold. If it is greater, the result of the static object verification is determined to be that the current suspicious target is a non-stationary target, and an alarm operation is triggered. If it is not greater, the result of the static object verification is determined to be that the current suspicious target is a stationary target.

[0073] As an optional embodiment, the current position information also includes the first height, first width, and first horizontal and vertical coordinates of the lower left vertex of the current detection box; the historical position information also includes the second height, second width, and second horizontal and vertical coordinates of the lower left vertex of the historical detection box. Correspondingly, the diagonal length determination module includes: The first determining module is used to take the smaller of the first and second horizontal coordinates as the first target horizontal coordinate; and to take the smaller of the first and second vertical coordinates as the first target vertical coordinate. The second determining module is used to add the first horizontal coordinate to the first width to obtain the third horizontal coordinate; add the first vertical coordinate to the first height to obtain the third vertical coordinate; add the second horizontal coordinate to the second width to obtain the fourth horizontal coordinate; and add the second vertical coordinate to the second height to obtain the fourth vertical coordinate. The third determining module is used to take the larger of the third and fourth horizontal coordinates as the second target horizontal coordinate; and to take the larger of the third and fourth vertical coordinates as the second target vertical coordinate. The fourth determination module is used to use the x-coordinate and y-coordinate of the first target as the coordinates of the lower left vertex of the outer detection box, and the x-coordinate and y-coordinate of the second target as the coordinates of the upper right vertex of the outer detection box. The fifth determining module is used to determine the width of the outer detection frame based on the x-coordinate of the first target and the x-coordinate of the second target; and to determine the height of the outer detection frame based on the y-coordinate of the first target and the y-coordinate of the second target. The detection box construction module is used to construct an outer detection box based on the coordinates of the lower left vertex, the upper right vertex, the width, and the height of the outer detection box. The sixth determination module is used to determine the diagonal length of the external detection frame.

[0074] As an optional embodiment, the preset threshold is the threshold corresponding to the current unique identifier stored in a pre-built mapping relationship; the device further includes: The fifth determination module is used to determine the total number of alarm operations triggered by the current suspicious target and the threat level of the current suspicious target after determining the results of the static object verification. The threshold adjustment module is used to adjust the preset threshold based on the total number of times and the threat level to obtain the updated mapping relationship; wherein, both the total number of times and the threat level are negatively correlated with the preset threshold.

[0075] As an optional embodiment, the device further includes: The first list verification module is used to determine whether the comparison object list is empty after generating the current unique identifier corresponding to the current suspicious target; if it is empty, the first operation module is triggered; if it is not empty, the second list verification module is triggered. The first operation module is used to trigger the alarm operation and store the current information pair consisting of the current unique identifier and the current location information into the comparison object list; The second list verification module is used to determine whether a historical unique identifier exists in the comparison object list; if it exists, the historical information determination module 404 is triggered, and the historical location information is replaced with the current location information; if it does not exist, the second operation module is triggered. The second operation module is used to trigger alarm operations and update the comparison object list based on the current information.

[0076] As an optional embodiment, the second operation module includes: Alarm triggering module, used to trigger alarm operations; The quantity verification module is used to determine whether the total number of information pairs in the comparison object list is less than a first preset number; if it is less, the first update module is triggered; if it is not less, the second update module is triggered. The first update module is used to add the current information pair to the comparison object list; The second update module is used to determine the timestamp of each information pair; sort the information pairs in descending order according to the time difference between the timestamp and the current time; remove the top-ranked information pairs and add the current information pair to the comparison object list.

[0077] As an optional embodiment, the alarm determination module 405 includes: The still object verification module is used to perform still object verification on currently suspicious targets based on current location information and historical location information; The threat level determination module is used to assess the threat level of the current suspicious target from specified dimensions when the verification result indicates that the current suspicious target is a non-stationary target; when the threat level is low, the first alarm operation module is triggered; when the threat level is high, the second alarm operation module is triggered; wherein, the specified dimensions include at least one of the following: the distance between the current suspicious target and the vehicle, the duration of the current suspicious target's stay around the vehicle, the type of action of the current suspicious target, and the vehicle vibration amplitude determined based on the vibration sensor; The first alarm operation module is used to trigger the first alarm operation; the first alarm operation includes at least storing video files within a preset time range before and after triggering the first alarm operation, and performing a warning prompt operation through the vehicle-mounted prompting device; The second alarm operation module is used to trigger the second alarm operation; the second alarm operation includes at least sending alarm information to the vehicle owner's mobile terminal.

[0078] Please refer to Figure 5 , Figure 5 This application provides a structural schematic diagram of a vehicle, which includes: Memory 501 is used to store computer programs; The processor 502 is used to execute computer programs to implement the steps of any of the above-described vehicle safety monitoring methods.

[0079] It should be noted that the vehicles provided in the embodiments of this application may include, but are not limited to, sedans, sport utility vehicles (SUVs), multi-purpose vehicles (MPVs), off-road vehicles, pickup trucks, or other power-driven, non-rail-borne vehicles.

[0080] The aforementioned memory 501 includes all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROMs and DVD-ROMs. The processor and memory may be supplemented by or incorporated into dedicated logic circuitry.

[0081] In some embodiments, the vehicle may further include a display screen, input / output interfaces, communication interfaces, a power supply, and a communication bus, etc. Those skilled in the art will understand that... Figure 5 The structure shown does not constitute a limitation on the vehicle and may include more components than those shown.

[0082] For a detailed description of the vehicle provided in this application, please refer to the embodiments of the vehicle safety monitoring method described above; this application will not repeat the details here.

[0083] This application also provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described vehicle safety monitoring methods.

[0084] For a detailed description of the storage medium provided in this application, please refer to the embodiments of the vehicle safety monitoring method described above; this application will not repeat the details here.

[0085] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0086] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0087] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A vehicle safety monitoring method, characterized in that, The method includes: Acquire the current image captured by the vehicle's vision sensor; Target detection is performed on the current image to identify currently suspicious targets in the current image that meet preset conditions; Determine the current location information of the current suspicious target in the current image, and generate a current unique identifier corresponding to the current suspicious target; From a pre-built list of comparison objects, filter historical unique identifiers that are the same as the current unique identifier, and extract the historical location information corresponding to the historical unique identifier; wherein, the historical location information is the location information of the historical suspicious target corresponding to the historical unique identifier in historical images; Based on the current location information and the historical location information, a static object verification is performed on the current suspicious target. When the result of the static object verification indicates that the current suspicious target is a non-stationary target, an alarm operation is triggered.

2. The vehicle safety monitoring method as described in claim 1, characterized in that, The current location information includes at least the current coordinates of a specified point in the current detection box, and the historical location information includes at least the historical coordinates of a specified point in the historical detection box; wherein, the current detection box is the detection box in the current image that covers the current suspicious target, and the historical detection box is the detection box in the historical image that covers the historical suspicious target; Correspondingly, the step of performing static object verification on the current suspicious target based on the current location information and the historical location information includes: Determine the Euclidean distance between the current coordinates and the historical coordinates; Determine the outer detection frame that covers the current detection frame and the historical detection frame, as well as the diagonal length of the outer detection frame; The ratio of the Euclidean distance to the diagonal length is used as the relative displacement ratio, and it is determined whether the relative displacement ratio is greater than a preset threshold. If the result is greater than the value, then the result of the static object verification is determined to be that the current suspected target is a non-static target; If the result is not greater than the value, then the result of the static object verification is determined to be that the current suspicious target is a stationary target.

3. The vehicle safety monitoring method as described in claim 2, characterized in that, The current position information also includes the first height, first width, and first horizontal and vertical coordinates of the lower left vertex of the current detection box; the historical position information also includes the second height, second width, and second horizontal and vertical coordinates of the lower left vertex of the historical detection box. Correspondingly, the step of determining the outer detection box covering the current detection box and the historical detection boxes includes: The smaller of the first and second x-coordinates is taken as the first target x-coordinate; the smaller of the first and second y-coordinates is taken as the first target y-coordinate. Add the first horizontal coordinate to the first width to obtain the third horizontal coordinate; add the first vertical coordinate to the first height to obtain the third vertical coordinate; add the second horizontal coordinate to the second width to obtain the fourth horizontal coordinate; add the second vertical coordinate to the second height to obtain the fourth vertical coordinate. The larger of the third and fourth horizontal coordinates is taken as the second target horizontal coordinate; the larger of the third and fourth vertical coordinates is taken as the second target vertical coordinate. The first target's x-coordinate and y-coordinate are used as the coordinates of the lower left vertex of the outer detection box, and the second target's x-coordinate and y-coordinate are used as the coordinates of the upper right vertex of the outer detection box. Based on the x-coordinate of the first target and the x-coordinate of the second target, the width of the outer detection frame is determined; based on the y-coordinate of the first target and the y-coordinate of the second target, the height of the outer detection frame is determined. The circumscribed detection box is constructed based on the coordinates of the lower left vertex, the upper right vertex, the width, and the height of the circumscribed detection box.

4. The vehicle safety monitoring method as described in claim 2, characterized in that, The preset threshold is the threshold corresponding to the current unique identifier stored in a pre-constructed mapping relationship; correspondingly, after determining the result of the still life verification, the method further includes: Determine the total number of times the alarm operation is triggered by the currently suspicious target; Determine the threat level of the currently suspected target; Based on the total number of occurrences and the threat level, the preset threshold is adjusted to obtain an updated mapping relationship; wherein, both the total number of occurrences and the threat level are negatively correlated with the preset threshold.

5. The vehicle safety monitoring method as described in claim 1, characterized in that, After generating the current unique identifier corresponding to the current suspicious target, the method further includes: Determine whether the list of comparison objects is empty; If it is empty, the alarm operation is triggered, and the current information pair consisting of the current unique identifier and the current location information is stored in the comparison object list; If not empty, then determine whether the historical unique identifier exists in the comparison object list; If it exists, proceed to the step of extracting the historical location information corresponding to the historical unique identifier, and replace the historical location information with the current location information; if it does not exist, trigger the alarm operation, and update the comparison object list based on the current information.

6. The vehicle safety monitoring method as described in claim 5, characterized in that, The step of updating the comparison object list based on the current information includes: Determine whether the total number of information pairs in the comparison object list is less than a first preset number; If it is less than, then the current information pair is added to the comparison object list; If it is not less than, then determine the timestamp of each information pair; sort each information pair in descending order according to the time difference between the timestamp and the current time; remove the second preset number of information pairs that are ranked first, and add the current information pair to the comparison object list.

7. The vehicle safety monitoring method as described in claim 1, characterized in that, The steps for triggering the alarm operation include: The threat level of the current suspicious target is assessed from specified dimensions; wherein the specified dimensions include at least one of the following: the distance between the current suspicious target and the vehicle, the duration of the current suspicious target's stay around the vehicle, the type of action of the current suspicious target, and the vehicle vibration amplitude determined based on a vibration sensor; When the threat level is mild, a first alarm operation is triggered; the first alarm operation includes at least storing video files within a preset time range before and after the first alarm operation is triggered, and performing a warning prompt operation through the vehicle-mounted prompting device; When the threat level is severe, a second alarm operation is triggered; the second alarm operation includes at least sending alarm information to the vehicle owner's mobile terminal.

8. A vehicle safety monitoring device, characterized in that, The device includes: The image acquisition module is used to acquire the current image collected by the vehicle-mounted vision sensor; The target recognition module is used to perform target detection on the current image to identify current suspicious targets in the current image that meet preset conditions; The current information determination module is used to determine the current location information of the current suspicious target in the current image, and generate a current unique identifier corresponding to the current suspicious target; The historical information determination module is used to filter historical unique identifiers that are the same as the current unique identifier from a pre-built comparison object list, and extract the historical location information corresponding to the historical unique identifier; wherein, the historical location information is the location information of the historical suspicious target corresponding to the historical unique identifier in the historical image; The alarm determination module is used to perform static object verification on the current suspicious target based on the current location information and the historical location information, and to trigger an alarm operation when the result of the static object verification indicates that the current suspicious target is a non-stationary target.

9. A vehicle, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the vehicle safety monitoring method as described in any one of claims 1 to 7 when executing the computer program.

10. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the vehicle safety monitoring method as described in any one of claims 1 to 7.

Images