Alerting method, device, storage medium and program product for vehicle sentry mode

Abstract

The application provides a vehicle sentry mode alarm method, device, storage medium and program product, relates to the technical field of vehicle safety, and comprises the following steps: acquiring multi-modal sensing data in a vehicle sentry mode; the multi-modal sensing data comprises state data of the vehicle, basic environment data around the vehicle, and video image data; determining a target risk level of the vehicle based on the state data and the basic environment data; the target risk level is one of multiple risk levels of a vehicle parking scene, and the initial alarm reference threshold sets corresponding to the multiple risk levels are different from each other; determining personnel behavior data of the environment around the vehicle based on the video image data, and dynamically adjusting the initial alarm reference threshold set corresponding to the target risk level based on the personnel behavior data to obtain an adjusted alarm reference threshold set; and triggering vehicle sentry alarm based on the personnel behavior data and the adjusted alarm reference threshold set. The application can avoid the phenomenon of frequent false alarms of the vehicle sentry.

Description

Technical Field

[0001] This application relates to the field of vehicle safety technology, and in particular to an alarm method, device, storage medium, and program product for a vehicle sentry mode. Background Technology

[0002] In related technologies, the alarm triggering mechanism of vehicle sentry mode essentially involves collecting data through hardware sensors and comparing the collected data with preset fixed thresholds or logical rules. When the data meets the triggering conditions, the alarm process is immediately initiated. That is, the alarm is triggered based on whether the distance between the person and the vehicle is less than a preset distance threshold, or based on whether vibration signals generated by the vehicle are detected. However, relying solely on a single dimension of data to determine the triggering conditions leads to lengthy and low-value alarm messages pushed to the user after the vehicle sentry mode is triggered. Users can easily overlook the real safety risks of the vehicle due to frequent false alarms in vehicle sentry mode. Summary of the Invention

[0003] The main objective of this application is to provide an alarm method, device, storage medium, and program product for vehicle sentry mode, aiming to improve the threat identification accuracy of vehicle sentry mode, thereby avoiding frequent false alarms and effectively improving the user experience.

[0004] To achieve the above objectives, a first aspect of this application provides an alarm method for vehicle sentry mode, the method comprising:

[0005] Acquire multimodal perception data in vehicle sentry mode; the multimodal perception data includes: vehicle status data, and basic environmental data and video image data around the vehicle;

[0006] The target risk level of the vehicle is determined based on the status data and the basic environmental data; the target risk level is one of multiple risk levels in the vehicle parking scenario, and the initial alarm benchmark threshold sets corresponding to each of the multiple risk levels are different.

[0007] Based on the video image data, determine the human behavior data of the environment around the vehicle, and dynamically adjust the initial alarm baseline threshold set corresponding to the target risk level based on the human behavior data to obtain the adjusted alarm baseline threshold set.

[0008] The vehicle sentry alarm is triggered based on the personnel behavior data and the adjusted alarm baseline threshold set.

[0009] In some embodiments, determining the target risk level of the vehicle based on the state data and the basic environmental data includes:

[0010] At least one target risk weighting coefficient that matches the state data is determined from a plurality of preset risk weighting coefficients; the plurality of risk weighting coefficients are used to characterize the risk level of different vehicle states;

[0011] The basic environmental data is processed by fuzzy logic reasoning to obtain the initial risk classification result of the real-time vehicle parking scenario;

[0012] The target risk level for the real-time vehicle parking scenario is calculated based on the target risk weighting coefficient and the initial risk classification result.

[0013] In some embodiments, the initial alarm baseline threshold set includes an initial distance threshold and an initial duration threshold; among the plurality of risk levels, risk level M is higher than risk level N, the initial distance threshold of risk level M is greater than the initial distance threshold of risk level N, and the initial duration threshold of risk level M is less than the initial duration threshold of risk level N.

[0014] In some embodiments, dynamically adjusting the initial alarm baseline threshold set corresponding to the target risk level based on the personnel behavior data includes:

[0015] The behavioral data of the personnel is subjected to behavioral intent quantification to obtain behavioral intent scoring results;

[0016] The behavioral intention score result is corrected based on the state data to obtain the corrected behavioral intention score result.

[0017] Based on the matching results of the corrected behavioral intent score and the preset score interval, the initial alarm benchmark threshold set corresponding to the target risk level is nonlinearly adjusted.

[0018] In some embodiments, the method further includes:

[0019] Obtain the detection results of door opening and closing actions of adjacent vehicles in vehicle sentry mode;

[0020] If the door opening / closing action detection result indicates that the door opening / closing action of an adjacent vehicle does not trigger vehicle vibration, the target distance threshold in the target alarm reference threshold set is increased, and the target duration threshold in the target alarm reference threshold set is extended; the target alarm reference threshold set includes: the initial alarm reference threshold set corresponding to the target risk level or the adjusted alarm reference threshold set.

[0021] In some embodiments, the adjusted alarm baseline threshold set includes an adjusted distance threshold and an adjusted duration threshold;

[0022] The triggering of the vehicle sentry alarm based on the personnel behavior data and the adjusted alarm baseline threshold set includes at least one of the following:

[0023] If the personnel behavior data indicates that the real-time distance between at least one target person and the vehicle is less than or equal to the adjusted distance threshold, and the duration is greater than or equal to the adjusted duration threshold, a vehicle sentry alarm is triggered; the duration is the duration during which the real-time distance is less than or equal to the adjusted distance threshold.

[0024] If the personnel behavior data indicates that the real-time distance between at least one target person and the vehicle is less than or equal to the adjusted distance threshold, and the status data indicates that the vehicle is in a risky state, the vehicle sentry alarm shall be triggered immediately.

[0025] In some embodiments, the method further includes at least one of the following:

[0026] The system acquires user command data sent by a target terminal associated with the vehicle, and controls the vehicle and plays user voice signals through an external onboard speaker according to the user command data; the target terminal is a terminal device that receives and displays alarm information when the vehicle sentry alarm is triggered.

[0027] If the status data indicates that the vehicle is in a risky state, trigger the vehicle sentry alarm;

[0028] If the status data indicates that the vehicle is in a risky state, and the target risk level is greater than or equal to a preset level threshold, and the behavioral intent score corresponding to the personnel behavior data is greater than or equal to a preset score threshold, the vehicle sentry alarm is triggered and the vehicle is controlled to remove the risky state.

[0029] In some embodiments, the personnel behavior data includes trajectory tracking results and behavior classification results, and the step of determining the personnel behavior data of the vehicle's surrounding environment based on the video image data includes:

[0030] The video image data is processed for personnel detection to obtain the detection result of at least one target person in the environment around the vehicle; the detection result is used to characterize the spatial pose information of the at least one target person.

[0031] Based on the detection results, cross-frame trajectory tracking processing is performed on the at least one target person to obtain the trajectory tracking results of the at least one target person.

[0032] The video image data is processed for behavior detection to obtain the behavior classification result of at least one target person in the environment around the vehicle.

[0033] In some embodiments, the method further includes:

[0034] Acquire vehicle vibration data;

[0035] If the vehicle vibration data is greater than or equal to a preset vibration threshold, and the behavior classification result indicates that at least one target person has performed abnormal behavior on the vehicle, the vehicle sentry alarm will be triggered immediately.

[0036] To achieve the above objectives, a second aspect of this application provides a vehicle sentry mode alarm device, the device comprising:

[0037] The acquisition module is used to acquire multimodal perception data of the vehicle in sentry mode; the multimodal perception data includes: vehicle status data, and basic environmental data and video image data around the vehicle.

[0038] The risk level classification module is used to determine the target risk level of the vehicle based on the status data and the basic environmental data; the target risk level is one of multiple risk levels in the vehicle parking scenario, and the initial alarm benchmark threshold sets corresponding to each of the multiple risk levels are different.

[0039] The threshold dynamic adjustment module is used to determine the human behavior data of the vehicle's surrounding environment based on the video image data, and to dynamically adjust the initial alarm benchmark threshold set corresponding to the target risk level based on the human behavior data, so as to obtain the adjusted alarm benchmark threshold set.

[0040] The alarm triggering module is used to trigger the vehicle sentry alarm based on the personnel behavior data and the adjusted alarm baseline threshold set.

[0041] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the vehicle sentry mode alarm method described in the first aspect.

[0042] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the vehicle sentry mode alarm method described in the first aspect.

[0043] To achieve the above objectives, a fifth aspect of this application provides a computer program product, which includes a computer program that, when executed by a processor, implements the vehicle sentry mode alarm method provided in the first aspect above.

[0044] The vehicle sentry mode alarm method, device, electronic device, computer-readable storage medium, and computer program product proposed in this application acquire multimodal perception data in vehicle sentry mode. The multimodal perception data includes: vehicle status data, and basic environmental data and video image data surrounding the vehicle. Based on the status data and the basic environmental data, a target risk level of the vehicle is determined. The target risk level is one of multiple risk levels in a vehicle parking scenario, and the initial alarm baseline threshold sets corresponding to each of the multiple risk levels are different. Based on the video image data, personnel behavior data of the environment surrounding the vehicle is determined, and the initial alarm baseline threshold set corresponding to the target risk level is dynamically adjusted based on the personnel behavior data to obtain an adjusted alarm baseline threshold set. A vehicle sentry alarm is triggered based on the personnel behavior data and the adjusted alarm baseline threshold set.

[0045] Compared to traditional methods that rely on determining whether a vehicle sentry alarm is triggered, this embodiment acquires vehicle status data, surrounding environmental data, and video image data in vehicle sentry mode for vehicle security protection. Then, based on the status data and environmental data from these multimodal perception data sets, a risk level classification for the vehicle parking scenario is performed. A target risk level is determined from multiple preset risk levels, resulting in an initial alarm baseline threshold set corresponding to that target risk level. Furthermore, based on the video image data from the multimodal perception data, personnel behavior data of the surrounding environment is determined. This personnel behavior data is then used to dynamically adjust the initial alarm baseline threshold set corresponding to the target risk level, resulting in an adjusted alarm baseline threshold set. Finally, the vehicle sentry alarm trigger condition is determined based on the personnel behavior data and the adjusted alarm baseline threshold set to trigger the vehicle sentry alarm. Thus, this application embodiment constructs a dynamic threshold system of "environment-vehicle-person" by integrating multimodal perception data such as basic environmental information, vehicle status, and personnel behavior data. Based on this system, the risk level of the vehicle parking scenario is divided, and the relevant thresholds are dynamically adjusted in combination with personnel behavior to determine whether to trigger the vehicle sentry alarm. This not only solves the problem of poor adaptability of traditional fixed thresholds, but also improves the accuracy of the vehicle sentry mode in identifying threat behaviors in the vehicle parking scenario, realizes accurate risk quantification, and avoids the phenomenon of frequent false alarms in the vehicle sentry mode, effectively improving the user experience. Attached Figure Description

[0046] Figure 1 A flowchart illustrating the steps of the vehicle sentry mode alarm method provided in some embodiments of this application;

[0047] Figure 2 for Figure 1A detailed flowchart of step S102;

[0048] Figure 3 for Figure 1 A detailed flowchart of step S103;

[0049] Figure 4 for Figure 1 A schematic diagram of another detailed step in step S103;

[0050] Figure 5 A flowchart illustrating the steps involved in the vehicle sentry mode alarm method provided in this application embodiment in other embodiments;

[0051] Figure 6 for Figure 1 A detailed flowchart of step S104;

[0052] Figure 7 A flowchart illustrating the steps of the vehicle sentry mode alarm method provided in some other embodiments of this application;

[0053] Figure 8 A flowchart illustrating the steps of the vehicle sentry mode alarm method provided in some other embodiments of this application;

[0054] Figure 9 A flowchart illustrating the steps of the vehicle sentry mode alarm method provided in this application embodiment in a complete embodiment;

[0055] Figure 10 for Figure 9 A detailed flowchart of step S903;

[0056] Figure 11 A schematic diagram of the vehicle sentry mode alarm device provided in an embodiment of this application;

[0057] Figure 12 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0059] It should be noted that although functional modules are divided in the device / system schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device / system or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0061] First, the overall concept of the vehicle sentry mode alarm method provided in the embodiments of this application will be explained.

[0062] Vehicle Sentry Mode is an intelligent safety feature for vehicles. When the vehicle is parked and locked, Vehicle Sentry Mode uses cameras and various sensors around the vehicle to monitor the surrounding environment. Once it detects actions that threaten vehicle safety, such as collisions, scratches, or door prying, it will immediately execute response mechanisms such as recording video, issuing alarms, and flashing headlights. It will also push notifications to the vehicle user's (owner's) mobile phone and other terminal devices to trigger the vehicle sentry alarm. At the same time, it will also retain video image data of the scene as evidence, thereby achieving the purpose of safeguarding vehicle safety.

[0063] In related technologies, the alarm triggering mechanism of vehicle sentry mode essentially involves collecting data through hardware sensors and comparing the collected data with preset fixed thresholds or logical rules. When the data meets the triggering conditions, the alarm process is immediately initiated. That is, the alarm is triggered based on whether the distance between the person and the vehicle is less than a preset distance threshold, or based on whether vibration signals generated by the vehicle are detected. However, this judgment based solely on single-dimensional data such as distance or acceleration leads to lengthy and low-value alarm messages pushed to users after the vehicle sentry mode is triggered. Users can easily overlook the real safety risks of the vehicle due to frequent false alarms. Furthermore, judging whether to trigger the vehicle sentry alarm based on fixed threshold rules cannot adapt to complex and changing real-world scenarios, which also leads to frequent false alarms in vehicle sentry mode.

[0064] To address these shortcomings, this application provides an alarm method, device, electronic device, computer-readable storage medium, and computer program product for vehicle sentry mode, aiming to overcome the deficiencies of the aforementioned related technologies, improve the threat identification accuracy of vehicle sentry mode, thereby avoiding frequent false alarms and effectively enhancing the user experience.

[0065] The vehicle sentry mode alarm method, device, electronic device, computer-readable storage medium, and computer program product proposed in this application acquire multimodal perception data in vehicle sentry mode. The multimodal perception data includes: vehicle status data, and basic environmental data and video image data surrounding the vehicle. Based on the status data and the basic environmental data, a target risk level of the vehicle is determined. The target risk level is one of multiple risk levels in a vehicle parking scenario, and the initial alarm baseline threshold sets corresponding to each of the multiple risk levels are different. Based on the video image data, personnel behavior data of the environment surrounding the vehicle is determined, and the initial alarm baseline threshold set corresponding to the target risk level is dynamically adjusted based on the personnel behavior data to obtain an adjusted alarm baseline threshold set. A vehicle sentry alarm is triggered based on the personnel behavior data and the adjusted alarm baseline threshold set.

[0066] Compared to traditional methods that rely on determining whether a vehicle sentry alarm is triggered, this embodiment acquires vehicle status data, surrounding environmental data, and video image data in vehicle sentry mode for vehicle security protection. Then, based on the status data and environmental data from these multimodal perception data sets, a risk level classification for the vehicle parking scenario is performed. A target risk level is determined from multiple preset risk levels, resulting in an initial alarm baseline threshold set corresponding to that target risk level. Furthermore, based on the video image data from the multimodal perception data, personnel behavior data of the surrounding environment is determined. This personnel behavior data is then used to dynamically adjust the initial alarm baseline threshold set corresponding to the target risk level, resulting in an adjusted alarm baseline threshold set. Finally, the vehicle sentry alarm trigger condition is determined based on the personnel behavior data and the adjusted alarm baseline threshold set to trigger the vehicle sentry alarm. Thus, this application embodiment constructs a dynamic threshold system of "environment-vehicle-person" by integrating multimodal perception data such as basic environmental information, vehicle status, and personnel behavior data. Based on this system, the risk level of the vehicle parking scenario is divided, and the relevant thresholds are dynamically adjusted in combination with personnel behavior to determine whether to trigger the vehicle sentry alarm. This not only solves the problem of poor adaptability of traditional fixed thresholds, but also improves the accuracy of the vehicle sentry mode in identifying threat behaviors in the vehicle parking scenario, realizes accurate risk quantification, and avoids the phenomenon of frequent false alarms in the vehicle sentry mode, effectively improving the user experience.

[0067] Next, the alarm method, device, electronic device, computer-readable storage medium, and computer program product of vehicle sentry mode provided in this application will be specifically described through the following embodiments, and firstly, the various detailed embodiments of the alarm method of vehicle sentry mode provided in this application will be described in detail.

[0068] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0069] It should be noted that the vehicle sentry mode alarm method provided in this application embodiment can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be an in-vehicle terminal device (e.g., an in-vehicle computing platform) or a terminal device associated with the vehicle. The association between the terminal device and the vehicle means that the terminal device can communicate and interact with the vehicle via a network. For example, the terminal device can be a cloud server device that communicates and interacts with the vehicle. Furthermore, the terminal can also be a smartphone, tablet, laptop, desktop computer, or other computer device. The server can be a backend server terminal device, which can be configured as an independent physical server, a server cluster consisting of multiple physical servers, or a distributed system. It can also be configured as a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The software can be an application implementing the vehicle sentry mode alarm method, a computer program, and a storage medium carrying the computer program. It should be understood that, based on different design needs of practical applications, the terminal, server, and software of the vehicle sentry mode alarm method provided in this application embodiment may also be other forms not listed here, and the vehicle sentry mode alarm method provided in this application embodiment does not specifically limit these.

[0070] Furthermore, this application can also be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer computer devices, personal computers (PCs), minicomputers, mainframe computers, vehicles, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0071] For ease of understanding and explanation, the following text will use the vehicle sentry mode alarm method provided in the embodiments of this application as an example to describe the various specific embodiments of this application in detail. The implementation of the vehicle sentry mode alarm method provided in the embodiments of this application by any other subject can refer to the implementation process of the vehicle sentry mode alarm method described below.

[0072] Please refer to Figure 1 , Figure 1 The flowchart illustrates the steps of the vehicle sentry mode alarm method provided in some embodiments of this application. It should be understood that, although... Figure 1 The flowcharts illustrating subsequent steps show the execution order of some method steps. However, based on different design needs in practical applications, the vehicle sentry mode alarm method provided in this application embodiment can, of course, employ a different execution order of method steps than shown in the figures. That is, Figure 1 The order of the method steps shown does not constitute a limitation on the execution logic order of the vehicle sentry mode alarm method provided in the embodiments of this application. Any other order based on Figure 1 Reasonable changes to the sequence of steps shown should all be included within the protection scope of the vehicle sentry mode alarm method provided in the embodiments of this application.

[0073] like Figure 1 As shown, in some embodiments, the alarm method of vehicle sentry mode provided in this application embodiment for cloud devices may include steps S101 to S104 as shown below.

[0074] Step S101: Acquire multimodal perception data in vehicle sentry mode; the multimodal perception data includes: vehicle status data, and basic environmental data and video image data around the vehicle.

[0075] During the process of activating the vehicle sentry mode to protect the vehicle, the vehicle terminal equipment continuously collects the vehicle's own status data, the basic environmental data around the vehicle, and the video image data around the vehicle, thereby obtaining multimodal perception data.

[0076] In some embodiments, the in-vehicle terminal device can collect multimodal perception data through a multimodal sensor array configured on the vehicle. This includes basic environmental data such as time characteristics, scene attributes, light intensity, and historical risk records; video image data can consist of a 360° video stream of the vehicle's surroundings collected by visual sensors (such as surround-view cameras); and status data covering core status parameters of windows, doors, sunroof, and trunk. For example, this status data can include the degree of window opening, door lock / close status, sunroof status, and trunk status.

[0077] In some embodiments, the temporal characteristics in the basic environmental data can be the vehicle terminal device's division of the current parking time period, such as early morning (5:00-7:00), daytime (7:00-18:00), evening (18:00-20:00), nighttime (20:00-5:00), etc. Furthermore, the scene attributes in the basic environmental data can be the area determination made by the vehicle terminal device based on the Global Positioning System (GPS) and Points of Interest (POIs), such as surface parking lots, underground parking lots, remote road sections, street parking spaces, etc. Moreover, the light intensity in the basic environmental data can be measured by the vehicle terminal device according to the international standard unit lux (Lux), such as strong light (>100,000 Lux), medium strong light (10,000–100,000 Lux), normal light (1,000-10,000 Lux), weak light (10-1,000 Lux), and no light (<10 Lux). Finally, the historical risk records in the basic environmental data can be the number of historical alarms that occurred in the current vehicle parking scene area as recorded by the vehicle terminal device (or obtained from the network platform).

[0078] In some embodiments, the vehicle-mounted terminal device can also dynamically divide the current parking time period according to the season, month, or sunrise and sunset times, thereby obtaining the time characteristics in the basic environmental data.

[0079] In some embodiments, the vehicle-mounted terminal device can also determine the vehicle parking scene through methods such as visual models, thereby obtaining the scene attributes in the basic environmental data.

[0080] In some embodiments, when the in-vehicle terminal device acquires video image data of the vehicle's surrounding environment through a visual sensor, it can dynamically adjust the frame rate and resolution of the camera. For example, the in-vehicle terminal device can adopt the following adjustment logic: in a silent state (no person / object appears within the pre-calibrated or defined warning range of the vehicle's surrounding environment), the camera uses the lowest frame rate (e.g., 2 frames per second) and the lowest resolution (e.g., 320*320); when a person / object appears within the warning range, the camera uses a medium frame rate (e.g., 30 frames per second) and a medium resolution (e.g., reduced to 320*320 after oversampling); and when a person / object appears within the warning range and a vibration signal is triggered, the camera uses a high frame rate (e.g., 60 frames per second) and the highest resolution for 30 seconds.

[0081] In some embodiments, the vehicle's own status data can be categorized as follows: window opening degree can be fully closed (0% opening), slightly open (1%-30% opening), half open (31%-70% opening), and fully open (71%-100% opening); door locking status can be categorized as fully locked, half locked, and unlocked; sunroof status can include three typical states: closed, tilted up, and fully open; and trunk status can be categorized as closed, unlocked, and open.

[0082] Step S102: Determine the target risk level of the vehicle based on the status data and the basic environmental data; the target risk level is one of multiple risk levels in the vehicle parking scenario, and the initial alarm benchmark threshold sets corresponding to each of the multiple risk levels are different.

[0083] It should be noted that the vehicle-mounted terminal device can pre-construct a scenario risk classification model based on the vehicle's basic environmental sample information and its own state sample data, dividing the vehicle parking scenario into several risk levels, with each risk level corresponding to a different initial alarm baseline threshold set. In some embodiments, the initial alarm baseline threshold set may include an initial distance threshold (D0) and an initial duration threshold (T0). Thus, the vehicle-mounted terminal device can determine whether to trigger a vehicle sentry alarm based on D0 and T0 in this initial alarm baseline threshold set to push relevant notifications to the user. For example, the vehicle-mounted terminal device may determine to trigger a vehicle sentry alarm if the real-time distance between a target person and the vehicle in a vehicle parking scenario is less than D0, and the duration of the real-time distance being less than D0 is greater than T0.

[0084] After acquiring multimodal perception data, the vehicle terminal device first classifies the risk level of the vehicle parking scenario based on the state data and basic environmental data in the multimodal perception data, and then determines one of the preset risk levels as the target risk level of the current parking scenario of the vehicle.

[0085] In some embodiments, when determining the target risk level of a vehicle based on state data and basic environmental data, the in-vehicle terminal device first uses fuzzy logic reasoning based on the basic environmental data to determine the initial risk level of the real-time parking scenario. Then, it further adds to this risk level based on the vehicle's own state data. For example, assuming the initial risk level determined by the in-vehicle terminal device based on the basic environmental data is level 1, but the door lock status in the vehicle's own state data is unlocked, the in-vehicle terminal device can add level 2 to the initial risk level, thus obtaining a target risk level of level 3 for the real-time parking scenario.

[0086] Step S103: Determine the human behavior data of the surrounding environment of the vehicle based on the video image data, and dynamically adjust the initial alarm benchmark threshold set corresponding to the target risk level based on the human behavior data to obtain the adjusted alarm benchmark threshold set.

[0087] After obtaining video image data of the vehicle's surrounding environment, the vehicle-mounted terminal equipment can perform intelligent analysis and processing based on the video image data to determine the behavior data of people in the surrounding environment. Then, based on the behavior data of people, it can dynamically adjust the initial alarm baseline threshold set corresponding to the target risk level of the real-time parking scenario of the vehicle to obtain the adjusted alarm baseline threshold set.

[0088] In some embodiments, when performing intelligent analysis and processing based on video image data, the in-vehicle terminal device can first use a target detection model to identify target personnel in the image in real time, outputting the personnel's position coordinates, bounding boxes, and key point coordinates. Then, it can combine a tracking model to obtain the target personnel's motion velocity vector and trajectory pattern, and dynamically calculate the real-time distance between the target personnel and the vehicle. Simultaneously, it can also use a spatiotemporal behavior classification model to classify the behavior of target personnel in the acquired video stream, thereby identifying the behavioral characteristics of the target personnel. In this way, the in-vehicle terminal device can ultimately integrate the results output by various intelligent models to form personnel behavior data of the vehicle's surrounding environment.

[0089] In some embodiments, the vehicle-mounted terminal device can pre-build a quantitative assessment system for the behavioral intentions of target personnel in the environment surrounding the vehicle. Thus, when dynamically adjusting the initial alarm baseline threshold set corresponding to the target risk level based on personnel behavior data, the device first performs feature extraction and weighted calculation on the personnel behavior data based on the assessment system to generate a behavioral risk score for the target personnel. Then, the device dynamically corrects the initial baseline thresholds (distance baseline threshold D0 and duration baseline threshold T0) in the initial alarm baseline threshold set based on the behavioral risk score.

[0090] Step S104: Trigger the vehicle sentry alarm based on the personnel behavior data and the adjusted alarm baseline threshold set.

[0091] After receiving the adjusted alarm baseline threshold set, the vehicle-mounted terminal device can determine whether to trigger a vehicle sentry alarm by comparing personnel behavior data with the thresholds in the alarm baseline threshold set. For example, if the real-time distance between the target person and the vehicle in the personnel behavior data is less than the distance threshold corrected by the alarm baseline threshold set, and the duration for which the real-time distance is less than the corrected distance threshold is greater than or equal to the duration threshold corrected by the alarm baseline threshold set, then the vehicle-mounted terminal device determines that a vehicle sentry alarm has been triggered.

[0092] In some embodiments, the vehicle-mounted terminal device can be configured with a dual triggering mechanism for the vehicle sentry alarm. The conventional triggering condition is that the real-time distance to the target person is less than a corrected distance threshold and the duration of the incident is greater than or equal to a corrected duration threshold. The immediate triggering condition is that when the vehicle sentry mode detects abnormal behavior or vibrations that meet the force requirements, the alarm is triggered directly without threshold limitations. Abnormal behavior can include kicking, scratching, or pulling door handles. Furthermore, the force requirements can be that the vehicle vibration sensor detects a vibration acceleration greater than or equal to a threshold (e.g., 0.5g).

[0093] In some embodiments, the in-vehicle terminal device may also set a dynamic exemption mechanism, which activates temporary threshold protection when the vehicle sentry mode detects an authorized user or predefined behavior, and then determines whether to trigger a vehicle sentry alarm based on the temporary threshold. Here, the temporary threshold can be increased by raising the modified distance threshold and / or extended by extending the modified time threshold.

[0094] In some embodiments, the in-vehicle terminal device can also control the vehicle to perform a linked response after the vehicle sentry alarm is triggered. Specifically, the in-vehicle terminal device stores and records 360° surround-view video, and when the vehicle sentry alarm is triggered, pushes tiered alarm information (which may include risk level, vehicle status, and keyframe screenshots) to the vehicle user's (e.g., the owner's) mobile terminal. After viewing the real-time video through the mobile terminal's application (APP), the user sends a voice command to the vehicle, and the in-vehicle terminal device receives the voice command and plays it through the vehicle's speakers. Furthermore, if the in-vehicle terminal device detects that the vehicle is unlocked and there is a risk of forced entry, it can also control the vehicle to automatically trigger window closing / door locking operations.

[0095] In this embodiment, during the vehicle's security protection process by activating the vehicle sentry mode, the in-vehicle terminal device continuously collects the vehicle's own status data, the basic environmental data surrounding the vehicle, and video image data of the surrounding area, thereby obtaining multimodal perception data. Then, the in-vehicle terminal device first classifies the risk level of the vehicle parking scenario based on the status data and basic environmental data from this multimodal perception data, thus determining one of several preset risk levels as the target risk level for the current parking scenario. Furthermore, after obtaining the video image data of the vehicle's surrounding environment, the in-vehicle terminal device performs intelligent analysis and processing on this video image data to determine the human behavior data of the people in the surrounding environment. Then, based on this human behavior data, it dynamically adjusts the initial alarm baseline threshold set corresponding to the target risk level of the real-time parking scenario, thus obtaining an adjusted alarm baseline threshold set. Finally, the in-vehicle terminal device compares the human behavior data with the thresholds in the alarm baseline threshold set to determine whether to trigger the vehicle sentry alarm.

[0096] Compared to traditional methods that rely on determining whether a vehicle sentry alarm is triggered, this embodiment acquires vehicle status data, surrounding environmental data, and video image data in vehicle sentry mode for vehicle security protection. Then, based on the status data and environmental data from these multimodal perception data sets, a risk level classification for the vehicle parking scenario is performed. A target risk level is determined from multiple preset risk levels, resulting in an initial alarm baseline threshold set corresponding to that target risk level. Furthermore, based on the video image data from the multimodal perception data, personnel behavior data of the surrounding environment is determined. This personnel behavior data is then used to dynamically adjust the initial alarm baseline threshold set corresponding to the target risk level, resulting in an adjusted alarm baseline threshold set. Finally, the vehicle sentry alarm trigger condition is determined based on the personnel behavior data and the adjusted alarm baseline threshold set to trigger the vehicle sentry alarm. Thus, this application embodiment constructs a dynamic threshold system of "environment-vehicle-person" by integrating multimodal perception data such as basic environmental information, vehicle status, and personnel behavior data. Based on this system, the risk level of the vehicle parking scenario is divided, and the relevant thresholds are dynamically adjusted in combination with personnel behavior to determine whether to trigger the vehicle sentry alarm. This not only solves the problem of poor adaptability of traditional fixed thresholds, but also improves the accuracy of the vehicle sentry mode in identifying threat behaviors in the vehicle parking scenario, realizes accurate risk quantification, and avoids the phenomenon of frequent false alarms in the vehicle sentry mode, effectively improving the user experience.

[0097] In some embodiments, the initial alarm baseline threshold set includes an initial distance threshold and an initial duration threshold. The initial distance threshold can be the initial distance threshold (D0) described above, and the initial duration threshold can be the initial duration threshold (T0) described above.

[0098] In some embodiments, among the plurality of risk levels, risk level M is higher than risk level N, the initial distance threshold of risk level M is greater than the initial distance threshold of risk level N, and the initial duration threshold of risk level M is less than the initial duration threshold of risk level N. The vehicle-mounted terminal device can pre-divide the vehicle parking scenario into multiple risk levels, each level corresponding to an initial alarm baseline threshold set. Furthermore, the vehicle-mounted terminal device configures a larger initial distance threshold and a smaller initial duration threshold for higher risk levels. For example, assuming the vehicle-mounted terminal device divides the vehicle parking scenario into 5 risk levels based on basic environmental and status data, where risk level 5 is higher than risk level 4, and the initial distance threshold D0 corresponding to risk level 5 is greater than the initial distance threshold D0 of risk level 4, and the initial duration threshold T0 of risk level 5 is less than the initial duration threshold T0 of risk level 4.

[0099] Please refer to Figure 2 , Figure 2 for Figure 1 A detailed flowchart of step S102.

[0100] like Figure 2 As shown, in some embodiments, the step of "determining the target risk level of the vehicle based on the state data and the basic environmental data" in step S102 above may include steps S201 to S203 as shown below.

[0101] Step S201: Determine at least one target risk weighting coefficient that matches the state data from a plurality of preset risk weighting coefficients; the plurality of risk weighting coefficients are used to characterize the risk level of different vehicle states.

[0102] When determining the target risk level of a vehicle based on state data and basic environmental data from multimodal perception data, the vehicle terminal equipment can determine at least one target risk weighting coefficient that matches the state data from a set of preset risk weighting coefficients used to characterize the risk level of different vehicle states.

[0103] In some embodiments, the in-vehicle terminal device can pre-set risk weighting coefficients corresponding to different state parameters based on the vehicle's own state data. For example: risk level +1 when the window opening is >30%, risk level +2 when the doors / trunk are unlocked, and risk level +1 when the sunroof is fully open. A higher risk weighting coefficient indicates a higher level of risk in the vehicle's current state. For instance, a risk weighting coefficient of "risk level +2 when doors are unlocked" represents a higher level of risk when the vehicle is in an unlocked state than a risk weighting coefficient of "risk level +1 when the sunroof is fully open" represents a higher level of risk when the sunroof is open.

[0104] For example, when the vehicle terminal device has the status data of unlocked doors and fully open sunroof, it can match the two target risk weighting coefficients from the above risk weighting coefficients: "risk level when doors are unlocked + 2" and "risk level when sunroof is fully open + 1".

[0105] Step S202: Perform fuzzy logic reasoning on the basic environmental data to obtain the initial risk classification result of the real-time vehicle parking scenario.

[0106] When determining the target risk level of a vehicle based on state data and basic environmental data from multimodal perception data, the vehicle-mounted terminal equipment can perform fuzzy logic reasoning based on the basic environmental data to obtain an initial risk classification result for the real-time parking scenario. This initial risk classification result can represent the initial risk level for the aforementioned real-time parking scenario.

[0107] In some embodiments, when performing fuzzy logic reasoning based on basic environmental data, the vehicle-mounted terminal device can use the time characteristics, scene attributes, light intensity, and historical risk records in the basic environmental data as input to the fuzzy logic algorithm. The algorithm first constructs a fuzzy set, then uses a membership function to calculate the degree to which each input variable belongs to the fuzzy set. Next, fuzzy reasoning is performed using a preset rule base (e.g., nighttime + remote road sections + no light + high alarm frequency corresponding to high risk, etc.). Finally, after defuzzification, a risk level of 1-5 is output. In this way, the vehicle-mounted terminal device can use the risk level output by the fuzzy logic algorithm as the initial risk classification result for the real-time parking scenario of the vehicle.

[0108] In some embodiments, the in-vehicle terminal device may also process the basic environmental data using a fixed rule-based approach to obtain an initial risk classification result within the fixed rule. Alternatively, the in-vehicle terminal device may also use an artificial intelligence (AI) model to process the basic environmental data of personnel to predict the initial risk classification result for the real-time parking scenario of the vehicle.

[0109] Step S203: Calculate the target risk level of the real-time vehicle parking scenario based on the target risk weighting coefficient and the initial risk classification result.

[0110] After obtaining the initial risk classification result of the real-time parking scenario, the vehicle terminal device can add to the initial risk classification result based on the target risk weighting coefficient that matches the vehicle's own state data. That is, the target risk weighting coefficient is superimposed on the initial risk classification result to obtain the final target risk level of the real-time parking scenario.

[0111] For example, assuming the risk level (initial risk classification result) output by the fuzzy logic algorithm is 1, and the target risk weighting coefficient is "risk level when the car door is unlocked + 2 levels" and "risk level when the sunroof is fully open + 1 level", the vehicle terminal device can obtain the final target risk level of 4 for the real-time parking scenario by superimposing the target risk weighting coefficient and the initial risk classification result.

[0112] In this embodiment, an in-vehicle terminal device constructs a scenario risk classification model based on basic environmental data and the vehicle's own state data. Based on fuzzy logic reasoning of the basic environmental data, the current real-time parking scenario is divided into several risk levels (each level corresponds to an initial baseline threshold parameter set, including a distance baseline threshold D0 and a duration baseline threshold T0; higher-risk scenarios are configured with larger initial distance thresholds and smaller initial duration thresholds). Furthermore, the in-vehicle terminal device adds a bonus to the target risk level of the real-time parking scenario obtained through fuzzy logic reasoning based on the vehicle's own state data, thus arriving at the final target risk level for the real-time parking scenario. Subsequently, the in-vehicle terminal device obtains the initial alarm baseline threshold set corresponding to this target risk level and dynamically adjusts it to determine whether to trigger a vehicle sentry alarm. This improves the accuracy of threat behavior identification in the vehicle sentry mode within a parking scenario, achieving precise risk quantification. It also enhances the scenario adaptability of the vehicle sentry mode, enabling it to accurately trigger vehicle sentry alarms in complex and ever-changing real-world scenarios to push valuable alarm information to users.

[0113] In some embodiments, the personnel behavior data includes trajectory tracking results and behavior classification results.

[0114] It should be noted that the trajectory tracking results may include: the target person's velocity vector, motion trajectory pattern, and the real-time distance between the target person and the vehicle. Furthermore, the behavior classification results are used to characterize whether at least one target person in the vehicle's surrounding environment engages in abnormal behaviors such as patting the vehicle body, peering into the vehicle, pulling door handles, kicking the vehicle, scratching the vehicle, damaging tires, loitering, or leaning against the vehicle.

[0115] Please refer to Figure 3 , Figure 3 for Figure 1 A detailed flowchart of step S103.

[0116] like Figure 3 As shown, in some embodiments, the step of "determining the human behavior data of the vehicle's surrounding environment based on the video image data" in step S103 above may include steps S301 to S303 as shown below.

[0117] Step S301: Perform personnel detection processing on the video image data to obtain the detection result of at least one target person in the environment around the vehicle; the detection result is used to characterize the spatial pose information of the at least one target person.

[0118] It should be noted that the detection results for the target person can be the aforementioned person position coordinates, bounding box, and key point coordinates. This information can characterize the spatial pose information of the target person in the image (also known as the spatial positioning and morphological information of the target person in the image). Among them, the position coordinates and bounding box are used to directly describe the spatial location range of the target person in the image, while the key point coordinates (such as human joints and facial feature points) are used to further supplement the pose and morphological details of the target person.

[0119] When in-vehicle terminal equipment performs intelligent analysis on video image data to determine human behavior data in the vehicle's surrounding environment, it can first preprocess the video image data, and then input the processed video image data into a detection model for human detection. This yields the location coordinates, bounding box, and keypoint coordinates of at least one target person in the video image data, output by the detection model. In this way, the in-vehicle terminal equipment can use the location coordinates, bounding box, and keypoint coordinates of at least one target person as the detection result for at least one target person in the vehicle's surrounding environment.

[0120] In some embodiments, the detection model invoked by the vehicle terminal device can be obtained by secondary training (fine-tuning training) of the pre-trained personnel detection model using personnel behavior sample data specific to the vehicle sentry mode.

[0121] In some embodiments, when the vehicle-mounted terminal device preprocesses video image data, it can convert the original video image data into a frame sequence in RGB image format, and then perform standardized preprocessing (including noise reduction, illumination equalization, and size unification, etc.) to ensure the consistency of subsequent model processing.

[0122] Step S302: Based on the detection results, perform cross-frame trajectory tracking processing on the at least one target person to obtain the trajectory tracking results of the at least one target person.

[0123] After obtaining the detection results of at least one target person in the environment surrounding the vehicle, the vehicle terminal equipment further inputs the detection results into a tracking model (such as a multi-target tracking algorithm). Based on the detection results, the tracking model performs correlation matching on at least one target person in consecutive video frames, thereby realizing cross-frame trajectory tracking processing of at least one target person and obtaining the trajectory tracking result of at least one target person output by the tracking model.

[0124] The vehicle-mounted terminal device invokes a tracking model to perform cross-frame trajectory tracking processing on at least one target person. This tracking model can calculate the real-time distance and velocity vector (approaching / moving speed and direction of movement) of the target person from the vehicle based on the changes in the coordinates of the foot key points of the same target person in consecutive frames, combined with pre-calibrated vehicle parameters (such as position coordinates). Furthermore, by accumulating the real-time distance and velocity vector of the target person from the vehicle in multiple frames, a motion trajectory pattern (also known as motion trajectory feature) of the target person can be formed.

[0125] In some embodiments, the movement trajectory pattern of the target person may include: straight-line passing, stationary stopping, peripheral wandering, and reciprocating movement. Specifically, the vehicle-mounted terminal device can predefine straight-line passing as: the mean square error of the fitted straight line is ≤5 pixels, and the continuous movement distance is ≥5 meters; furthermore, the vehicle-mounted terminal device can predefine stationary stopping as: the position change within 30 consecutive frames is ≤0.5 meters; furthermore, the vehicle-mounted terminal device can predefine peripheral wandering as: the trajectory is an irregular loop, and the person does not leave the area with a radius of 2 meters centered on the initial position within 10 seconds; finally, the vehicle-mounted terminal device can predefine reciprocating movement as: moving back and forth between two points ≥2 times, and the time for each round trip is ≤30 seconds.

[0126] Step S303: Perform behavior detection processing on the video image data to obtain the behavior classification result of at least one target person in the environment around the vehicle.

[0127] When determining the behavior data of people in the vehicle's surrounding environment based on video image data, in addition to using detection and tracking models to perform cross-frame trajectory tracking of at least one target person in the vehicle's surrounding environment, the vehicle-mounted terminal equipment can also input the video image data into a behavior detection model to perform behavior detection processing on at least one target person, thereby obtaining the behavior of at least one target person output by the behavior detection model (including patting the vehicle body, peeking into the vehicle, pulling door handles, kicking the vehicle, scratching the vehicle, damaging tires, loitering, leaning against the vehicle, etc.). In this way, the vehicle-mounted terminal equipment can obtain the behavior classification results of at least one target person in the vehicle's surrounding environment.

[0128] In some embodiments, the behavior detection model invoked by the vehicle terminal device can also be obtained by using the personnel behavior sample data specific to the vehicle sentry mode to perform secondary training (fine-tuning training) on ​​the pre-trained behavior detection model.

[0129] Please refer to Figure 4 , Figure 4 for Figure 1 A schematic diagram of another detailed step in step S103.

[0130] like Figure 4As shown, in some embodiments, the step of "dynamically adjusting the initial alarm baseline threshold set corresponding to the target risk level based on the personnel behavior data" in step S103 above may include steps S401 to S403 as shown below.

[0131] Step S401: Perform behavioral intent quantification processing on the personnel behavior data to obtain behavioral intent scoring results.

[0132] When the vehicle-mounted terminal device dynamically adjusts the initial alarm baseline threshold set corresponding to the target risk level based on personnel behavior data, it can perform intent quantification assessment of the personnel behavior data to correct the baseline thresholds, thereby obtaining the adjusted alarm baseline threshold set. Furthermore, when quantifying the intent of personnel behavior data, the vehicle-mounted terminal device can use a scoring algorithm to quantify the behavioral intent of the personnel behavior data, thereby obtaining the behavioral intent score output by the scoring algorithm.

[0133] In some embodiments, the vehicle terminal device may set the behavioral intent score output by the scoring algorithm to 1-10 points, and in the scoring algorithm, the weight of distance change rate is set to 40%, the weight of dwelling behavior characteristics is set to 30%, the weight of trajectory regularity is set to 20%, and the weight of surrounding people is set to 10%.

[0134] In some embodiments, the in-vehicle terminal device may also use a fixed rule-based approach to quantify the behavioral intent of the personnel behavior data. Alternatively, the in-vehicle terminal device may also use fuzzy logic algorithms and / or artificial intelligence (AI) models to quantify the behavioral intent of the personnel behavior data.

[0135] Step S402: Correct the behavior intention score result based on the state data to obtain the corrected behavior intention score result.

[0136] When quantifying the intent of human behavior data, the in-vehicle terminal device can also correct the behavioral intent score output by the scoring algorithm based on the vehicle's own state data, thereby obtaining a corrected behavioral intent score. For example, if the vehicle's own state data indicates that the vehicle is in a state such as an open window or an unlocked door, the in-vehicle terminal device can multiply the behavioral intent score by a factor of 1.2 for correction.

[0137] Step S403: Based on the matching result of the corrected behavioral intent score and the preset score interval, the initial alarm benchmark threshold set corresponding to the target risk level is nonlinearly adjusted.

[0138] After obtaining the corrected behavioral intent score, the vehicle terminal device matches the behavioral intent score with a preset score range (such as score ≤ 2 points, 2 points < score ≤ 5 points, 5 points < score ≤ 8 points, and score > 8 points, etc.) to obtain the matching result between the behavioral intent score and the score range. Based on the matching result, the initial alarm benchmark threshold set corresponding to the target risk level is nonlinearly adjusted.

[0139] For example, the vehicle-mounted terminal device can pre-establish a nonlinear correction function: when the score is ≤2 points, the corrected distance threshold = D0 × 0.8 and the corrected duration threshold = T0 × 1.5; when 2 < score ≤ 5 points, the initial baseline threshold is maintained; when 5 < score ≤ 8 points, the corrected distance threshold = D0 × 1.2 and the corrected duration threshold = T0 × 0.8; when the score > 8 points, the corrected distance threshold = D0 × 1.5 and the corrected duration threshold = T0 × 0.5. Thus, by substituting the corrected behavioral intent score into this nonlinear correction function, the vehicle terminal device can automatically match the behavioral intent score with the preset score range and calculate the adjusted distance threshold and the adjusted duration threshold, thereby obtaining the adjusted alarm baseline threshold set.

[0140] In some embodiments, after obtaining the behavioral intent score output by the scoring algorithm, the vehicle-mounted terminal device can directly match the behavioral intent score with a preset scoring range, and make nonlinear adjustments to the initial alarm benchmark threshold set corresponding to the target risk level based on the matching result.

[0141] In this embodiment, the vehicle-mounted terminal device intelligently analyzes video image data to determine the behavior data of at least one target person in the environment surrounding the vehicle. Then, based on this behavior data, the initial alarm baseline threshold set corresponding to the target risk level is dynamically adjusted non-linearly to obtain the adjusted alarm baseline threshold set. In this way, based on a personnel behavior analysis and scoring mechanism, target detection and behavior classification processing are utilized to first extract features such as personnel motion vectors and behavioral actions from video images. Then, based on a scoring and quantification system, abstract behaviors are transformed into the basis for correcting alarm baseline thresholds. This allows for more accurate capture of potential threats in parked vehicle scenarios, thereby further improving the accuracy of threat behavior identification in the vehicle sentry mode in parked vehicle scenarios.

[0142] Please refer to Figure 5 , Figure 5 The alarm method for vehicle sentry mode provided in this application is illustrated in some other embodiments as a flowchart of the steps involved.

[0143] like Figure 5As shown, in some embodiments, the vehicle sentry mode alarm method provided in this application may further include steps S501 and S502 as shown below.

[0144] Step S501: Obtain the detection results of door opening and closing actions of adjacent vehicles in vehicle sentry mode.

[0145] It should be noted that adjacent vehicles can be other vehicles that are close to the vehicle's location in the current parking scenario and have the potential to affect each other.

[0146] When the vehicle-mounted terminal device activates the vehicle sentry mode to provide security protection for the vehicle, if there are adjacent vehicles, it can continuously collect video image data and / or millimeter-wave radar data around the vehicle through the vehicle sentry mode, and monitor the door opening and closing actions of the adjacent vehicles in real time, thereby obtaining the detection results of the door opening and closing actions of the adjacent vehicles. Here, the video image data can be the same data as the video image data in the aforementioned multimodal perception data, or the video image data can be data specifically collected by the vehicle sentry mode using the vehicle's vision sensors to monitor the door opening and closing actions of adjacent vehicles.

[0147] In some embodiments, when the vehicle terminal device monitors the door opening and closing actions of adjacent vehicles through the vehicle sentry mode, it can also collect the vibration signal of its own vehicle through the vehicle's vibration sensor, thereby monitoring whether the door opening and closing actions of the adjacent vehicle have triggered vibration in its own vehicle.

[0148] Step S502: If the door opening / closing action detection result indicates that the door opening / closing action of an adjacent vehicle does not trigger vehicle vibration, increase the target distance threshold in the target alarm reference threshold set and extend the target duration threshold in the target alarm reference threshold set; the target alarm reference threshold set includes: the initial alarm reference threshold set corresponding to the target risk level or the adjusted alarm reference threshold set.

[0149] After receiving the detection results of the door opening / closing actions of adjacent vehicles, if the detection results indicate that the door opening / closing actions of the adjacent vehicles did not trigger vibration in its own vehicle, the vehicle terminal equipment can adjust the currently obtained target alarm reference threshold set to a temporary threshold to initiate temporary threshold protection for its own vehicle. That is, it increases the target distance threshold in the target alarm reference threshold set and extends the target duration threshold in the target alarm reference threshold set. Here, the target alarm reference threshold set can be the initial alarm reference threshold set corresponding to the aforementioned target risk level, or it can be the adjusted alarm reference threshold set mentioned above. For example, after obtaining the initial alarm reference threshold set corresponding to the target risk level, or after dynamically adjusting the initial alarm reference threshold set to obtain the adjusted alarm reference threshold set, if the vehicle sentry mode detects a special event of a door opening / closing action of an adjacent vehicle (without triggering vehicle vibration), the vehicle terminal equipment will initiate temporary threshold protection, increasing the distance threshold in the current target alarm reference threshold set to 1.5 times its original value and extending the duration threshold to twice its original value.

[0150] In some embodiments, when the vehicle terminal device activates temporary threshold protection, if the vehicle's own status data indicates that the vehicle is in a risky state (such as the vehicle having unlocked doors / open windows), the vehicle terminal device reduces the adjustment range of the distance threshold and market threshold in the target alarm baseline threshold set (e.g., by 50%).

[0151] In this embodiment, when the vehicle terminal device detects the door opening and closing action of an adjacent vehicle in vehicle sentry mode but does not trigger vehicle body vibration, a temporary threshold protection is activated. This can form a dynamic exemption mechanism for authorized users or predefined behaviors, thereby forming a balanced mechanism of "sensitive early warning - false alarm filtering" and improving the intelligence of vehicle sentry mode alarm.

[0152] In some embodiments, the adjusted alarm baseline threshold set includes an adjusted distance threshold and an adjusted duration threshold.

[0153] Please refer to Figure 6 , Figure 6 for Figure 1 A detailed flowchart of step S104.

[0154] like Figure 6 As shown, in some embodiments, step S104 above: triggering a vehicle sentry alarm based on the personnel behavior data and the adjusted alarm baseline threshold set may include at least one of the following steps S601 and S602.

[0155] Step S601: If the personnel behavior data indicates that the real-time distance between at least one target person and the vehicle is less than or equal to the adjusted distance threshold, and the duration is greater than or equal to the adjusted duration threshold, trigger the vehicle sentry alarm; the duration is the duration during which the real-time distance is less than or equal to the adjusted distance threshold.

[0156] When the vehicle-mounted terminal device matches personnel behavior data with adjusted distance thresholds and adjusted duration thresholds to determine whether to trigger a vehicle sentry alarm, if the personnel behavior data indicates that the real-time distance between at least one target person and the vehicle is less than or equal to the adjusted distance threshold, and the duration for which the real-time distance is less than the distance threshold is greater than or equal to the adjusted duration threshold, then in this case, the vehicle-mounted terminal device determines that the vehicle sentry mode meets the normal triggering conditions for an alarm, thereby triggering the vehicle sentry alarm.

[0157] Step S602: When the personnel behavior data indicates that the real-time distance between at least one target person and the vehicle is less than or equal to the adjusted distance threshold, and the status data indicates that the vehicle is in a risky state, the vehicle sentry alarm is triggered immediately.

[0158] It should be noted that the status data indicating that the vehicle is in a risky state can be the same as the status data indicating that the vehicle's windows are open or the doors are unlocked.

[0159] When the vehicle-mounted terminal device triggers the vehicle sentry alarm based on personnel behavior data and the adjusted alarm baseline threshold set, it can also determine whether the immediate triggering conditions are met based on the vehicle's own status data. That is, when the personnel behavior data indicates that the real-time distance between at least one target person and the vehicle is less than or equal to the adjusted distance threshold, it no longer considers whether the continuous market has reached the adjusted duration threshold, but detects whether the status data indicates that the vehicle is in a risky state. Thus, when the status data indicates that the vehicle is in a risky state, the vehicle-mounted terminal device determines that the vehicle sentry mode meets the immediate triggering conditions for the alarm, thereby triggering the vehicle sentry alarm.

[0160] For example, the vehicle-mounted terminal device can be pre-set with one of the immediate trigger conditions as abnormal intrusion: when a person approaches an open window / unlocked door at a distance of <0.4m, an alarm is triggered directly. Thus, the vehicle-mounted terminal device will immediately trigger the vehicle sentry alarm when, based on personnel behavior data, the real-time distance between at least one target person and the vehicle is less than the adjusted distance threshold of 0.4m, and the vehicle's own status data indicates that the vehicle is in a dangerous state due to an open window / unlocked door.

[0161] Please refer to Figure 7 , Figure 7A flowchart illustrating the steps of the vehicle sentry mode alarm method provided in some other embodiments of this application.

[0162] like Figure 7 As shown, in some embodiments, the vehicle sentry mode alarm method provided in this application may further include steps S701 and S702 as shown below.

[0163] Step S701: Obtain vehicle vibration data.

[0164] When the vehicle-mounted terminal device activates the vehicle sentry mode to protect the vehicle, it can collect the vehicle's vibration signals through the vehicle's vibration sensors, thereby obtaining vehicle vibration data.

[0165] Step S702: If the vehicle vibration data is greater than or equal to a preset vibration threshold, and the behavior classification result indicates that at least one target person has performed abnormal behavior on the vehicle, the vehicle sentry alarm is triggered immediately.

[0166] When determining the behavior data of people in the vehicle's surrounding environment based on video image data, in addition to using detection and tracking models to perform cross-frame trajectory tracking of at least one target person in the vehicle's surrounding environment, the vehicle terminal equipment can also combine vehicle vibration data and the behavior classification results of at least one target person in the vehicle's surrounding environment to determine whether the vehicle has triggered an abnormal force event. That is, if the vehicle vibration data is greater than or equal to a preset vibration threshold (such as 0.5g), and the behavior classification result indicates that at least one target person has performed abnormal behavior on the vehicle (kicking the car, scratching the car, pulling the door handle, etc.), it is determined that the vehicle has triggered an abnormal force event, thereby triggering the vehicle sentry alarm in real time.

[0167] For example, the vehicle-mounted terminal device can be pre-set with one of the immediate trigger conditions as abnormal force: the vehicle vibration sensor detects a vibration acceleration ≥0.5g, and the behavior classification model confirms through image analysis that there are abnormal behaviors such as kicking, scratching, or pulling the door handle. The two are combined to determine an abnormal force event, thereby directly triggering an alarm. In this way, when the vehicle vibration data is greater than or equal to 0.5g, and at least one target person performs abnormal behaviors such as kicking, scratching, or pulling the door handle on the vehicle, the vehicle-mounted terminal device will immediately trigger the vehicle sentry alarm.

[0168] In this embodiment, a dual triggering mechanism is set up through the vehicle terminal device to determine whether to trigger an alarm. The conventional triggering condition depends on the adjusted distance threshold and the adjusted duration threshold to determine whether to trigger the vehicle sentry mode alarm. At the same time, an instant triggering condition is set to directly trigger the vehicle sentry alarm when the vehicle is determined to have received dangerous behavior. This forms a balanced mechanism of "sensitive early warning - false alarm filtering", which makes the vehicle sentry mode alarm more intelligent.

[0169] Please refer to Figure 8 , Figure 8 A flowchart illustrating the steps of the vehicle sentry mode alarm method provided in some other embodiments of this application.

[0170] like Figure 8 As shown, in some embodiments, the vehicle sentry mode alarm method provided in this application may further include at least one of the following steps S801 to S803.

[0171] Step S801: Obtain user instruction data sent by the target terminal associated with the vehicle, and control the vehicle and play user voice signals through the vehicle's external speakers according to the user instruction data; the target terminal is a terminal device that receives alarm information and displays it when the vehicle sentry alarm is triggered.

[0172] It should be noted that the target terminal can be the mobile terminal of the aforementioned vehicle user. Alternatively, the target terminal can also be the vehicle's cloud network platform.

[0173] When the vehicle-mounted terminal device triggers the vehicle sentry alarm, it can integrate information such as risk level identification, trigger time, vehicle status details, keyframe screenshots, and links to 360° surround view video clips into an alarm message, which is then sent to the target terminal for vehicle management. Upon receiving the alarm message, the target terminal can display it to the vehicle user via pop-up windows or other means. The vehicle user can then interact with the application (APP) on the target terminal, converting the voice signal into text-based user command data. This user command data is transmitted to the vehicle-mounted terminal device via the communication link between the target terminal and the vehicle. After receiving the user command data from the target terminal, the vehicle-mounted terminal device parses and verifies permissions, then controls the corresponding operating system on the vehicle to execute the operation instructed by the user command data (such as closing windows / locking doors, etc.). Furthermore, it plays the user's voice signal from the user command data through the vehicle's external speakers.

[0174] Step S802: If the status data indicates that the vehicle is in a risky state, trigger the vehicle sentry alarm.

[0175] The vehicle terminal equipment can also directly determine whether to trigger the vehicle sentry alarm based on the vehicle's own status data. That is, if the status data indicates that the vehicle is in a risky state (such as the vehicle being unlocked / the windows are wide open), the vehicle sentry alarm will be triggered directly.

[0176] Step S803: When the status data indicates that the vehicle is in a risky state, and the target risk level is greater than or equal to a preset level threshold, and the behavioral intent score result corresponding to the personnel behavior data is greater than or equal to a preset score threshold, trigger the vehicle sentry alarm and control the vehicle to remove the risky state.

[0177] When the vehicle's own status data indicates that the vehicle is in a risky state, thus triggering the vehicle sentry alarm, the vehicle terminal device can also simultaneously control the vehicle to de-risk (e.g., control the vehicle's actuator to close the windows / lock the doors, etc.). That is, when the status data indicates that the vehicle is in a risky state, and the target risk level is greater than or equal to a preset level threshold (e.g., level 4), and the behavioral intent score obtained by quantitatively evaluating the behavioral intent of the personnel's behavior data is greater than or equal to a preset score threshold (e.g., 8), then in this case, the vehicle terminal device will trigger the vehicle sentry alarm and simultaneously control the vehicle to de-risk.

[0178] In this embodiment, after the vehicle-mounted terminal device pushes an alarm message to the vehicle user when the vehicle sentry alarm is triggered, the vehicle user can view the real-time footage through a mobile application (APP) and send commands to the vehicle. Upon receiving the commands, the vehicle-mounted terminal device plays the user's voice message through the vehicle's speakers. Furthermore, if the vehicle is unlocked and there is a risk of forced entry, the device simultaneously controls the vehicle to automatically close the windows and lock the doors. This not only further enhances the intelligence of the vehicle sentry mode alarm triggering but also ensures the effectiveness of the vehicle sentry mode in providing security protection for the vehicle.

[0179] Next, a complete embodiment of the vehicle sentry mode alarm method provided in this application is presented.

[0180] Please refer to Figure 9 , Figure 9 A flowchart illustrating the steps of a complete embodiment of the vehicle sentry mode alarm method provided in this application.

[0181] like Figure 9 As shown, in some embodiments, the vehicle-mounted terminal device can construct a dynamic threshold adjustment method for environment-vehicle-personnel through steps S901 to S905 as shown below, thereby improving the accuracy and scene adaptability of triggering vehicle sentry alarms.

[0182] Step S901: The sensor group collects basic environmental information, raw video data, and vehicle status data.

[0183] It should be noted that the basic environmental information refers to the basic environmental data mentioned above, the raw video data refers to the video image data mentioned above, and the vehicle's own status data refers to the status data mentioned above. Repeated descriptions of the same content will not be repeated here.

[0184] Step S902: Process the video stream data to obtain personnel behavior data. This personnel behavior data includes personnel movement speed vectors, trajectory patterns, real-time distance between personnel and vehicles, and personnel behavior classification.

[0185] For example, the vehicle-mounted terminal device converts the acquired raw video image stream into a frame sequence in RGB image format and performs standardized preprocessing (including noise reduction, illumination equalization, and size uniformity). The processed image is then input into a detection model to obtain the bounding box and key point information of the target person. The detection results are input into a tracking model to perform correlation matching on the target person in consecutive video frames, achieving cross-frame tracking. Based on the changes in the coordinates of the foot key points of the same person in consecutive frames, and combined with pre-calibrated vehicle parameters, the distance between the target person and the vehicle, and the motion vector (approaching / moving speed and direction of movement) are calculated. Multiple frames are accumulated to form motion trajectory features, specifically including: straight-line passing, fixed-point stopping, surrounding wandering, and reciprocating movement. The definitions of straight-line passing, fixed-point stopping, surrounding wandering, and reciprocating movement are consistent with the definitions of the same features in the above embodiments, and will not be repeated here.

[0186] Meanwhile, the vehicle-mounted terminal equipment will also input the frame sequence into the behavior detection model to detect the behavior of the person instance, and obtain the person's behavior: patting the car body, peeking into the car, pulling the door handle, kicking the vehicle, scratching the car, damaging the tires, lingering, leaning against the vehicle, etc.

[0187] Step S903: Obtain the adjusted threshold based on basic environmental information, vehicle status, and personnel behavior data.

[0188] like Figure 10 As shown, step S903 includes two parts: step S1001 and step S1002.

[0189] Step S1001: Based on basic environmental information and the vehicle's own status, construct a scenario risk classification and obtain basic thresholds. The vehicle-mounted terminal device divides the parking scenario into 5 risk levels, each level corresponding to an initial baseline threshold parameter set, which includes a distance baseline threshold (D0) and a duration baseline threshold (T0).

[0190] For example, the in-vehicle terminal device can classify risk scenarios based on fuzzy logic algorithms. Using time features, scene attributes, light intensity, and historical risk records as input, a fuzzy set is constructed. A membership function is used to calculate the degree to which each variable belongs to the fuzzy set. Then, fuzzy inference is performed using a preset rule base (e.g., nighttime + remote road sections + no light + high alarm frequency corresponds to high risk). Finally, after defuzzification, a risk level of 1-5 is output, realizing risk classification for parking scenarios. Simultaneously, the risk weighting coefficient for the vehicle's own status data is set as follows: risk level +1 when window opening > 30%, risk level +2 when doors / trunk are unlocked, and risk level +1 when sunroof is fully open.

[0191] In addition, high-risk scenarios are configured with larger initial distance thresholds and smaller initial duration thresholds.

[0192] Step S1002: Quantitatively assess and correct the basic threshold for behavioral intent to obtain the adjusted threshold.

[0193] The in-vehicle terminal device is pre-set with a behavioral intent score of 1-10. The scoring algorithm uses the following weights: distance change rate (40%), dwell time behavior characteristics (30%), trajectory regularity (20%), and surrounding crowd size (10%). When the vehicle has open windows or unlocked doors, the behavioral risk score is adjusted (multiplied by a coefficient of 1.2). Finally, the in-vehicle terminal device dynamically adjusts the initial baseline threshold based on the behavioral risk score. Specifically, based on a non-linear correction function, when the score is ≤2, the corrected distance threshold = D0 × 0.8 and the corrected duration threshold = T0 × 1.5; when 2 < score ≤ 5, the initial baseline threshold is maintained; when 5 < score ≤ 8, the corrected distance threshold = D0 × 1.2 and the corrected duration threshold = T0 × 0.8; when the score > 8, the corrected distance threshold = D0 × 1.5 and the corrected duration threshold = T0 × 0.5.

[0194] Step S904: Determine whether to trigger an alarm based on the adjusted threshold and the dual triggering mechanism.

[0195] It should be noted that the dual triggering mechanism includes immediate triggering of abnormal force and abnormal intrusion conditions. Abnormal force is defined as vibration acceleration ≥0.5g detected by the vehicle vibration sensor, and abnormal behavior such as kicking, scratching, or pulling door handles confirmed by the behavior classification model through image analysis. The two are combined to determine an "abnormal force event". In addition, abnormal intrusion is defined as a person approaching and opening a window / unlocking a door within a distance of <0.4m, which directly triggers the alarm.

[0196] In addition, the in-vehicle terminal equipment is also equipped with a dynamic exemption mechanism for vehicle owners or authorized users: no alarm will be triggered when the image recognizes the vehicle owner / authorized user (face comparison).

[0197] Furthermore, the in-vehicle terminal equipment can also be configured with an exemption mechanism for special events: when a neighboring vehicle's door is detected to be opening or closing (without triggering vehicle vibration), a temporary threshold protection is activated, increasing the distance threshold to 1.5 times the current value and extending the duration threshold to twice the current value. Specifically, when the vehicle is unlocked / windows are wide open, the threshold adjustment for this exemption mechanism is reduced by 50%.

[0198] Step S905: After the alarm is triggered, the linkage response is executed.

[0199] The in-vehicle terminal device can generate a 360° surround-view video in real time using image stitching algorithms. Video data is stored in the in-vehicle storage module. The in-vehicle communication module (supporting 4G / 5G networks) sends tiered alarm information to the owner's mobile app via push notifications. This information includes a risk level indicator, trigger time, vehicle status details, keyframe screenshots, and links to 360° surround-view video clips, ensuring the owner quickly understands any abnormal vehicle conditions. The owner can convert voice signals into text commands via the app and transmit them to the in-vehicle terminal through the communication link. After command parsing and authorization verification, the in-vehicle terminal controls the corresponding in-vehicle system to execute the operation and plays the voice signal through the external speaker. Furthermore, when the vehicle is unlocked or the windows are wide open, an alarm is triggered. If the scenario risk level is >=4 and the behavioral intent quantification assessment result is >8, the in-vehicle terminal device simultaneously activates actuators to close the windows and lock the doors.

[0200] Please refer to Figure 11 This application also provides a vehicle sentry mode alarm device, which can implement the above-mentioned vehicle sentry mode alarm method.

[0201] like Figure 8 As shown in the embodiment of this application, the vehicle sentry mode alarm device may include:

[0202] The acquisition module is used to acquire multimodal perception data of the vehicle in sentry mode; the multimodal perception data includes: vehicle status data, and basic environmental data and video image data around the vehicle.

[0203] The risk level classification module is used to determine the target risk level of the vehicle based on the status data and the basic environmental data; the target risk level is one of multiple risk levels in the vehicle parking scenario, and the initial alarm benchmark threshold sets corresponding to each of the multiple risk levels are different.

[0204] The threshold dynamic adjustment module is used to determine the human behavior data of the vehicle's surrounding environment based on the video image data, and to dynamically adjust the initial alarm benchmark threshold set corresponding to the target risk level based on the human behavior data, so as to obtain the adjusted alarm benchmark threshold set.

[0205] The alarm triggering module is used to trigger the vehicle sentry alarm based on the personnel behavior data and the adjusted alarm baseline threshold set.

[0206] In some embodiments, the risk level classification module is further configured to determine at least one target risk weighting coefficient that matches the state data from a plurality of preset risk weighting coefficients; the plurality of risk weighting coefficients are used to characterize the risk level of different vehicle states; perform fuzzy logic reasoning processing on the basic environmental data to obtain an initial risk classification result for the real-time parking scenario of the vehicle; and calculate the target risk level of the real-time parking scenario of the vehicle based on the target risk weighting coefficient and the initial risk classification result.

[0207] In some embodiments, the initial alarm baseline threshold set includes an initial distance threshold and an initial duration threshold; among the plurality of risk levels, risk level M is higher than risk level N, the initial distance threshold of risk level M is greater than the initial distance threshold of risk level N, and the initial duration threshold of risk level M is less than the initial duration threshold of risk level N.

[0208] In some embodiments, the personnel behavior data includes trajectory tracking results. The threshold dynamic adjustment module is further configured to perform personnel detection processing on the video image data to obtain the detection results of at least one target person in the environment surrounding the vehicle; the detection results are used to characterize the spatial pose information of the at least one target person; and, based on the detection results, perform cross-frame trajectory tracking processing on the at least one target person to obtain the trajectory tracking results of the at least one person target.

[0209] In some embodiments, the personnel behavior data further includes behavior classification results, and the threshold dynamic adjustment module is further used to perform behavior detection processing on the video image data to obtain the behavior classification results of at least one target person in the vehicle's surrounding environment.

[0210] The alarm triggering module is also used to acquire vehicle vibration data; when the vehicle vibration data is greater than or equal to a preset vibration threshold, and the behavior classification result indicates that at least one target person has performed abnormal behavior on the vehicle, the vehicle sentry alarm is triggered immediately.

[0211] In some embodiments, the threshold dynamic adjustment module is further configured to perform behavioral intent quantification processing on the personnel behavior data to obtain a behavioral intent score result; perform correction processing on the behavioral intent score result based on the state data to obtain a corrected behavioral intent score result; and perform nonlinear adjustment on the initial alarm benchmark threshold set corresponding to the target risk level based on the matching result of the corrected behavioral intent score result and the preset score interval.

[0212] In some embodiments, the threshold dynamic adjustment module is further configured to acquire the detection results of the door opening and closing actions of adjacent vehicles in the vehicle sentry mode; when the detection results indicate that the door opening and closing actions of adjacent vehicles do not trigger vehicle vibration, the module increases the target distance threshold in the target alarm reference threshold set and extends the target duration threshold in the target alarm reference threshold set; the target alarm reference threshold set includes: the initial alarm reference threshold set corresponding to the target risk level or the adjusted alarm reference threshold set.

[0213] In some embodiments, the adjusted alarm baseline threshold set includes an adjusted distance threshold and an adjusted duration threshold;

[0214] The alarm triggering module is further configured to trigger a vehicle sentry alarm when the personnel behavior data indicates that the real-time distance between at least one target person and the vehicle is less than or equal to the adjusted distance threshold, and the duration is greater than or equal to the adjusted duration threshold; the duration is the duration during which the real-time distance is less than or equal to the adjusted distance threshold; and to immediately trigger a vehicle sentry alarm when the personnel behavior data indicates that the real-time distance between at least one target person and the vehicle is less than or equal to the adjusted distance threshold, and the status data indicates that the vehicle is in a risky state.

[0215] In some embodiments, the alarm triggering module is further configured to acquire user instruction data sent by a target terminal associated with the vehicle, and control the vehicle and play user voice signals through an external vehicle speaker according to the user instruction data; the target terminal is a terminal device that receives and displays alarm information when the vehicle sentry alarm is triggered; the vehicle sentry alarm is triggered when the status data indicates that the vehicle is in a risky state; and the vehicle sentry alarm is triggered and the vehicle is controlled to de-risk when the status data indicates that the vehicle is in a risky state, the target risk level is greater than or equal to a preset level threshold, and the behavioral intent score result corresponding to the personnel behavior data is greater than or equal to a preset score threshold.

[0216] It should be noted that the specific implementation of the vehicle sentry mode alarm device provided in this application embodiment is basically the same as the specific implementation of the vehicle sentry mode alarm method described above, and will not be repeated here.

[0217] Please see Figure 12 This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-mentioned alarm method for vehicle sentry mode.

[0218] In some embodiments, the electronic device can be any smart terminal such as a tablet computer, smartphone, in-vehicle hardware platform (e.g., in-vehicle computer), or wearable device.

[0219] like Figure 12 As shown, the electronic device provided in this application embodiment may include:

[0220] The processor 1201 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0221] The memory 1202 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1202 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1202 and is called and executed by the processor 1201 to execute the vehicle sentry mode alarm method of the embodiments of this application.

[0222] The input / output interface 1203 is used to implement information input and output;

[0223] The communication interface 1204 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0224] Bus 1205 transmits information between various components of the device (e.g., processor 1201, memory 1202, input / output interface 1203, and communication interface 1204);

[0225] The processor 1201, memory 1202, input / output interface 1203 and communication interface 1204 are connected to each other within the device via bus 1205.

[0226] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described alarm method for vehicle sentry mode.

[0227] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0228] This application also provides a computer program product, including a computer program. The steps implemented by the computer program when executed by a processor are basically the same as those in the specific embodiments of the alarm method for vehicle sentry mode described above, and will not be repeated here.

[0229] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0230] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0231] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0232] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0233] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0234] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0235] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or units may be electrical, mechanical, or other forms.

[0236] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0237] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0238] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0239] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method of alerting in a vehicle sentry mode, characterized by, The method comprises: acquiring multi-modal perception data in a vehicle sentry mode; the multi-modal perception data comprises: state data of the vehicle, and, basic environment data and video image data around the vehicle; determining a target risk level of the vehicle based on the state data and the basic environment data; the target risk level is one of a plurality of risk levels of a vehicle parking scene, and each of the plurality of risk levels corresponds to a set of initial alarm reference thresholds that are different from each other; determining personnel behavior data of the environment around the vehicle based on the video image data, and performing behavior intention quantization processing on the personnel behavior data to obtain a behavior intention score result; performing correction processing on the behavior intention score result based on the state data to obtain a corrected behavior intention score result; performing nonlinear adjustment on the set of initial alarm reference thresholds corresponding to the target risk level based on a matching result of the corrected behavior intention score result and a preset score interval to obtain a set of adjusted alarm reference thresholds; triggering a vehicle sentry alarm based on the personnel behavior data and the set of adjusted alarm reference thresholds.

2. The method of claim 1, wherein, The determination of the target risk level of the vehicle based on the state data and the basic environment data comprises: determining at least one target risk weighting coefficient matched with the state data from a plurality of preset risk weighting coefficients; the plurality of risk weighting coefficients are used to represent the risk degree of different states of the vehicle; performing fuzzy logic reasoning processing on the basic environment data to obtain an initial risk classification result of a real-time vehicle parking scene; calculating the target risk level of the real-time vehicle parking scene based on the target risk weighting coefficient and the initial risk classification result.

3. The method of claim 1, wherein, The set of initial alarm reference thresholds comprises an initial distance threshold and an initial time threshold; in the plurality of risk levels, a risk level M is higher than a risk level N, the initial distance threshold of the risk level M is greater than the initial distance threshold of the risk level N, and the initial time threshold of the risk level M is less than the initial time threshold of the risk level N.

4. The method of claim 1, wherein, The method further comprises: acquiring a door opening and closing action detection result of a neighboring vehicle in the vehicle sentry mode; in a case where the door opening and closing action detection result indicates that the door opening and closing action of the neighboring vehicle does not trigger vehicle vibration, increasing a target distance threshold in a target alarm reference threshold set, and, prolonging a target time threshold in the target alarm reference threshold set; the target alarm reference threshold set comprises: the set of initial alarm reference thresholds corresponding to the target risk level or the set of adjusted alarm reference thresholds.

5. The method of claim 1, wherein, The set of adjusted alarm reference thresholds comprises an adjusted distance threshold and an adjusted time threshold; The triggering of the vehicle sentry alarm based on the personnel behavior data and the set of adjusted alarm reference thresholds comprises at least one of the following: trigger the vehicle sentry alarm when the personnel behavior data indicates that the real-time distance between at least one target person and the vehicle is less than or equal to the adjusted distance threshold and the duration is greater than or equal to the adjusted duration threshold; the duration is a duration during which the real-time distance is less than or equal to the adjusted distance threshold; trigger the vehicle sentry alarm immediately when the personnel behavior data indicates that the real-time distance between at least one target person and the vehicle is less than or equal to the adjusted distance threshold and the state data indicates that the vehicle is in a risk state.

6. The method of claim 1, wherein, The method further comprises at least one of the following: acquire user instruction data sent by a target terminal associated with the vehicle, and control the vehicle and play a user voice signal through a vehicle-mounted external loudspeaker according to the user instruction data; the target terminal is a terminal device that receives alarm information and displays when the vehicle sentry alarm is triggered; trigger the vehicle sentry alarm when the state data indicates that the vehicle is in a risk state; trigger the vehicle sentry alarm and control the vehicle to remove the risk state when the state data indicates that the vehicle is in a risk state, the target risk level is greater than or equal to a preset level threshold, and the behavior intention score result corresponding to the personnel behavior data is greater than or equal to a preset score threshold.

7. The method according to any one of claims 1 to 6, characterized in that, The personnel behavior data comprises a trajectory tracking result and a behavior classification result, and the personnel behavior data of the vehicle surrounding environment is determined based on the video image data, comprising: perform personnel detection processing on the video image data to obtain a detection result of at least one target person in the vehicle surrounding environment; the detection result is used to represent spatial posture information of the at least one target person; perform cross-frame trajectory tracking processing on the at least one target person based on the detection result to obtain a trajectory tracking result of the at least one target person; perform behavior detection processing on the video image data to obtain a behavior classification result of at least one target person in the vehicle surrounding environment.

8. The method of claim 7, wherein, The method further comprises: acquire vehicle vibration data; trigger the vehicle sentry alarm immediately when the vehicle vibration data is greater than or equal to a preset vibration threshold and the behavior classification result indicates that the at least one target person exerts an abnormal behavior on the vehicle.

9. An electronic device, comprising: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the alarm method of the vehicle sentry mode according to any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the alarm method of the vehicle sentry mode according to any one of claims 1 to 8.

11. A computer program product, characterised in that, The computer program product comprises a computer program, and the computer program is executed by the processor to implement the alarm method of the vehicle sentry mode according to any one of claims 1 to 8.

Images