Control method and device of sentry mode, electronic equipment and storage medium

Abstract

The invention relates to a sentry mode control method and device, electronic equipment and a storage medium, relates to the technical field of vehicle control, and is used for solving the problem of excessive power consumption after a vehicle starts a sentry mode in the related technology. The method comprises the steps that under the condition that a vehicle starts a sentry mode, environment information around the vehicle is acquired; identifying the type of the current scene of the vehicle based on the environment information around the vehicle; under the condition that the vehicle is currently in the high-risk scene, using a first sentry mode; under the condition that the vehicle is currently in the low-risk scene, using a second sentry mode; wherein the monitoring performance of the first sentry mode is higher than the monitoring performance of the second sentry mode, and the adaptive sentry mode can be used according to the environmental risk level, so that hardware resources are accurately allocated, the real-time change of the environment is adapted, the monitoring definition of a high-risk scene is guaranteed, and the invalid power consumption of a low-risk scene is reduced; therefore, high-risk, high-power-consumption and low-risk and low-power-consumption precise regulation and control are realized.

Description

Technical Field

[0001] The present invention relates to the field of vehicle control technology, and in particular to a sentry mode control method, device, electronic device and storage medium. Background Art

[0002] Vehicle Sentry Mode continuously monitors the surrounding environment through sensors and cameras and is an important technology for ensuring vehicle safety.

[0003] In related technologies, after the vehicle enters the parking state, the vehicle sentry mode is manually turned on or automatically triggered, the sensors and cameras continue to work, the camera records video at the highest resolution, and the radar scans the surrounding environment at a high frequency.

[0004] However, in the related art, when the vehicle sentry mode is turned on, the vehicle's sensors and cameras all maintain full load operation, resulting in excessive power consumption. Summary of the Invention

[0005] The object of the present invention is to provide a sentry mode control method, device, electronic device and storage medium, aiming to solve the problem of how to reduce power consumption after the vehicle turns on the sentry mode.

[0006] In a first aspect, a method for controlling a sentinel mode is provided, the method comprising: obtaining environmental information around the vehicle when the sentinel mode is turned on for the vehicle; identifying the type of scene the vehicle is currently in based on the environmental information around the vehicle; using a first sentinel mode when the vehicle is currently in a high-risk scene; and using a second sentinel mode when the vehicle is currently in a low-risk scene; wherein the monitoring performance of the first sentinel mode is higher than the monitoring performance of the second sentinel mode.

[0007] In a second aspect, a sentinel mode control device is provided, comprising: an acquisition unit, an identification unit, and a processing unit.

[0008] The acquisition unit is used to acquire environmental information around the vehicle when the vehicle is in sentry mode.

[0009] The identification unit is used to identify the type of scene the vehicle is currently in based on the environmental information around the vehicle.

[0010] The processing unit is configured to use a first sentry mode when the vehicle is currently in a high-risk scenario.

[0011] The processing unit is further configured to use a second sentinel mode when the vehicle is currently in a low-risk scenario, wherein the monitoring performance of the first sentinel mode is higher than the monitoring performance of the second sentinel mode.

[0012] In a third aspect, an electronic device is provided, comprising: a processor and a memory; the memory is used to store processor-executable instructions; wherein the processor is configured to execute instructions to implement the method of the above-mentioned first aspect and any possible implementation method thereof.

[0013] In a fourth aspect, a computer-readable storage medium is provided. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method of the above-mentioned first aspect and any possible implementation method thereof.

[0014] In a fifth aspect, a computer program product is provided, which includes computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method of the first aspect and any possible implementation method thereof.

[0015] Beneficial effects of the present invention:

[0016] The embodiment of the present invention obtains environmental information about the vehicle's surroundings when the vehicle is in sentry mode, and identifies the type of scene the vehicle is currently in based on the environmental information about the vehicle's surroundings; when the vehicle is currently in a high-risk scene, the first sentinel mode is used; when the vehicle is currently in a low-risk scene, the second sentinel mode is used, and the monitoring performance of the first sentinel mode is limited to be higher than the monitoring performance of the second sentinel mode. Adaptive sentinel modes can be used according to the environmental risk level to accurately allocate hardware resources and adapt to real-time changes in the environment, so that high-risk scenes ensure monitoring clarity and low-risk scenes reduce ineffective power consumption, thereby achieving precise control of high-risk high power consumption and low-risk low power consumption. For example, when a vehicle moves from a home garage to an open-air parking lot, the monitoring intensity is automatically increased to ensure safety protection without delay. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 A schematic flow chart of a sentinel mode control method provided by an embodiment of the present invention;

[0019] Figure 2 A schematic diagram of a data processing flow provided by an embodiment of the present invention;

[0020] Figure 3 A schematic diagram of a modular system architecture provided by an embodiment of the present invention;

[0021] Figure 4 A schematic structural diagram of a sentinel mode control device provided by an embodiment of the present invention;

[0022] Figure 5 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The following description of exemplary embodiments of the present disclosure is provided in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0024] In the embodiments of the present invention, the terms "first," "second," "third," "fourth," "fifth," and "sixth" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly specifying the number of the technical features indicated. Therefore, a feature specified as "first," "second," "third," "fourth," "fifth," and "sixth" may explicitly or implicitly include one or more of the features.

[0025] In embodiments of the present invention, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0026] Vehicle sentry mode continuously monitors the surrounding environment through sensors and cameras, making it a crucial technology for ensuring vehicle safety. However, related technologies often suffer from excessive energy consumption: continuously operating sensors, cameras, and data processing modules rapidly deplete the power battery, significantly shortening the vehicle's range. For example, traditional sentry mode consumes 0.8-1.2 kWh per hour, while 24-hour parking consumes 19.2-28.8 kWh, representing 15%-20% of an electric vehicle's range. Furthermore, existing technologies lack the ability to dynamically perceive environmental risks and are unable to adjust monitoring intensity based on actual safety needs, leading to the contradiction of "insufficient protection in high-risk areas and wasted resources in low-risk areas."

[0027] Currently, related technologies include two control methods for vehicle sentry mode.

[0028] One approach utilizes a hardware architecture combining multiple sensors, high-definition cameras, and a local processor. For example, Tesla's Sentry Mode utilizes four surround-view cameras (front, side, and rear), millimeter-wave radar, and ultrasonic sensors, with data processed in real time by an onboard MCU (such as the NVIDIA Jetson). Once the vehicle is parked, Sentry Mode is manually activated or automatically triggered (e.g., upon detecting a locked vehicle). The sensors and cameras continue to operate, with the cameras recording video at the highest resolution (e.g., 4K) and the radar scanning the surrounding environment at 20Hz. Upon detecting movement (e.g., an approaching pedestrian) or vibration, the system wakes up the storage module and saves the video data before and after the abnormal event.

[0029] However, this method wastes a lot of energy: regardless of the level of environmental risk, it maintains full load operation, resulting in excessive power consumption. For example, when the vehicle is parked in a safe area such as a home garage, it still runs at 4K resolution and full sensors, and the energy consumption is no different from that in high-risk areas. The monitoring strategy is fixed: it relies on preset rules (such as triggering HD recording only when a moving object is detected), and cannot actively identify the level of environmental risk. For example, open-air parking lots with dense traffic and closed communities use the same monitoring intensity. High-risk areas may miss shots due to excessive data volume, while low-risk areas will waste electricity. High computing resource usage: High-definition video processing and multi-sensor data fusion require high on-board computing power, which may affect the operating efficiency of other vehicle systems (such as autonomous driving).

[0030] Another approach is to add a simple "zone marking" function to the traditional architecture. Car owners can manually set "safe zones" (low-risk scenarios, such as home addresses and company parking lots) through the in-vehicle system. Car owners mark low-risk scenarios in the system in advance. When the vehicle enters the marked area, the system automatically reduces the camera resolution (for example, from 4K to 1080P) and disables some non-critical sensors (such as ultrasonic sensors). When the vehicle leaves the marked area, the system resumes full-load monitoring mode.

[0031] However, this approach relies on manual marking: drivers must actively set safety zones, which doesn't cover temporary parking spots (such as unfamiliar parking lots), limiting its applicability. Risk assessment is crude: risk is determined solely based on whether a vehicle is marked, without considering the area's real-time security status (such as a temporary theft in a marked area), potentially leading to lagging protection strategies. Dynamic adaptability is poor: monitoring intensity cannot be adjusted in real time based on environmental changes (such as a sudden increase in foot traffic in a marked area), resulting in security vulnerabilities such as misjudging low risk when it is actually high risk.

[0032] Against this background, an embodiment of the present invention provides a method for controlling a sentinel mode to solve the problem of excessive power consumption when a vehicle turns on the sentinel mode in the related art.

[0033] For example, the execution entity of the sentinel mode control method provided in the embodiment of the present invention may be an on-board edge computing unit, a control system in the vehicle, or other vehicle-related control systems. No specific limitation is imposed on the execution entity of the method herein.

[0034] Figure 1 Schematic diagram of a control method of sentinel mode provided by an embodiment of the present invention. Figure 1 As shown, the control method of the sentry mode may include S101-S104.

[0035] S101. When a vehicle is in sentry mode, obtain environmental information around the vehicle.

[0036] Sentry Mode is a safety feature on the vehicle that uses multiple cameras and sensors to monitor the vehicle's surroundings. When a potential threat is detected, it automatically records video and sends an alert to the owner.

[0037] As a possible implementation, when a vehicle is in Sentry Mode, it can utilize its multiple cameras and sensors to acquire environmental information surrounding the vehicle. Specifically, the onboard camera can capture 360-degree image data of the vehicle's surroundings, while the vehicle's millimeter-wave radar and ultrasonic sensor can simultaneously collect distance data to form a visual-range multimodal dataset. The onboard camera supports 4K / 1080P / 720P resolution and a frame rate of 15-30 frames per second (fps). The resolution of the onboard camera is adjustable. The sampling frequency of the millimeter-wave radar can be 5-20 hertz (Hz), and the ranging range of the ultrasonic sensor can be 0.1-5 meters. After acquiring this data, it can be cleaned (i.e., removing noisy frames), normalized (i.e., normalizing image pixel values ​​to [0, 1]), and feature aligned (i.e., synchronizing image and distance data using timestamps) to produce processed data. Figure 2 A flow chart of data processing provided by an embodiment of the present invention. Figure 2 As shown, the data acquired by the on-board camera can be used as raw image data, and the data acquired by the millimeter-wave radar and ultrasonic sensor can be used as raw distance data. The raw image data is then subjected to Gaussian filtering for denoising and image normalization, while the raw distance data is then subjected to Kalman filtering for smoothing and distance value normalization. The processed raw image data and raw distance data are then synchronized using timestamps, aligning their timestamps. This improves data quality, reduces the interference of outliers on the reasoning of the following model, speeds up reasoning, and ensures a classification accuracy of ≥92% for the lightweight model. It also reduces the amount of invalid data stored, saving 50% of on-board storage resources.

[0038] For example, the environmental information around the vehicle may include information about people or objects around the vehicle, the flow of people around the vehicle, the distance between the vehicle and the surrounding people or objects, the speed of the surrounding people or objects relative to the vehicle, road conditions, weather conditions, and ambient sounds. The information about people or objects around the vehicle may include whether there are pedestrians, cyclists, other vehicles (e.g., cars, trucks, and motorcycles), animals, and road obstacles (e.g., construction site signs, fallen cargo, and other foreign objects) around the vehicle.

[0039] S102: Based on the environmental information surrounding the vehicle, identify the type of scene the vehicle is currently in.

[0040] As a possible implementation method, the types of scenarios can include high-risk scenarios and low-risk scenarios. A lightweight multimodal large model can be used to perform binary classification on the environmental information around the vehicle, thereby identifying the type of scenario the vehicle is currently in. Specifically, a MobileNet v3 or EfficientNet-Lite lightweight architecture can be used to build a risk detection model. The model parameter size is less than 5MB, the floating-point operation (FLOPs) is less than 1TOPS, and it is adapted to the edge computing requirements of an on-board MCU (such as Qualcomm SA8155P). The data processed in S101 is used as input. The data processed in S101 is multimodal features (i.e., image RGB features + distance point cloud features). The model first extracts the input image features and distance features separately, then performs multimodal feature fusion on the extracted features, and finally classifies the risk level based on the fused features, outputting a risk assessment value for the vehicle's current scenario. Based on this risk assessment value, the type of scenario the vehicle is currently in can be determined. The risk detection model, trained using a cross-entropy loss function, can achieve 95% classification accuracy on a dataset of over 100,000 annotated images. Feature-level fusion (combining image and distance features) further improves risk classification accuracy. Environmental risk assessment is performed on the vehicle's edge computing platform, avoiding cloud communication delays and reducing computing power consumption, bringing model inference power consumption to less than 1.5W.

[0041] For example, when the volume of pedestrians around the vehicle is greater than or equal to a preset threshold, the vehicle's current scene can be determined to be high-risk. When the volume of pedestrians around the vehicle is less than the threshold, the vehicle's current scene can be determined to be low-risk. When there are people or objects around the vehicle whose speed relative to the vehicle is greater than or equal to a preset relative speed threshold, the vehicle's current scene can be determined to be high-risk. When the ambient sound around the vehicle is greater than or equal to a preset volume threshold, the vehicle's current scene can be determined to be high-risk.

[0042] S103: When the vehicle is currently in a high-risk scenario, use the first sentinel mode.

[0043] S104: When the vehicle is currently in a low-risk scenario, use the second sentinel mode; wherein the monitoring performance of the first sentinel mode is higher than the monitoring performance of the second sentinel mode.

[0044] As a possible implementation method, environmental data can be continuously collected and the type of scene the vehicle is currently in can be identified.

[0045] When the vehicle's current scenario is identified as a high-risk scenario, the first sentinel mode is used. When the vehicle's current scenario is identified as a low-risk scenario, the second sentinel mode is used. When the type of scenario changes (for example, from a low-risk scenario to a high-risk scenario), the vehicle's sentinel mode can be dynamically switched to ensure that the first sentinel mode is used when the vehicle is in a high-risk scenario, and the second sentinel mode is used when the vehicle is in a low-risk scenario.

[0046] For example, by using 10Hz high-frequency data acquisition and 200ms power switching delay to ensure that the monitoring intensity is adjusted immediately when the environmental risk changes (for example, when a vehicle moves from a garage to an open-air parking lot), the abnormal event capture rate can be maintained at above 98%.

[0047] The embodiment of the present invention obtains environmental information about the vehicle's surroundings when the vehicle is in sentry mode, and identifies the type of scene the vehicle is currently in based on the environmental information about the vehicle's surroundings; when the vehicle is currently in a high-risk scene, the first sentinel mode is used; when the vehicle is currently in a low-risk scene, the second sentinel mode is used, and the monitoring performance of the first sentinel mode is limited to be higher than the monitoring performance of the second sentinel mode. Adaptive sentinel modes can be used according to the environmental risk level to accurately allocate hardware resources and adapt to real-time changes in the environment, so that high-risk scenes ensure monitoring clarity and low-risk scenes reduce ineffective power consumption, thereby achieving precise control of high-risk high power consumption and low-risk low power consumption. For example, when a vehicle moves from a home garage to an open-air parking lot, the monitoring intensity is automatically increased to ensure safety protection without delay.

[0048] In some embodiments, the first sentinel mode includes: the vehicle's camera uses a first resolution to shoot; the second sentinel mode includes: the vehicle's camera uses a second resolution to shoot, and the first resolution is higher than the second resolution.

[0049] For example, the vehicle's camera may include a camera. When the vehicle is currently in a high-risk scenario, the camera resolution can be increased to 4K (3840×2160) with a frame rate of 30fps when the first Sentry Mode is used. When the vehicle is currently in a low-risk scenario, the camera resolution can be reduced to 720P (1280×720) with a frame rate of 15fps when the second Sentry Mode is used.

[0050] In some embodiments, the first sentinel mode includes: the vehicle's radar sampling at a first sampling frequency; the second sentinel mode includes: the vehicle's radar sampling at a second sampling frequency, and the first sampling frequency is higher than the second sampling frequency.

[0051] For example, a vehicle's radar may include a millimeter-wave radar. When the vehicle is in a high-risk scenario and uses the first Sentinel Mode, the millimeter-wave radar sampling frequency is increased to 20 Hz. When the vehicle is in a low-risk scenario and uses the second Sentinel Mode, the millimeter-wave radar sampling frequency can be appropriately reduced based on actual scenario requirements.

[0052] In some embodiments, the first sentinel mode includes: the vehicle's ultrasonic sensors are in a fully activated state; the second sentinel mode includes: the vehicle's non-critical ultrasonic sensors are in a dormant state.

[0053] As a possible implementation, when the vehicle is currently in a high-risk scenario, when using the first sentinel mode, the vehicle's ultrasonic sensors are fully activated, that is, all the vehicle's ultrasonic sensors are working at high load. When the vehicle is currently in a low-risk scenario, when using the second sentinel mode, the vehicle's non-critical ultrasonic sensors (such as the vehicle's side ultrasonic sensors) can be put into a dormant state, leaving only the vehicle's front ultrasonic sensor.

[0054] As another possible implementation, when the vehicle is currently in a high-risk scenario, the first Sentry Mode can be used, with the vehicle's ultrasonic sensors, radar, and camera all fully activated. When the vehicle is currently in a low-risk scenario, the second Sentry Mode can be used, with the vehicle's non-critical ultrasonic sensors, non-critical radar, and auxiliary cameras in a dormant state, leaving only the vehicle's front ultrasonic sensor, front radar, and main camera active.

[0055] Optionally, based on the above embodiments, when the vehicle is currently in a high-risk scenario, when using the first sentinel mode, the vehicle's camera can be controlled to shoot with the first resolution, the vehicle's radar can be sampled with the first sampling frequency, and the vehicle's ultrasonic sensor can be fully activated, or the vehicle's camera can be controlled to shoot with the first resolution and the vehicle's radar can be sampled with the first sampling frequency, or the vehicle's camera can be controlled to shoot with the first resolution and the vehicle's ultrasonic sensor can be fully activated, or the vehicle's radar can be controlled to sample with the first sampling frequency and the vehicle's ultrasonic sensor can be fully activated.

[0056] When the vehicle is currently in a low-risk scenario, when using the second sentinel mode, the vehicle's camera can be controlled to shoot with the second resolution, the vehicle's radar can be sampled with the second sampling frequency, and the vehicle's non-critical ultrasonic sensor can be in a dormant state, or the vehicle's camera can be controlled to shoot with the second resolution and the vehicle's radar can be sampled with the second sampling frequency, or the vehicle's camera can be controlled to shoot with the second resolution and the vehicle's non-critical ultrasonic sensor can be in a dormant state, or the vehicle's radar can be controlled to sample with the second sampling frequency and the vehicle's non-critical ultrasonic sensor can be in a dormant state.

[0057] The above embodiment limits the hardware usage parameters of the vehicle when using different sentinel modes under different environmental risk levels, and can accurately allocate hardware resources according to the environmental risk level, so that the high-risk area can ensure monitoring clarity and the safe area, that is, the low-risk area, can reduce invalid power consumption.

[0058] In some embodiments, identifying the type of scene the vehicle is currently in based on environmental information surrounding the vehicle may include: identifying the type of scene the vehicle is currently in based on the vehicle type and environmental information. The vehicle type includes any of the following: a household vehicle, a logistics vehicle, a police vehicle, a public transportation vehicle, and an ambulance.

[0059] For example, different types of vehicles have different application scenarios, and different types of vehicles have different perceptions of environmental information in different application scenarios. For example, when a family car is in a home parking scenario, the garage environment is usually identified as a low-risk scenario through the risk detection model. It can automatically switch to 720P camera + 5Hz radar sampling, reducing energy consumption by 50%, and solving the problem of electric vehicle battery life loss when parked at night. When a family car is in a commercial parking lot, crowded scenes are usually judged as high-risk scenes based on the flow of people. 4K high-definition monitoring + full sensor operation can be enabled to ensure that video evidence of incidents such as scratches and thefts is retained.

[0060] In a specific embodiment, experiments have shown that for models such as Tesla Model 3 and BYD Han, the hourly power consumption in low-risk scenarios in Sentry Mode is reduced from 0.8kWh to 0.4kWh. Parking for 12 hours a day can reduce energy consumption by 4.8kWh, and the equivalent range can be increased by 6-8 kilometers.

[0061] For example, if the vehicle is a logistics vehicle, when the logistics vehicle stops at a service area during long-distance transportation, the on-board camera can perform image recognition of the surrounding environment of the truck (such as remote corners, whether there are people wandering around), and determine the type of scene the vehicle is currently in based on the image data. In high-risk scenarios, infrared night vision + high-frequency radar scanning is enabled to prevent cargo theft.

[0062] At the same time, in fleet management scenarios, the power consumption strategy of each vehicle can be uniformly configured through the cloud platform, and the monitoring intensity can be automatically reduced in low-risk scenarios to reduce the overall power consumption of the fleet.

[0063] For example, if the vehicle is a bus or a school bus, when the vehicle is parked in a parking lot or bus stop at night, if the model determines that it is a low-risk scenario, only the 720P monitoring of the front camera will be retained, and other sensors will be dormant, which reduces power consumption by 60% compared to traditional solutions and extends the life of the vehicle battery.

[0064] This embodiment can achieve full coverage of the vehicle field and reduce energy consumption of different vehicles in different environments by identifying the type of scene the vehicle is currently in based on the vehicle type and environmental information.

[0065] In some embodiments, the above-mentioned identification of the type of scene in which the vehicle is currently located based on the type of vehicle and environmental information may include: S201-S202.

[0066] S201: When the vehicle is a household vehicle, input environmental information into a risk detection model to obtain a risk assessment value of the current scenario by the risk detection model.

[0067] Among them, environmental information includes pedestrian flow, the relative distance between the vehicle and surrounding pedestrians or objects, the speed of surrounding pedestrians or objects relative to the vehicle, road conditions, weather conditions and environmental sounds.

[0068] The principle of the risk detection model can refer to the relevant description of the risk detection model in S102 above, which will not be repeated here.

[0069] S202: When the risk assessment value is greater than the threshold, determine that the scene type in which the vehicle is currently located is a high-risk scene.

[0070] For example, when a family car drives from a garage to a crowded area, the currently detected pedestrian flow data is processed and input into the risk detection model. The risk detection model outputs a risk assessment value of 0.85 for the current scene, and the threshold is 0.5. It can be determined that the current scene type of the vehicle is a high-risk scene.

[0071] In some embodiments, based on the type of vehicle and environmental information, identifying the type of scene the vehicle is currently in includes: S301-S302.

[0072] S301: When the vehicle is a police vehicle, identify whether the vehicle is on duty based on environmental information.

[0073] S302: When the vehicle is on duty, determine that the scene type the vehicle is currently in is a high-risk scene.

[0074] For example, when a police vehicle is parked in an area with complex public security, it can be identified based on environmental information whether the police vehicle is on duty. If the police vehicle is on duty, it is determined that the current scene type of the police vehicle is a high-risk scene. The first sentinel mode can be used to enable 4K monitoring + full sensors to synchronously record the surrounding environment and provide evidence for law enforcement.

[0075] Optionally, when the vehicle type is an ambulance, the parking location of the ambulance can be identified based on environmental information. When the ambulance is detected to be parked in the emergency area of ​​the hospital, it is determined that the current scene type of the ambulance is a low-risk scene. The second sentinel mode can be used to enable 1080P monitoring + key sensors to ensure equipment safety and avoid interference with the emergency process.

[0076] In some embodiments, the control method of the sentinel mode provided by the embodiments of the present invention can also be applied to intelligent security robots (for example, park patrol robots, warehouse security robots, etc.), smart home security systems (for example, villa courtyard monitoring, shop security, etc.), and drone inspection systems (for example, power inspection, agricultural monitoring, etc.).

[0077] For example, when this method is applied to an intelligent security robot, the on-board camera can be replaced with a panoramic gimbal camera, and the distance sensor can be replaced with a lidar; the risk detection model adds pedestrian behavior recognition (such as "wandering" and "carrying", etc.), and dynamically adjusts the camera resolution and radar scanning frequency; low-risk scenarios (such as open parks) switch to 720P+10Hz radar, reducing power consumption by 40%.

[0078] When applied to smart home security systems, this method can replace the hardware with home cameras (such as Hikvision IPCs with 4K / 1080P switching) and ultrasonic ranging modules. The risk detection model is optimized to identify home security events such as "stranger intrusion" and "object removal." For low-risk scenarios (such as normal family activities), the recording resolution is reduced, saving 50% of storage hard drive space.

[0079] When applied to drone inspection systems, this method can deploy risk detection models on the drone's edge (e.g., the computing unit of the DJI M300 RTK). 4K aerial photography is activated when identifying foreign objects (high-risk) near power transmission lines, while switching to 1080P in low-risk scenarios. Dynamically adjusting camera power consumption and image transmission bitrate can extend drone flight time by 30%.

[0080] When applying the sentry mode control method provided by the embodiments of the present invention in different scenarios, the following conditions need to be met: a camera with adjustable resolution (supporting at least two resolution switching levels); a programmable sensor (such as a radar / ultrasonic module that supports sampling frequency adjustment); an edge computing unit (computing power ≥ 2TOPS, supporting INT8 quantized inference); the ability to use a multimodal data time synchronization algorithm (image and distance data timestamp alignment error < 50ms); and fine-tuning the risk detection model for new scenarios (for example, a security robot needs to add a "robot movement" background filtering mechanism).

[0081] In some embodiments, Figure 3 A schematic diagram of a modular system architecture provided by an embodiment of the present invention. Figure 3 As shown in the figure, the modular system architecture is a distributed structure consisting of "data acquisition module - data processing module - environment classification module - power regulation module - control module - camera / sensor hardware." The data acquisition module is used for synchronous data acquisition from multimodal sensors (such as camera / radar / ultrasonic); the environment classification module has a built-in lightweight model inference engine; and the power regulation module controls hardware parameter switching via the CAN bus.

[0082] For example, the data acquisition module's hardware components may include an 8MP front camera (supporting 4K output, model IMX477), two 2MP cameras (720P, model IMX377) on both sides, a 5MP rear camera (1080P, model IMX386); a 77GHz millimeter-wave radar (with adjustable sampling frequency, model TI IWR1642), and a 6-channel ultrasonic sensor (ranging accuracy of ±2cm). The data acquisition module operates by automatically activating sentry mode after the vehicle has been stationary for 5 minutes. The sensor collects data at a preset frequency and transmits it to the data processing module via the LVDS bus.

[0083] The data processing module's hardware components include an onboard DSP (TI TDA4VM) with an integrated NPU computing power of 2TOPS. The processing flow of the data processing module includes image data: performing noise reduction and size normalization (scaling to 224×224); distance data: filtering invalid point clouds (e.g., distance > 5m) and converting them into 128-dimensional feature vectors; data fusion: aligning multimodal data using timestamps to generate model input tensors.

[0084] Core components of the environment classification module include a lightweight model inference engine (optimized with TensorFlow Lite and quantized to INT8 format); a risk level mapping table that stores the correspondence between classification probabilities and risk levels (e.g., a probability greater than 0.7 is considered high risk). The inference process of the environment classification module includes inputting multimodal features, model forward propagation, softmax activation to output risk probabilities, and a table lookup to map risk levels.

[0085] The power regulation module's hardware interfaces include camera control (configuring resolution and frame rate via the MIPI-CSI interface), sensor control (adjusting radar sampling frequency via the SPI bus, and controlling sensor power on and off via the GPIO port). Table 1 shows the power regulation module's power consumption strategy.

[0086] Table 1

[0087] Risk Level Camera resolution Radar frequency Number of sensors Power consumption (W) High-risk scenarios 4K 20Hz full amount 12 Low-risk scenarios 720P 5Hz part 6

[0088] In a specific embodiment, Table 2 compares the experimental results of the sentinel mode control method provided in an embodiment of the present invention with the sentinel mode control method in the related art. As shown in Table 2, Table 2 includes the comparison results of hourly power consumption, abnormal event missed reporting rate, environmental risk misjudgment rate, and hardware computing power utilization of the sentinel mode control method provided in an embodiment of the present invention and the sentinel mode control method in the related art.

[0089] Table 2

[0090]

[0091] The control module's core components include an onboard MCU (STM32H743VI) running at 480MHz. The module's control logic coordinates the operating timing of each module, ensuring synchronization of the data acquisition, processing, classification, and control processes. It also communicates with the vehicle's power management system via the CAN bus to obtain battery status and optimize power consumption strategies.

[0092] The above mainly introduces the solution from the perspective of method. In order to realize the above functions, the control device or electronic device of the sentry mode includes hardware structures and / or software modules corresponding to the execution of each function. It should be easy for those skilled in the art to realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present invention can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0093] In an embodiment of the present invention, the control device or electronic device in sentry mode can be divided into functional modules according to the above method. For example, the control device or electronic device in sentry mode can include functional modules corresponding to the functional divisions, or two or more functions can be integrated into one processing module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present invention is schematic and is only a logical functional division. In actual implementation, there may be other division methods.

[0094] In an exemplary embodiment, the present invention further provides a sentinel mode control device, which can be used to implement the sentinel mode control method as described in the aforementioned embodiment. Figure 4 This is a schematic diagram of the structure of a sentinel mode control device provided by an embodiment of the present invention. Figure 4 As shown, the control device of the sentry mode may include: an acquisition unit 401 , an identification unit 402 and a processing unit 403 .

[0095] The acquisition unit 401 is configured to acquire environmental information surrounding the vehicle when the vehicle is in sentry mode.

[0096] The identification unit 402 is configured to identify the type of scene the vehicle is currently in based on the environmental information surrounding the vehicle.

[0097] The processing unit 403 is configured to use the first sentinel mode when the vehicle is currently in a high-risk scenario.

[0098] The processing unit 403 is further configured to use the second sentinel mode when the vehicle is currently in a low-risk scenario, wherein the monitoring performance of the first sentinel mode is higher than that of the second sentinel mode.

[0099] In some embodiments, the first sentinel mode includes: the vehicle's camera uses a first resolution to shoot; the second sentinel mode includes: the vehicle's camera uses a second resolution to shoot, and the first resolution is higher than the second resolution.

[0100] In some embodiments, the first sentinel mode includes: the vehicle's radar sampling at a first sampling frequency; the second sentinel mode includes: the vehicle's radar sampling at a second sampling frequency, and the first sampling frequency is higher than the second sampling frequency.

[0101] In some embodiments, the first sentinel mode includes: the vehicle's ultrasonic sensors are in a fully activated state; the second sentinel mode includes: the vehicle's non-critical ultrasonic sensors are in a dormant state.

[0102] In some embodiments, the identification unit 402 is specifically used to identify the type of scene the vehicle is currently in based on the type of vehicle and environmental information; wherein the type of vehicle includes any of the following: household vehicles, logistics vehicles, police vehicles, public transportation vehicles, and ambulances.

[0103] In some embodiments, the identification unit 402 is specifically used to input environmental information into the risk detection model when the type of the vehicle is a household vehicle, to obtain a risk assessment value of the risk detection model for the current scene; wherein the environmental information includes pedestrian flow, the relative distance between the vehicle and surrounding pedestrians or objects, the speed of surrounding pedestrians or objects relative to the vehicle, road conditions, weather conditions and environmental sounds; when the risk assessment value is greater than a threshold, it is determined that the current scene type of the vehicle is a high-risk scene.

[0104] In some embodiments, the identification unit 402 is specifically used to identify whether the vehicle is on duty based on environmental information when the vehicle is a police vehicle; when the vehicle is on duty, determine that the current scene type of the vehicle is a high-risk scene.

[0105] In an exemplary embodiment, the present invention further provides an electronic device, such as Figure 5 As shown, the electronic device 500 provided by the embodiment of the present invention includes but is not limited to: a processor 501 and a memory 502.

[0106] The memory 502 is used to store executable instructions of the processor 501. It is understandable that the processor 501 is configured to execute instructions to implement the method in the above embodiment.

[0107] It should be noted that those skilled in the art can understand that Figure 5 The structure of the electronic device 500 shown in FIG. 5 does not limit the electronic device 500. The electronic device 500 may include Figure 5More or fewer components may be shown, or certain components may be combined, or the components may be arranged differently.

[0108] The processor 501 is the control center of the electronic device 500. It connects the various components of the electronic device 500 using various interfaces and lines. By running or executing software programs and / or modules stored in the memory 502 and accessing data stored in the memory 502, it performs various functions of the electronic device 500 and processes data, thereby monitoring the entire electronic device 500. The processor 501 may include one or more processing units. Optionally, the processor 501 may integrate an application processor and a modem processor, wherein the application processor primarily processes the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into the processor 501.

[0109] The memory 502 can be used to store software programs and various data. The memory 502 may primarily include a program storage area and a data storage area. The program storage area may store an operating system and application programs required by at least one functional module (such as a determination unit, a processing unit, etc.). Furthermore, the memory 502 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0110] In an exemplary embodiment, a computer-readable storage medium including instructions is further provided, such as a memory 502 including instructions. The instructions can be executed by the processor 501 of the electronic device 500 to implement the method in the above embodiment.

[0111] In actual implementation, Figure 4 The functions of the acquisition unit 401, the identification unit 402 and the processing unit 403 can all be obtained by Figure 5 The processor 501 in the embodiment calls the computer program stored in the memory 502. The specific execution process can be referred to the description of the method part in the above embodiment, which will not be repeated here.

[0112] Optionally, the computer-readable storage medium may be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0113] In an exemplary embodiment, the present invention further provides a computer program product including one or more instructions, which can be executed by the processor 501 of the electronic device to implement the method in the above embodiment.

[0114] It should be noted that when the instructions in the above-mentioned computer-readable storage medium or one or more instructions in the computer program product are executed by the processor of the electronic device, the various processes of the above-mentioned method embodiment are implemented and the same technical effect as the above-mentioned method can be achieved. To avoid repetition, they will not be repeated here.

[0115] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0116] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0117] Units described as separate components may or may not be physically separate, and components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0118] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0119] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions for causing a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, ROM, RAM, disk or optical disk, etc. Various media that can store program code.

[0120] In the description of the embodiments of the present invention, specific features, structures, materials or characteristics may be combined in an appropriate manner in any one or more embodiments or examples.

[0121] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A sentry mode control method, characterized in that: The method comprises: When the vehicle is in sentry mode, obtaining environmental information around the vehicle; Based on the environmental information around the vehicle, identifying the type of scene the vehicle is currently in; In the case where the vehicle is currently in a high-risk scenario, using a first sentry mode; When the vehicle is currently in a low-risk scenario, a second sentinel mode is used; wherein the monitoring performance of the first sentinel mode is higher than the monitoring performance of the second sentinel mode.

2. The method according to claim 1, characterized in that The first sentinel mode includes: the vehicle's camera uses a first resolution to shoot; the second sentinel mode includes: the vehicle's camera uses a second resolution to shoot, and the first resolution is higher than the second resolution.

3. The method according to claim 1, characterized in that The first sentinel mode includes: the vehicle's radar adopts a first sampling frequency for sampling; the second sentinel mode includes: the vehicle's radar adopts a second sampling frequency for sampling, and the first sampling frequency is higher than the second sampling frequency.

4. The method according to claim 1, wherein The first sentinel mode includes: the ultrasonic sensors of the vehicle are in a fully activated state; the second sentinel mode includes: the non-critical ultrasonic sensors of the vehicle are in a dormant state.

5. The method according to claim 1, wherein The identifying, based on the environmental information surrounding the vehicle, the type of the scene in which the vehicle is currently located, includes: Based on the type of the vehicle and the environmental information, the type of the scene in which the vehicle is currently located is identified; wherein the type of the vehicle includes any one of the following: a household vehicle, a logistics vehicle, a police vehicle, a public transportation vehicle, and an ambulance.

6. The method according to claim 5, characterized in that The identifying, based on the type of the vehicle and the environmental information, the type of the scene in which the vehicle is currently located, includes: In the case where the vehicle is a family vehicle, the environmental information is input into a risk detection model to obtain a risk assessment value of the risk detection model for the current scenario; wherein the environmental information includes pedestrian flow, relative distance between the vehicle and surrounding pedestrians or objects, speed of surrounding pedestrians or objects relative to the vehicle, road conditions, weather conditions, and ambient sound; When the risk assessment value is greater than a threshold, it is determined that the scene type in which the vehicle is currently located is a high-risk scene.

7. The method according to claim 5, characterized in that The identifying, based on the type of the vehicle and the environmental information, the type of the scene in which the vehicle is currently located, includes: In a case where the vehicle is a police vehicle, identifying whether the vehicle is on duty based on the environmental information; When the vehicle is on duty, it is determined that the scene type in which the vehicle is currently located is a high-risk scene.

8. A sentry mode control device, characterized in that: The device comprises: an acquisition unit, configured to acquire environmental information surrounding the vehicle when the vehicle is in sentry mode; an identification unit, configured to identify a type of scene in which the vehicle is currently located based on environmental information surrounding the vehicle; a processing unit configured to use a first sentry mode if the vehicle is currently in a high-risk scenario; The processing unit is further configured to use a second sentinel mode when the vehicle is currently in a low-risk scenario; wherein the monitoring performance of the first sentinel mode is higher than the monitoring performance of the second sentinel mode.

9. An electronic device, characterized in that: The electronic device includes: a processor and a memory; The memory stores instructions executable by the processor; When the processor is configured to execute the instructions, the electronic device implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium includes: computer software instructions; When the computer software instructions are executed in an electronic device, the electronic device is enabled to implement the method according to any one of claims 1 to 7.

Images