Security terminal control method and device and security terminal

Abstract

The invention relates to a security and protection terminal control method and device and a security and protection terminal. The method comprises the following steps: acquiring current multi-mode sensing data of the security and protection terminal; based on a preset weighted fusion algorithm, processing the current multi-modal sensing data to obtain a current sensing scene; querying a preset scene level mapping relationship according to the current sensing scene to obtain a current monitoring level; according to the current monitoring level and the preset scheduling model, the corresponding hardware module and software function module of the security terminal are scheduled to work so as to monitor the current sensing scene, unnecessary power consumption caused by resource surplus is reduced, power consumption redundancy in the absence of monitoring requirements is avoided, and the monitoring efficiency is improved. Meanwhile, the equipment wakeup delay in the transaction scene is eliminated, the scene judgment accuracy is improved, invalid high-power-consumption operation triggered by misjudgment is avoided, and the precise adaptability of low-power-consumption control is enhanced.

Description

Technical Field

[0001] This application relates to the field of security terminal technology, and in particular to a security terminal control method, device and security terminal. Background Technology

[0002] With the development of smart home security technology, embedded security terminals such as home smart cameras and portable security monitoring terminals have integrated functions such as high-definition acquisition, edge AI recognition, and IoT communication. However, in wireless deployment scenarios, users' demand for low power consumption and long battery life of embedded security terminals is becoming increasingly prominent.

[0003] In existing embedded security terminals, most adopt fixed-level switching or fixed-time trigger sleep, resulting in power redundancy when there is no monitoring need and wake-up delay when there is abnormality; static resource scheduling leads to resource surplus when the scene demand is low, which exacerbates the pressure on battery life; and the single scene perception is prone to misjudgment, resulting in low accuracy of power consumption control. Summary of the Invention

[0004] Based on this, in order to solve the problems existing in the above-mentioned embedded security terminals, a security terminal control method, device and security terminal are provided.

[0005] In a first aspect, this application provides a security terminal control method, including:

[0006] Acquire the current multimodal sensing data of the security terminal;

[0007] Based on a preset weighted fusion algorithm, the current multimodal perception data is processed to obtain the current perception scene;

[0008] Based on the current sensing scenario, query the preset scenario level mapping relationship to obtain the current monitoring level;

[0009] Based on the current monitoring level and the preset scheduling model, the corresponding hardware modules and software function modules of the security terminal are scheduled to work in order to monitor the current sensing scene.

[0010] In one embodiment, the preset scene level mapping relationship is established through the following steps:

[0011] Acquire multiple sets of experimental multimodal sensing data from security terminals;

[0012] Based on a pre-defined weighted fusion algorithm, the multimodal sensing data of each group of experiments are processed to obtain multiple experimental sensing scenarios;

[0013] The various experimental sensing scenarios were classified into different levels, resulting in multiple experimental monitoring levels;

[0014] Based on the monitoring levels and perception scenarios of each test, a preset scenario level mapping relationship is established.

[0015] In one embodiment, the current multimodal perception data includes at least visual data and environmental perception data;

[0016] The steps for processing the current multimodal sensing data based on a preset weighted fusion algorithm to obtain the current sensing scene include:

[0017] Human feature detection processing is performed on visual data to obtain visual detection results;

[0018] Environmental sensing data is processed for environmental detection to obtain sensing detection results;

[0019] Based on a preset weighted fusion algorithm, the visual detection results and the perception detection results are weighted to obtain a weighted score, and the current perception scene is obtained based on the weighted score.

[0020] In one embodiment, the hardware module includes at least a processor, an image acquisition and encoding module, and a sensor module; the preset scene level mapping relationship includes at least low monitoring level, medium monitoring level, and high monitoring level;

[0021] Based on the current monitoring level and the preset scheduling model, the steps for scheduling the corresponding hardware modules of the security terminal to operate include:

[0022] Based on the current monitoring level, the number of CPU cores and operating frequency of the scheduled processor are determined, so that the processor can work based on the scheduled number of CPU cores and operating frequency.

[0023] Based on the current monitoring level, the resolution, frame rate, and encoding method of the image acquisition and encoding module are adjusted so that the image acquisition and encoding module can operate based on the adjusted resolution, frame rate, and encoding method.

[0024] Based on the current monitoring level, the number of sensors and the sampling frequency of the sensor module are scheduled, so that the sensor module works based on the scheduled number of sensors and the sampling frequency;

[0025] Specifically, the scheduling intensity of hardware modules corresponding to low monitoring levels is less than that of hardware modules corresponding to medium monitoring levels, and the scheduling intensity of hardware modules corresponding to medium monitoring levels is less than that of hardware modules corresponding to high monitoring levels.

[0026] In one embodiment, the step of scheduling the number of CPU cores and operating frequency of the processor according to the current monitoring level, and causing the processor to operate based on the scheduled number of CPU cores and operating frequency, includes:

[0027] Based on the current monitoring level, the number of CPU cores, operating frequency, and operating voltage of the scheduled processor are adjusted so that the processor can operate based on the adjusted number of CPU cores, operating frequency, and operating voltage.

[0028] In one embodiment, the software functional modules include at least an AI algorithm module, a communication module, and an auxiliary function module;

[0029] Based on the current monitoring level and the preset scheduling model, the steps for scheduling the corresponding software function modules of the security terminal to operate include:

[0030] Based on the current monitoring level, schedule the algorithm model of the AI ​​algorithm module so that the AI ​​algorithm module can work based on the scheduled algorithm model;

[0031] Based on the current monitoring level, the connection time of the communication module is scheduled so that the communication module can work based on the scheduled connection time;

[0032] Based on the current monitoring level, the number of auxiliary function modules to be activated is scheduled, so that the auxiliary function modules can work based on the scheduled number of activations.

[0033] Among them, the scheduling intensity of software function modules corresponding to low monitoring levels is less than that of software function modules corresponding to medium monitoring levels, and the scheduling intensity of software function modules corresponding to medium monitoring levels is less than that of software function modules corresponding to high monitoring levels.

[0034] In one embodiment, the sensor module includes at least an infrared sensor, a sound sensor, and an image sensor;

[0035] Based on the current monitoring level and the preset scheduling model, after scheduling the corresponding hardware and software functional modules of the security terminal to work in order to monitor the current sensing scene, the following steps are included:

[0036] When the current monitoring level is low and the monitoring time of the current sensing scene reaches the preset duration, the control security terminal enters deep sleep mode; deep sleep mode maintains only one of the infrared sensor, sound sensor and image sensor working.

[0037] In one embodiment, after the step of controlling the security terminal to enter deep sleep mode when the current monitoring level is low and the monitoring time of the current sensing scene reaches a preset duration, the following steps are included:

[0038] When the first signal detected by the sensor that is maintaining operation meets the preset wake-up condition, it controls another sensor in the sensor module to work.

[0039] When the second signal detected by another sensor meets the preset verification conditions, the current monitoring level is switched to medium monitoring level or high monitoring level, so as to schedule the corresponding hardware modules and software function modules of the security terminal to work based on the medium monitoring level or high monitoring level;

[0040] If the second signal detected by another sensor does not meet the preset verification conditions, the security terminal is controlled to return to deep sleep mode.

[0041] In one embodiment, after the step of scheduling the corresponding hardware modules and software function modules of the security terminal to work according to the current monitoring level and the preset scheduling model, the following steps are included:

[0042] The system acquires the operational data of the security terminal and transmits the operational data to the cloud server, so that the cloud server updates the weighted fusion weight and scheduling intensity parameters based on the operational data, thus obtaining the updated weighted fusion weight and updated scheduling intensity parameters.

[0043] Obtain the updated weighted fusion weights and updated scheduling intensity parameters, and adjust the preset weighted fusion algorithm and the preset scheduling model according to the updated weighted fusion weights and the updated scheduling intensity parameters.

[0044] Secondly, this application also provides a security terminal control device, comprising:

[0045] The data acquisition unit is used to acquire the current multimodal sensing data of the security terminal;

[0046] The weighted fusion unit is used to process the current multimodal perception data based on a preset weighted fusion algorithm to obtain the current perception scene;

[0047] The monitoring level matching unit is used to query the preset scene level mapping relationship based on the current sensing scene to obtain the current monitoring level;

[0048] The module scheduling unit is used to schedule the corresponding hardware modules and software function modules of the security terminal to work according to the current monitoring level and the preset scheduling model, so as to monitor the current sensing scene.

[0049] Thirdly, this application also provides a security terminal, including a sensor module and a processor; the processor is connected to the sensor module and is used to execute the steps of the security terminal control method described above.

[0050] One of the above technical solutions has the following advantages and beneficial effects:

[0051] In the aforementioned security terminal control method, the current multimodal perception data of the security terminal is acquired; the current multimodal perception data is processed based on a preset weighted fusion algorithm to obtain the current perception scene; based on the current perception scene, a preset scene level mapping relationship is queried to obtain the current monitoring level; and based on the current monitoring level and a preset scheduling model, the corresponding hardware modules and software function modules of the security terminal are scheduled to work to monitor the current perception scene. This application acquires the current perception scene from the current multimodal perception data of the security terminal, determines the current monitoring level based on the current perception scene, and then dynamically adjusts the resource allocation of the corresponding hardware modules and software function modules based on the current monitoring level. This reduces unnecessary power consumption caused by resource excess, avoids power redundancy when there is no monitoring requirement, eliminates device wake-up delay in abnormal scenarios, improves the accuracy of scene judgment, avoids invalid high-power operation triggered by misjudgment, and enhances the accuracy and adaptability of low-power control. Attached Figure Description

[0052] Figure 1 This is a schematic diagram illustrating the application environment of the security terminal control method in the embodiments of this application;

[0053] Figure 2 This is a schematic diagram of the first process of the security terminal control method in the embodiments of this application;

[0054] Figure 3 This is a flowchart illustrating the scene acquisition steps in an embodiment of this application.

[0055] Figure 4 This is a flowchart illustrating the hardware module scheduling steps in an embodiment of this application.

[0056] Figure 5 This is a flowchart illustrating the software function module scheduling steps in an embodiment of this application;

[0057] Figure 6 This is a flowchart illustrating the rapid wake-up steps in an embodiment of this application;

[0058] Figure 7 This is a flowchart illustrating the iterative optimization steps in an embodiment of this application;

[0059] Figure 8 This is a block diagram of the security terminal in the embodiments of this application. Detailed Implementation

[0060] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0061] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0062] In addition, the term "multiple" should mean two or more.

[0063] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0064] The security terminal control method provided in this application can be applied to, for example... Figure 1 The application environment shown is illustrated. The security terminal includes a sensor module 20 and a processor 10. The processor 10 is connected to the sensor module 20, which collects sensing data about the environment in which the security terminal is located. For example, the sensor module 20 can collect image data, sound data, and infrared data. The processor 10 may include a storage unit 104 and a processing unit 102. The processing unit 102 is connected to the storage unit 104, which stores current multimodal sensing data, a preset weighted fusion algorithm, a preset scene level mapping relationship, and a preset scheduling model. The processing unit 102 acquires the current multimodal sensing data of the security terminal collected by the sensor module 20; processes the current multimodal sensing data based on the preset weighted fusion algorithm to obtain the current sensing scene; queries the preset scene level mapping relationship based on the current sensing scene to obtain the current monitoring level; and schedules the corresponding hardware modules and software function modules of the security terminal to work according to the current monitoring level and the preset scheduling model to monitor the current sensing scene.

[0065] In one embodiment, such as Figure 2 As shown, a security terminal control method is also provided, which is applied to... Figure 1 Taking the aforementioned processor as an example, the process includes the following steps:

[0066] Step S210: Obtain the current multimodal perception data of the security terminal.

[0067] The security terminal can be a smart camera or a portable security monitoring terminal, etc. Currently, multimodal perception data is obtained by collecting multi-dimensional scene perception data of the environment in which the security terminal is located. For example, current multimodal perception data may include visual data, human perception data, and sound data of the environment in which the security terminal is located. In another example, current multimodal perception data may also include status data of the security terminal, such as the current system time, remaining battery power, and network connection status. Based on the status data of the security terminal, the user's daily routine and the working status of the security terminal can be matched, thereby improving the accuracy of scene judgment in subsequent steps.

[0068] For example, the sensor module can collect multi-dimensional scene perception data of the environment in which the security terminal is located in real time based on a preset period, obtain the current multimodal perception data, and transmit the current multimodal perception data to the processor, so that the processor can obtain the current multimodal perception data of the security terminal.

[0069] Step S220: Based on a preset weighted fusion algorithm, process the current multimodal perception data to obtain the current perception scene.

[0070] The preset weighted fusion algorithm can be obtained based on the system preset. For example, the detection results of the perceived data in the weighted fusion algorithm can be preset with weights to obtain the preset weighted fusion algorithm.

[0071] The current multimodal sensing data is processed by a preset weighted fusion algorithm to obtain the current sensing scene of the security terminal. It should be noted that the sensing scene can be divided according to the actual application scenario; for example, the sensing scene can be a densely populated activity area, a temporary visitor scenario, an empty and quiet room scenario, or a nighttime hibernation scenario, etc.

[0072] Step S230: Based on the current sensing scene, query the preset scene level mapping relationship to obtain the current monitoring level.

[0073] The preset scenario level mapping relationship can be established based on historical experimental data and can be presented in tabular or database format. The preset scenario level mapping relationship includes the correspondence between sensing scenarios and monitoring levels. For example, monitoring levels can be divided into low monitoring level, medium monitoring level, and high monitoring level; the scenarios corresponding to the low monitoring level are empty, quiet rooms and nighttime dormant scenarios, the scenarios corresponding to the medium monitoring level are temporary visitor scenarios, and the scenarios corresponding to the high monitoring level are densely populated activity areas.

[0074] Step S240: Based on the current monitoring level and the preset scheduling model, schedule the corresponding hardware modules and software function modules of the security terminal to work in order to monitor the current sensing scene.

[0075] The preset scheduling model can be obtained based on system presets. Different monitoring levels correspond to different scheduling intensities for hardware and software functional modules. For example, the higher the monitoring level, the greater the scheduling intensity of hardware and software functional modules.

[0076] For example, the current monitoring level is input into a preset scheduling model for processing to obtain the scheduling results of the corresponding hardware modules and software functional modules of the anti-terminal. Then, the corresponding hardware modules and software functional modules work according to the control scheduling to monitor the current sensing scene. This realizes the dynamic adjustment of the computing power and working mode of the corresponding hardware modules and the dynamic adjustment of the operating status of the corresponding software functional modules according to the current monitoring level, which significantly reduces the overall power consumption of the equipment.

[0077] In the above embodiments, the current multimodal perception data of the security terminal is acquired; the current multimodal perception data is processed based on a preset weighted fusion algorithm to obtain the current perception scene; based on the current perception scene, a preset scene level mapping relationship is queried to obtain the current monitoring level; and based on the current monitoring level and a preset scheduling model, the corresponding hardware modules and software function modules of the security terminal are scheduled to work to monitor the current perception scene. This application acquires the current perception scene through the current multimodal perception data of the security terminal, determines the current monitoring level based on the current perception scene, and then dynamically adjusts the resource allocation of the corresponding hardware modules and software function modules based on the current monitoring level. This reduces unnecessary power consumption caused by resource excess, avoids power redundancy when there is no monitoring demand, eliminates device wake-up delay in abnormal scenarios, improves the accuracy of scene judgment, avoids invalid high-power operation triggered by misjudgment, and enhances the accurate adaptability of low-power control.

[0078] In one embodiment, the preset scene level mapping relationship is established through the following steps:

[0079] Acquire multiple sets of multimodal sensing data from security terminals; process each set of multimodal sensing data based on a preset weighted fusion algorithm to obtain multiple test sensing scenarios; classify each test sensing scenario into multiple test monitoring levels; and establish a preset scenario level mapping relationship based on each test monitoring level and each test sensing scenario.

[0080] The various test sensing scenarios may include scenarios of dense crowds, temporary visitors, empty rooms, and nighttime rest; the test monitoring levels can be divided into three levels: high, medium, and low.

[0081] For example, high monitoring levels are matched with scenarios of dense crowds, medium monitoring levels with scenarios of temporary visitors, and low monitoring levels with scenarios of empty rooms and nighttime sleep. This establishes a preset scenario level mapping relationship. When the current sensing scenario is acquired, the current monitoring level can be quickly determined by querying the preset scenario level mapping relationship. Based on the current monitoring level, the resource allocation of the corresponding hardware and software functional modules can be dynamically adjusted, enabling on-demand scheduling of security terminal hardware and software resources. This improves the accuracy of scenario recognition, reduces unnecessary power consumption, and enhances the precise adaptability of low-power control.

[0082] In one embodiment, the current multimodal perception data includes at least visual data and environmental perception data.

[0083] Visual data can be obtained by acquiring real-time frame data of the environment in which the security terminal is located through an image sensor. In some examples, when the security terminal initially acquires data, the image sensor can be controlled to acquire visual data in a low-resolution preview mode to reduce the basic power consumption of the security terminal. Environmental perception data may include human feature data and sound intensity data of the environment in which the security terminal is located; for example, human pyroelectric signals can be acquired through an infrared sensor, and sound intensity signals can be acquired through a sound sensor, and then environmental perception data can be obtained based on the human pyroelectric signals and sound intensity signals.

[0084] In one example, such as Figure 3 As shown, the steps for processing the current multimodal sensing data based on a preset weighted fusion algorithm to obtain the current sensing scene include:

[0085] Step S310: Perform human feature detection processing on the visual data to obtain visual detection results;

[0086] For example, human features can be extracted from visual data, and the extracted human feature data can be compared with preset human feature data to obtain visual detection results, thereby enabling the determination of whether there are human bodies in the environment where the security terminal is located.

[0087] Step S320: Perform environmental detection processing on the environmental perception data to obtain the perception detection results.

[0088] For example, environmental sensing data is compared with preset environmental thresholds to obtain sensing detection results, thereby further determining whether there is human activity in the environment where the security terminal is located. Exemplarily, the environmental detection processing of environmental sensing data includes: determining the presence or absence of a living person by analyzing pyroelectric signals from a human body, and classifying sound intensity signals as abnormal sounds (such as cracking sounds) and ambient sounds. In one example, the environmental sensing data can also be radar data detected by a millimeter-wave radar sensor. By processing the radar data, the presence or absence of a living person can be determined by judging subtle movements and distance.

[0089] Step S330: Based on the preset weighted fusion algorithm, the visual detection results and the perception detection results are weighted to obtain a weighted score, and the current perception scene is obtained based on the weighted score.

[0090] Different perception scenarios correspond to different weighted scores. For example, scenarios with more frequent human activity receive higher weighted scores; similarly, scenarios with less human activity receive lower weighted scores.

[0091] For example, the visual detection result has a weight of 60%, and the perception detection result has a weight of 40%. By inputting the visual and perception detection results into a preset weighted fusion algorithm for weighted processing, a weighted score is obtained. The current perception scene can then be determined by looking up the weighted score in a table, achieving accurate scene recognition and significantly reducing misjudgments associated with existing single-image perception methods. For instance, common household scenes (empty rooms, people moving, pets moving, curtains slightly moving, etc.) can be selected, and compared with existing solutions relying solely on image perception, the false trigger rate of this application's multimodal scene recognition is reduced from 28% in existing solutions to 7%, avoiding high-power continuous operation due to misjudgments, and reducing the proportion of invalid power consumption within a single scene cycle (e.g., 24 hours) by 21%.

[0092] In one embodiment, the hardware module includes at least a processor, an image acquisition and encoding module, and a sensor module; the preset scene level mapping relationship includes at least low monitoring level, medium monitoring level, and high monitoring level.

[0093] The processor includes at least two CPU cores; the image acquisition and encoding module is used to acquire image signals detected in real time by the image sensor and to encode the acquired data; the sensor module is used to detect the environment in which the security terminal is located in real time to obtain the corresponding sensing signals.

[0094] The scheduling intensity of hardware modules corresponding to low monitoring levels is less than that corresponding to medium monitoring levels, and the scheduling intensity of hardware modules corresponding to medium monitoring levels is less than that corresponding to high monitoring levels. Similarly, the scheduling intensity of software function modules corresponding to low monitoring levels is less than that corresponding to medium monitoring levels, and the scheduling intensity of software function modules corresponding to medium monitoring levels is less than that corresponding to high monitoring levels.

[0095] In one example, such as Figure 4 As shown, the steps for scheduling the corresponding hardware modules of the security terminal to work according to the current monitoring level and the preset scheduling model include:

[0096] Step S410: Based on the current monitoring level, schedule the number of CPU cores and operating frequency of the processor, so that the processor works based on the scheduled number of CPU cores and operating frequency.

[0097] For example, when the current monitoring level is high, two or more CPU cores of the processor are scheduled and maintained at 100% operating frequency, so that the processor operates based on the scheduled number of CPU cores and operating frequency. When the current monitoring level is medium, one CPU core of the processor is scheduled and maintained at 75% operating frequency, so that the processor operates based on the scheduled number of CPU cores and operating frequency. When the current monitoring level is low, one CPU core of the processor is scheduled and maintained at 50% operating frequency, so that the processor operates based on the scheduled number of CPU cores and operating frequency. This allows the computing power and operating mode of the security terminal's processor to be dynamically adjusted according to the scenario's requirements, thereby reducing unnecessary power consumption caused by excess resources.

[0098] Step S420: Based on the current monitoring level, schedule the resolution, frame rate, and encoding method of the image acquisition and encoding module so that the image acquisition and encoding module works based on the scheduled resolution, frame rate, and encoding method.

[0099] For example, when the current monitoring level is high, the image acquisition and encoding module is scheduled to have a high-definition resolution (e.g., 1080P), a frame rate of 25fps, and H.265 full encoding, so that the image acquisition and encoding module operates based on the scheduled resolution, frame rate, and encoding method. When the current monitoring level is medium, the image acquisition and encoding module is scheduled to have a standard-definition resolution (e.g., 720P), a frame rate of 15fps, and H.265 lightweight encoding, so that the image acquisition and encoding module operates based on the scheduled resolution, frame rate, and encoding method. When the current monitoring level is low, the system switches to interval acquisition mode (i.e., acquiring one low-resolution frame every 5 seconds) and disables continuous encoding, so that the image acquisition and encoding module operates based on the scheduled resolution, frame rate, and encoding method. This allows the computing power and operating mode of the security terminal's image acquisition and encoding module to be dynamically adjusted according to the scene's requirements, reducing unnecessary power consumption caused by excess resources.

[0100] Step S430: Based on the current monitoring level, schedule the number of sensors and the acquisition frequency of the sensor module so that the sensor module can work based on the scheduled number of sensors and acquisition frequency.

[0101] For example, at a high monitoring level, all sensors (image sensor, infrared sensor, and sound sensor) are continuously powered on, allowing the sensor modules to operate based on the scheduled number of sensors and sampling frequency. At a medium monitoring level, the image sensor is continuously powered on, while the infrared and sound sensors operate intermittently (e.g., detecting once every 2 seconds), allowing the sensor modules to operate based on the scheduled number of sensors and sampling frequency. At a low monitoring level, only the infrared sensor is powered on in low-power mode, while the image and sound sensors are turned off, allowing the sensor modules to operate based on the scheduled number of sensors and sampling frequency. This allows the number and operating mode of the security terminal's sensor modules to be dynamically adjusted according to the scenario's requirements, reducing unnecessary power consumption caused by excess resources.

[0102] For example, an experiment was conducted using a wireless battery-powered home camera as the security terminal. Compared with the existing static resource configuration scheme that operates with full computing power, the daily power consumption of the terminal in this application is reduced from 1200mAh to 450mAh based on the working mode of the processor, image acquisition and encoding module and sensor module according to the scenario requirements. The battery life is extended from 3 days to 8 days. At the same time, the terminal operating temperature is reduced from 42℃ to 33℃, the hardware lifespan is expected to be extended by 15%, and the overall power consumption of the terminal is significantly reduced.

[0103] In one embodiment, the step of scheduling the number of CPU cores and operating frequency of the processor according to the current monitoring level, and causing the processor to operate based on the scheduled number of CPU cores and operating frequency, includes:

[0104] Based on the current monitoring level, the number of CPU cores, operating frequency, and operating voltage of the scheduled processor are adjusted so that the processor can operate based on the adjusted number of CPU cores, operating frequency, and operating voltage.

[0105] For example, when the current monitoring level is high, all CPU cores are scheduled to run at the first voltage and first frequency, so that the processor operates based on the scheduled number of CPU cores and operating frequency. When the current monitoring level is medium, only one core is kept running at the second voltage and second frequency, while the remaining cores enter sleep mode, so that the processor operates based on the scheduled number of CPU cores and operating frequency. When the current monitoring level is low, only one core is kept running at the second voltage and second frequency, so that the processor operates based on the scheduled number of CPU cores and operating frequency. This allows for dynamic adjustment of the security terminal's processor's computing power and operating mode according to the scenario's requirements, reducing unnecessary power consumption caused by excess resources. It should be noted that the first voltage is greater than the second voltage, and the second voltage is greater than the third voltage; the first frequency is greater than the second frequency, and the second frequency is greater than the third frequency.

[0106] In one embodiment, the software functional modules include at least an AI algorithm module, a communication module, and an auxiliary function module.

[0107] The AI ​​algorithm module may include a human behavior recognition model, an abnormal event detection model, and a lightweight human presence detection model. The communication module may include WiFi, 4G, and 5G modules. The auxiliary function module may include non-core modules such as a voice interaction module and a local storage redundant log module. It should be noted that the human behavior recognition model is used to identify human behavior; the abnormal event detection model is used to determine the movement of non-human objects; and the lightweight human presence detection model is used to detect the presence of a human body.

[0108] In one example, such as Figure 5 As shown, the steps for scheduling the corresponding software function modules of the security terminal to work according to the current monitoring level and the preset scheduling model include:

[0109] Step S510: Based on the current monitoring level, schedule the algorithm model of the AI ​​algorithm module so that the AI ​​algorithm module can work based on the scheduled algorithm model.

[0110] For example, when the current monitoring level is high, a complete human behavior recognition model and anomaly detection model are loaded, allowing the AI ​​algorithm module to operate based on the scheduled algorithm model. When the current monitoring level is medium, a lightweight human presence detection model is scheduled and loaded, allowing the AI ​​algorithm module to operate based on the scheduled algorithm model. When the current monitoring level is low, the AI ​​model is disabled, retaining only the basic moving object triggering logic, allowing the AI ​​algorithm module to operate based on the scheduled algorithm model. This allows for dynamic adjustment of the number and operating status of AI algorithms enabled on the security terminal according to the scenario's requirements, reducing unnecessary power consumption caused by excess resources.

[0111] Step S520: Based on the current monitoring level, schedule the connection time of the communication module so that the communication module works based on the scheduled connection time.

[0112] For example, when the current monitoring level is high, the WiFi, 4G, and 5G modules maintain continuous connections and transmit video data in real time, allowing the communication modules to operate based on the scheduled connection times. When the current monitoring level is medium, the WiFi, 4G, and 5G modules are scheduled to connect intermittently (uploading video every 10 seconds), allowing the communication modules to operate based on the scheduled connection times. When the current monitoring level is low, network connections are disabled, and communication is only restored when an anomaly is triggered, allowing the communication modules to operate based on the scheduled connection times. This allows for dynamic adjustment of the number of communication modules activated and their operating status in the security terminal according to the scenario's requirements, reducing unnecessary power consumption caused by excess resources.

[0113] Step S530: Based on the current monitoring level, schedule the number of auxiliary function modules to be activated, so that the auxiliary function modules can work based on the scheduled number of activations.

[0114] For example, when the current monitoring level is high, all auxiliary function modules (such as the voice interaction module and the local storage redundant log module) are enabled, allowing them to operate based on the scheduled number of enabled modules. When the current monitoring level is medium, half of the auxiliary function modules are enabled, again allowing them to operate based on the scheduled number of enabled modules. When the current monitoring level is low, all auxiliary function modules are disabled. This allows for dynamic adjustment of the number and operating status of auxiliary function modules on the security terminal based on the scenario's requirements, reducing unnecessary power consumption caused by excess resources.

[0115] In one embodiment, the sensor module includes at least an infrared sensor, a sound sensor, and an image sensor. The infrared sensor detects pyroelectric signals from the human body in the environment where the security terminal is located; the sound sensor detects sound intensity signals in the environment where the security terminal is located; and the image sensor detects image signals in the environment where the security terminal is located.

[0116] In one example, after the step of scheduling the corresponding hardware and software functional modules of the security terminal to work according to the current monitoring level and the preset scheduling model to monitor the current sensing scene, the following steps are included:

[0117] When the current monitoring level is low and the monitoring time of the current sensing scene reaches the preset duration, the control security terminal enters deep sleep mode; deep sleep mode maintains only one of the infrared sensor, sound sensor and image sensor working.

[0118] The preset duration can be obtained from system presets; for example, the preset duration can be set to 10 minutes.

[0119] For example, in low-level monitoring scenarios, if the monitoring time of the current sensing scene reaches a preset duration, the security terminal is controlled to enter a deep sleep mode, so that the security terminal maintains only one of the infrared sensor, sound sensor, and image sensor in operation. For example, when the security terminal enters deep sleep mode, only the infrared sensor is kept in low-power mode as the wake-up trigger node, and all hardware modules except the core control unit are turned off; or, when the security terminal enters deep sleep mode, only the sound sensor is kept in low-power mode as the wake-up trigger node, and all hardware modules except the core control unit are turned off, thus achieving low-power sleep mode for the security terminal.

[0120] In one embodiment, such as Figure 6 As shown, after the step of controlling the security terminal to enter deep sleep mode when the current monitoring level is low and the monitoring time of the current sensing scene reaches the preset duration, the following steps are included:

[0121] Step S610: When the first signal detected by the sensor that is maintaining operation meets the preset wake-up condition, control another sensor in the sensor module to work.

[0122] For example, when the sensor maintaining operation is an infrared sensor, the preset wake-up condition could be determining whether the first signal is a micro-motion signal from a living being. A micro-motion signal from a living being is used to indicate that there is movement of an object in the environment where the security terminal is located. When the sensor maintaining operation is a sound sensor, the preset wake-up condition could be determining whether the first signal exceeds a preset threshold (e.g., 60 decibels).

[0123] After the security terminal enters deep sleep mode, the sensors that remain operational can detect the first signal of the environment in which the security terminal is located based on a preset period. The detected first signal is compared with preset wake-up conditions. When the first signal meets the preset wake-up conditions, another sensor in the sensor module is activated to trigger a fast wake-up process. It should be noted that the other sensor in the sensor module refers to any one of the remaining sensors excluding the one that remains operational.

[0124] For example, when the first signal detected by the sensor that is maintaining operation meets the preset wake-up condition, the image sensor is controlled to start working in low-resolution preview mode.

[0125] Step S620: When the second signal detected by another sensor meets the preset verification conditions, the current monitoring level is switched to medium monitoring level or high monitoring level, so as to schedule the corresponding hardware modules and software function modules of the security terminal to work based on the medium monitoring level or high monitoring level.

[0126] For example, if the other sensor is an image sensor, the second signal is an image signal. The preset verification condition could be to determine whether the image signal includes motion features. It should be noted that if the image signal includes motion features, it is determined that a human has entered the environment where the security terminal is located.

[0127] After another sensor (such as an image sensor) starts working, the other sensor detects a second signal of the environment in which the security terminal is located based on a preset period, and compares the second signal with preset verification conditions. When the second signal detected by the other sensor meets the preset verification conditions, the current monitoring level is quickly switched (e.g., within 1 second) to a medium monitoring level or a high monitoring level, so as to schedule the corresponding hardware modules and software function modules of the security terminal to work based on the medium monitoring level or the high monitoring level.

[0128] Step S630: When the second signal detected by another sensor does not meet the preset verification conditions, control the security terminal to return to deep sleep mode.

[0129] If the second signal detected by another sensor does not meet the preset verification conditions, the verification is determined to be a false trigger (such as a pet moving or curtains swaying), and the system returns to deep sleep mode. This achieves linkage control between low-power sleep and fast wake-up, improving the reliability of security monitoring.

[0130] For example, the low-power sleep and fast wake-up linkage mechanism adopted in this application simulates a scenario where a person suddenly enters an empty house at night. Compared with the existing fixed-time trigger wake-up scheme, the low-power sleep and fast wake-up linkage mechanism adopted in this application reduces the response time of the terminal from deep sleep mode to switching to medium monitoring level (or high monitoring level) configuration from 3.2 seconds to 0.8 seconds. At the same time, the power consumption in deep sleep mode is only 20μA (the existing solution has a sleep power consumption of 150μA), and the power consumption during the sleep stage is reduced by 87%. This solves the response delay problem of the existing fixed sleep mode, enhances the accurate adaptability of low-power control, and balances the low power consumption and security reliability of the security terminal.

[0131] In one embodiment, such as Figure 7 As shown, after scheduling the corresponding hardware and software functional modules of the security terminal to work according to the current monitoring level and the preset scheduling model, the steps include:

[0132] Step S710: Obtain the operation data of the security terminal and transmit the operation data to the cloud server so that the cloud server can update the weighted fusion weight and scheduling intensity parameters based on the operation data, and obtain the updated weighted fusion weight and updated scheduling intensity parameters.

[0133] The operational data includes power consumption, wake-up response time, and false trigger rate. Scheduling parameters may include sleep trigger threshold and hardware frequency reduction.

[0134] For example, the processor can periodically collect operational data from security terminals and transmit this data synchronously to a cloud server. The cloud server then updates the weighted fusion weights and scheduling intensity parameters based on this data, resulting in updated weighted fusion weights and scheduling intensity parameters. For instance, by updating the weighted fusion weights and scheduling intensity parameters, the logic for distinguishing between nighttime pet activity and human activity can be optimized, thereby improving the accuracy of scene recognition.

[0135] Step S720: Obtain the updated weighted fusion weight and the updated scheduling intensity parameter, and adjust the preset weighted fusion algorithm according to the updated weighted fusion weight, and adjust the preset scheduling model according to the updated scheduling intensity parameter.

[0136] The cloud server can periodically feed back updated weighted fusion weights and updated scheduling intensity parameters to the processor, allowing the processor to obtain these updated parameters. Based on the updated weighted fusion weights, the processor adjusts the preset weighted fusion algorithm, optimizing it. The processor also adjusts the preset scheduling model based on the updated scheduling intensity parameters, optimizing it as well. This continuously improves the accuracy and adaptability of low-power control for security terminals, adapting to diverse application scenarios.

[0137] For example, two weeks of operation data from 100 security terminals were collected, and the weighted fusion weight and scheduling intensity parameters were iteratively updated to conduct experiments on the model optimization of this application. According to the experimental results, the power consumption adaptation accuracy under different home scenarios (such as living alone, multiple people, and having pets) was improved from 72% to 91%, further enhancing the accuracy and adaptability of low power consumption control.

[0138] It should be understood that, although Figures 2 to 7 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figures 2 to 7 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but may be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0139] In one embodiment, such as Figure 8 As shown in the figure, this application embodiment also provides a security terminal control device, including:

[0140] The data acquisition unit 810 is used to acquire the current multimodal sensing data of the security terminal.

[0141] The weighted fusion unit 820 is used to process the current multimodal perception data based on a preset weighted fusion algorithm to obtain the current perception scene.

[0142] The monitoring level matching unit 830 is used to query the preset scene level mapping relationship based on the current sensing scene to obtain the current monitoring level.

[0143] The module scheduling unit 840 is used to schedule the corresponding hardware modules and software function modules of the security terminal to work according to the current monitoring level and the preset scheduling model, so as to monitor the current sensing scene.

[0144] Specific limitations regarding the security terminal control device can be found in the limitations of the security terminal control method described above, and will not be repeated here. Each module in the aforementioned security terminal control device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the security terminal in hardware form or independent of it, or stored in the memory of the security terminal in software form, so that the processor can call and execute the corresponding operations of each module.

[0145] In one embodiment, such as Figure 1 As shown, this application embodiment also provides a security terminal, including a sensor module 20 and a processor 10; the processor 10 is connected to the sensor module 20, and the processor 10 is used to execute the steps of the security terminal control method of any of the above.

[0146] For a detailed description of the sensor module 20 and the processor 10, please refer to the description of the above embodiments, which will not be repeated here.

[0147] The processor 10 is connected to the sensor module 20. The sensor module 20 collects sensing data of the environment in which the security terminal is located, obtains current multimodal sensing data, and transmits the current multimodal sensing data to the processor 10. The processor 10 obtains the current multimodal sensing data of the security terminal; processes the current multimodal sensing data based on a preset weighted fusion algorithm to obtain the current sensing scene; queries a preset scene level mapping relationship based on the current sensing scene to obtain the current monitoring level; and schedules the corresponding hardware modules and software function modules of the security terminal to work according to the current monitoring level and a preset scheduling model to monitor the current sensing scene. This application obtains the current sensing scene through the current multimodal sensing data of the security terminal, determines the current monitoring level based on the current sensing scene, and then dynamically adjusts the resource allocation of the corresponding hardware modules and software function modules according to the current monitoring level. This reduces unnecessary power consumption caused by resource excess, avoids power redundancy when there is no monitoring demand, eliminates device wake-up delay in abnormal scenarios, improves the accuracy of scene judgment, avoids invalid high-power operation triggered by misjudgment, and enhances the accuracy and adaptability of low-power control.

[0148] In one embodiment, a computer storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of any of the above-described security terminal control methods.

[0149] For example, when a computer program is executed by a processor, it performs the following steps:

[0150] Acquire the current multimodal perception data of the security terminal; process the current multimodal perception data based on a preset weighted fusion algorithm to obtain the current perception scene; query the preset scene level mapping relationship according to the current perception scene to obtain the current monitoring level; and schedule the corresponding hardware modules and software function modules of the security terminal to work according to the current monitoring level and the preset scheduling model in order to monitor the current perception scene.

[0151] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the division operations described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), direct memory bus RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0152] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0153] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A security terminal control method, characterized in that, include: Acquire the current multimodal sensing data of the security terminal; Based on a preset weighted fusion algorithm, the current multimodal sensing data is processed to obtain the current sensing scene; Based on the current sensing scenario, query the preset scenario level mapping relationship to obtain the current monitoring level; Based on the current monitoring level and the preset scheduling model, the corresponding hardware modules and software function modules of the security terminal are scheduled to work in order to monitor the current sensing scene.

2. The security terminal control method according to claim 1, characterized in that, The preset scene level mapping relationship is established through the following steps: Acquire multiple sets of experimental multimodal sensing data from security terminals; Based on a preset weighted fusion algorithm, the multimodal perception data of each group of experiments are processed to obtain multiple experimental perception scenarios; The various experimental sensing scenarios are classified into different levels to obtain multiple experimental monitoring levels; Based on each of the aforementioned test monitoring levels and each of the aforementioned test perception scenarios, a mapping relationship for the preset scenario levels is established.

3. The security terminal control method according to claim 1, characterized in that, The current multimodal perception data includes at least visual data and environmental perception data; The step of processing the current multimodal perception data based on a preset weighted fusion algorithm to obtain the current perception scene includes: The visual data is processed for human feature detection to obtain visual detection results; The environmental sensing data is processed by environmental detection to obtain the sensing detection results; Based on a preset weighted fusion algorithm, the visual detection results and the perception detection results are weighted to obtain a weighted score, and the current perception scene is obtained based on the weighted score.

4. The security terminal control method according to claim 1, characterized in that, The hardware module includes at least a processor, an image acquisition and encoding module, and a sensor module; the preset scene level mapping relationship includes at least low monitoring level, medium monitoring level, and high monitoring level. The step of scheduling the corresponding hardware modules of the security terminal to work according to the current monitoring level and the preset scheduling model includes: Based on the current monitoring level, the number of CPU cores and operating frequency of the processor are scheduled so that the processor can work based on the scheduled number of CPU cores and operating frequency. Based on the current monitoring level, the resolution, frame rate, and encoding method of the image acquisition and encoding module are adjusted so that the image acquisition and encoding module operates based on the adjusted resolution, frame rate, and encoding method. Based on the current monitoring level, the number of sensors and the acquisition frequency of the sensor module are scheduled, so that the sensor module works based on the scheduled number of sensors and acquisition frequency; Specifically, the scheduling intensity of hardware modules corresponding to low monitoring levels is less than that of hardware modules corresponding to medium monitoring levels, and the scheduling intensity of hardware modules corresponding to medium monitoring levels is less than that of hardware modules corresponding to high monitoring levels.

5. The security terminal control method according to claim 4, characterized in that, The step of scheduling the number of CPU cores and operating frequency of the processor according to the current monitoring level, so that the processor operates based on the scheduled number of CPU cores and operating frequency, includes: Based on the current monitoring level, the number of CPU cores, operating frequency, and operating voltage of the processor are adjusted so that the processor operates based on the adjusted number of CPU cores, operating frequency, and operating voltage.

6. The security terminal control method according to claim 1, characterized in that, The software functional modules include at least an AI algorithm module, a communication module, and an auxiliary function module; The step of scheduling the corresponding software function modules of the security terminal to work according to the current monitoring level and the preset scheduling model includes: Based on the current monitoring level, the algorithm model of the AI ​​algorithm module is scheduled so that the AI ​​algorithm module works based on the scheduled algorithm model; Based on the current monitoring level, the connection time of the communication module is scheduled so that the communication module operates based on the scheduled connection time; Based on the current monitoring level, the number of auxiliary function modules that are activated is scheduled, so that the auxiliary function modules work based on the scheduled number of activations. Among them, the scheduling intensity of software function modules corresponding to low monitoring levels is less than that of software function modules corresponding to medium monitoring levels, and the scheduling intensity of software function modules corresponding to medium monitoring levels is less than that of software function modules corresponding to high monitoring levels.

7. The security terminal control method according to claim 4, characterized in that, The sensor module includes at least an infrared sensor, a sound sensor, and an image sensor; After the step of scheduling the corresponding hardware modules and software function modules of the security terminal to work according to the current monitoring level and the preset scheduling model to monitor the current sensing scene, the following steps are included: When the current monitoring level is low and the monitoring time of the current sensing scene reaches a preset duration, the security terminal is controlled to enter a deep sleep mode; the deep sleep mode maintains only one of the infrared sensor, the sound sensor and the image sensor working.

8. The security terminal control method according to claim 7, characterized in that, After the step of controlling the security terminal to enter deep sleep mode when the current monitoring level is low and the monitoring time of the current sensing scene reaches a preset duration, the following steps are included: When the first signal detected by the sensor that is maintaining operation meets the preset wake-up condition, another sensor in the sensor module is controlled to work. When the second signal detected by the other sensor meets the preset verification conditions, the current monitoring level is switched to medium monitoring level or high monitoring level, so as to schedule the corresponding hardware modules and software function modules of the security terminal to work based on the medium monitoring level or high monitoring level; When the second signal detected by the other sensor does not meet the preset verification conditions, the security terminal is controlled to return to deep sleep mode.

9. The security terminal control method according to any one of claims 1 to 8, characterized in that, After the step of scheduling the corresponding hardware modules and software function modules of the security terminal to work according to the current monitoring level and the preset scheduling model, the following steps are included: The system acquires the operational data of the security terminal and transmits the operational data to the cloud server, so that the cloud server updates the weighted fusion weight and scheduling intensity parameters based on the operational data, thereby obtaining the updated weighted fusion weight and updated scheduling intensity parameters. Obtain the updated weighted fusion weights and updated scheduling intensity parameters, and adjust the preset weighted fusion algorithm and the preset scheduling model according to the updated weighted fusion weights and the updated scheduling intensity parameters.

10. A security terminal control device, characterized in that, include: The data acquisition unit is used to acquire the current multimodal sensing data of the security terminal; The weighted fusion unit is used to process the current multimodal perception data based on a preset weighted fusion algorithm to obtain the current perception scene; The monitoring level matching unit is used to query a preset scene level mapping relationship based on the current sensing scene to obtain the current monitoring level; The module scheduling unit is used to schedule the corresponding hardware modules and software function modules of the security terminal to work according to the current monitoring level and the preset scheduling model, so as to monitor the current sensing scene.

11. A security terminal, characterized in that, It includes a sensor module and a processor; the processor is connected to the sensor module and is used to execute the steps of the security terminal control method according to any one of claims 1 to 9.

Images