Multi-modal computing power optimization method and system for security monitoring scene

Abstract

The application discloses a multi-modal computing power optimization method and system for security monitoring scenes, and relates to the technical field of multi-modal computing power data processing. The multi-modal computing power optimization method for security monitoring scenes comprises the following steps: computing power allocation lag judgment, cooperative scheduling lag monitoring, and scheduling lag level division. The application performs screening processing on security multi-modal data, analyzes the lag degree of computing power allocation after the screening processing is qualified, judges whether to start the computing power allocation lag optimization mechanism, and if not, performs cooperative scheduling lag degree analysis, otherwise, performs cooperative scheduling lag degree analysis after the start is completed. Based on the execution result of the cooperative scheduling lag degree analysis, it is judged whether to start the modal scheduling lag level division mechanism, which improves the real-time and accuracy of the computing power optimization data under security monitoring, and solves the problem of low real-time and accuracy of the computing power optimization data under security monitoring in the prior art.

Description

Multimodal computing power optimization method and system for security monitoring scenarios Technical Field

[0001] This invention relates to the field of multimodal computing power data processing technology, and in particular to a multimodal computing power optimization method and system for security monitoring scenarios. Background Technology

[0002] In typical monitoring scenarios such as park perimeter walls, warehouses, school perimeters, entrances and exits of commercial complexes, and residential community security, multiple modal data, including video images, audio, infrared, and vibration sensor data, are continuously generated. To achieve efficient utilization of computing resources in security monitoring scenarios, existing methods first perform layered processing in a "cloud-edge-device" collaborative architecture based on the different computing power requirements of different modal data such as video, audio, and sensor data: front-end devices perform modal filtering and lightweight preprocessing (such as video motion detection and audio abnormal sound detection) to filter invalid data and reduce the burden on the back-end; edge nodes then undertake the core real-time multimodal analysis. Through dynamic computing power scheduling strategies, more resources are allocated to video streams for high-frequency target detection and tracking, while deeper analysis, such as speech recognition or voiceprint analysis, is only initiated for audio streams when anomalies are detected in video analysis or specific rules are triggered. This achieves on-demand and elastic allocation of computing power. Meanwhile, the cloud center is responsible for centralized optimization of edge models and in-depth analysis of non-real-time data. It also utilizes data from various edge nodes to continuously update and lightweight models through federated learning, thus forming a closed-loop optimization process from accurate perception and collaborative computing to continuous evolution. Ultimately, this comprehensively improves the system's real-time response and comprehensive analysis accuracy under limited computing power.

[0003] For example, Chinese invention patent application CN119440830A discloses a high-computing-power, multi-service artificial intelligence system and method for the security industry. This includes: receiving configuration and task information from a security business platform through a platform interaction layer and saving this information to a database; a task scheduling layer judging based on the task information and scheduling to the corresponding business service; and a business service operating at the business implementation layer, executing tasks according to the task information, merging intelligent result data, storing alarm images or audio / video recordings, and returning the execution results to the platform interaction layer. The bitstreams that the business service needs to analyze are pulled through the data access layer and sent to the data processing service for decoding and algorithm analysis.

[0004] The above-mentioned technology has at least the following technical problems:

[0005] In security monitoring, various modalities of data are generated, such as video data, audio data, and sensor data. During the optimization of computing power based on this multimodal data, sudden warnings from sensor data (such as infrared and vibration) may cause delays in modal collaborative scheduling. Specifically, under current strategies, in-depth analysis of audio data typically relies on the triggering of abnormal video events. If video detection experiences missed detections or delays due to environmental interference (such as increased image noise in low-light environments) or complex background interference (such as changes in light and shadow, movement of irrelevant objects causing blurring of the boundary between the monitored target and the background, obscuring target features, or triggering a large number of false detection signals consuming computing power), it may directly lead to the associated deep... Audio analysis may fail to start or start with a delay, resulting in the loss of critical audio information and reducing the accuracy of anomaly identification. Existing edge nodes typically employ a fixed strategy that prioritizes allocating resources to video, neglecting the need for sudden early warnings from sensor data (such as infrared and vibration). This leads to a situation where, in emergency scenarios, computing resources cannot be promptly released from the video stream and reallocated to the most critical modalities. Consequently, sensor warning signals and associated sudden audio data cannot be analyzed in a timely manner due to delayed computing resource allocation or triggering conditions. Ultimately, this results in inaccurate allocation of computing power during critical periods and insufficient depth of multimodal information fusion, leading to low real-time performance and accuracy of computing power optimization data in security monitoring. Summary of the Invention

[0006] To address the technical problems of low real-time performance and accuracy of computing power optimization data in security monitoring applications, this invention provides a multimodal computing power optimization method and system for security monitoring scenarios. The technical solution is as follows:

[0007] On the one hand, a method for optimizing multimodal computing power in security monitoring scenarios is provided. This method includes: filtering multimodal security data for a specified security scenario to remove invalid multimodal security data; and after the filtering process is qualified, performing a lag analysis of computing power allocation; and determining whether to activate a lag optimization mechanism for computing power allocation based on the analysis results to improve the switching frequency of multimodal computing power allocation tasks. The multimodal security data represents the data set acquired in the security monitoring scenario that reflects the characteristics of video streams, audio signals, and sensor signals. If not activated, a lag analysis of collaborative scheduling is performed to quantify the lag situation when performing collaborative scheduling of computing power based on multimodal security data; otherwise, after the lag optimization mechanism for computing power allocation ends, a lag analysis of collaborative scheduling is performed. Based on the execution results corresponding to the lag analysis of collaborative scheduling, it is determined whether to activate a modal scheduling lag level classification mechanism to classify the lag level of computing power scheduling for multimodal security data, thereby transmitting the corresponding multimodal security data to the cloud center.

[0008] On the other hand, a multimodal computing power optimization system for security monitoring scenarios is provided. This system applies multimodal computing power optimization methods for security monitoring scenarios, including: a computing power allocation lag determination module, a collaborative scheduling lag monitoring module, and a scheduling lag level classification module. The computing power allocation lag determination module is used to filter out invalid security multimodal data for a specified security scenario. After the filtering process is qualified, it performs a lag degree analysis of computing power allocation and determines whether to activate the computing power allocation lag optimization mechanism to improve multimodal computing power based on the analysis results. The task switching frequency is assigned; the collaborative scheduling lag monitoring module is used to perform collaborative scheduling lag degree analysis to quantify the lag situation when computing power is collaboratively scheduled based on security multimodal data if it is not started, otherwise it will perform collaborative scheduling lag degree analysis after the computing power allocation lag optimization mechanism ends; the scheduling lag level classification module is used to determine whether to start the modal scheduling lag level classification mechanism based on the execution results corresponding to the collaborative scheduling lag degree analysis to classify the computing power scheduling lag degree level of security multimodal data, thereby transmitting the corresponding security multimodal data to the cloud center.

[0009] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0010] 1. By screening and processing multimodal security data, and after the screening and processing are qualified, a lag analysis of computing power allocation is performed. Based on the analysis results, it is determined whether to activate the computing power allocation lag optimization mechanism. This helps to achieve accurate allocation and dynamic adaptation of computing power resources, and reduce scheduling redundancy caused by invalid data occupying computing power. If not activated, a collaborative scheduling lag analysis is performed. Otherwise, after the computing power allocation lag optimization mechanism ends, a collaborative scheduling lag analysis is performed. This helps to achieve time synchronization and efficient collaboration in multimodal data processing, and reduce scheduling lag caused by single-modal triggering dependencies. Based on the execution results of the collaborative scheduling lag analysis, it is determined whether to activate the modality scheduling lag level classification mechanism. If activated, after the modality scheduling lag level classification mechanism ends, the corresponding security multimodal data is transmitted to the cloud center. Otherwise, the corresponding security multimodal data is directly transmitted to the cloud center. This helps to improve the real-time performance and accuracy of computing power optimization data under security monitoring, and solves the problem of low real-time performance and accuracy of computing power optimization data under security monitoring in existing technologies.

[0011] 2. By filtering and processing security multimodal data, filtered security multimodal data is obtained. Based on the security multimodal data, edge-side load reduction is performed to obtain the edge-side load level. When the edge-side load level is greater than the initially set edge-side load level, edge-side computing power processing is triggered. Compared with the shortcomings of the existing technology that adopts a fixed computing power allocation strategy for edge nodes and does not consider the dynamic changes of load, this helps to achieve dynamic matching between edge-side computing power and load status, thereby ensuring the stability of multimodal data processing and avoiding scheduling delays or loss of critical data due to overload.

[0012] 3. By analyzing the lag in computing power allocation, the observed values ​​of allocation lag deviation are obtained. When the observed value of allocation lag deviation is not greater than 0, the collaborative scheduling lag analysis is initiated; otherwise, the computing power allocation lag optimization mechanism is initiated. Compared with the shortcomings of existing technologies that lack quantitative evaluation and targeted optimization of computing power allocation lag, this helps to achieve accurate identification and rapid intervention of computing power allocation lag problems, thereby improving the real-time performance of multimodal data processing and solving the problem of missing key information caused by computing power allocation lag when sensors issue sudden warnings.

[0013] 4. When there is a sudden load, such as when an edge node simultaneously handles data processing for multiple monitoring areas or other non-security tasks temporarily occupy computing power, if the observed value of the allocation lag deviation obtained after activating the computing power allocation lag optimization mechanism is still greater than 0, or if the difference between the observed value of the allocation lag deviation in the initial state and the observed value of the allocation lag deviation in the final state of the computing power allocation lag optimization mechanism is not within the predefined range of allocation improvement, the computing power is configured according to the computing power resource allocation value for security multimodal data. Compared with the existing technology, which lacks an elastic computing power replenishment mechanism in the face of sudden loads and is prone to computing power starvation, this helps to achieve rapid replenishment and accurate configuration of computing power resources in sudden load scenarios, ensuring that the processing of key modal data is not affected by sudden loads, and improving the anti-interference capability and emergency response efficiency of the security monitoring system.

[0014] 5. By analyzing the degree of lag in collaborative scheduling, the observed values ​​of multimodal scheduling lag deviation are obtained. When the observed value of multimodal scheduling lag deviation is greater than 0, the multimodal scheduling lag level classification mechanism is activated. Compared with the shortcomings of existing technologies that do not classify the degree of collaborative scheduling lag and are difficult to achieve differentiated handling, this helps to realize the hierarchical control and precise optimization of collaborative scheduling lag problems. According to the lag level, the corresponding control strategy is matched to avoid the waste of computing power or insufficient optimization, thereby improving the precision level of multimodal collaborative scheduling and the overall security monitoring efficiency. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 is a flowchart of a multimodal computing power optimization method for security monitoring scenarios provided in an embodiment of the present invention;

[0017] Figure 2 is an overview diagram of the multimodal computing power optimization method for security monitoring scenarios provided in the embodiments of the present invention;

[0018] Figure 3 is a schematic diagram of the multimodal scheduling lag level classification mechanism of the multimodal computing power optimization method for security monitoring scenarios provided in the embodiments of the present invention;

[0019] Figure 4 is a schematic diagram of the structure of the multimodal computing power optimization system for security monitoring scenarios provided in an embodiment of the present invention. Detailed Implementation

[0020] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0021] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0022] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0023] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0024] This invention provides a multimodal computing power optimization method for security monitoring scenarios. As shown in Figure 1, the flowchart of the multimodal computing power optimization method for security monitoring scenarios includes the following steps:

[0025] Computing power allocation lag determination: For specified security scenarios, multimodal security data is filtered to remove invalid data. After successful filtering, a lag analysis of computing power allocation is performed. Based on the analysis results, it is determined whether to activate a computing power allocation lag optimization mechanism to increase the switching frequency of multimodal computing power allocation tasks. Security multimodal data represents a set of multimodal data acquired in security monitoring scenarios to reflect the characteristics of video streams, audio signals, and sensor signals. By determining computing power allocation lag, it helps to accurately filter invalid multimodal data, quantitatively identify the lag state of computing power allocation, avoid computing power waste and lag omissions caused by fixed allocation strategies, and ensure that computing power resources are tilted towards high-demand modes.

[0026] Collaborative scheduling lag monitoring: If not started, collaborative scheduling lag analysis will be performed to quantify the lag situation when computing power is collaboratively scheduled based on security multimodal data. Otherwise, collaborative scheduling lag analysis will be performed after the computing power allocation lag optimization mechanism ends. By performing collaborative scheduling lag monitoring, it is helpful to achieve accurate quantification of the lag degree of multimodal collaborative scheduling, verify the actual effect of computing power allocation optimization measures, and ensure the timing synchronization of video, audio, and sensor data processing.

[0027] Scheduling lag level classification: Based on the execution results corresponding to the collaborative scheduling lag degree analysis, determine whether to activate the modal scheduling lag level classification mechanism to classify the computing power scheduling lag degree level of security multimodal data, thereby transmitting the corresponding security multimodal data to the cloud center; by classifying scheduling lag levels, it is helpful to realize the graded and differentiated handling of scheduling lag problems, allowing the cloud center to conduct targeted in-depth analysis (such as prioritizing the analysis of severely lagging data).

[0028] Before designing the multimodal computing power optimization method for security monitoring scenarios provided in this application, a database storing various setting data is established. This database includes, but is not limited to, predefined early warning signal trigger values ​​and initial security video data filtering values. These values ​​are directly set by technical personnel, who consider multiple factors when setting them. On the one hand, they ensure that the set values ​​meet the basic requirements of the security monitoring field based on relevant industry standards and technical specifications. On the other hand, they make targeted adjustments and optimizations based on the characteristics and needs of the actual application scenario, such as the size of the monitoring area, personnel flow, and equipment performance. Simultaneously, technical personnel also refer to historical data and actual cases to analyze the reasonable range of various values ​​under different conditions, thereby improving the accuracy and practicality of the value settings. During the setting process, technical personnel conduct multiple simulation tests and verifications, repeatedly adjusting the values ​​based on the test results until the optimal setting effect is achieved, thus providing a solid data guarantee for the multimodal computing power optimization method in security monitoring scenarios.

[0029] Figure 2 shows an overall overview of the multimodal computing power optimization method for security monitoring scenarios provided in the embodiments of this application. As shown in Figure 2, the method involves: obtaining security multimodal data through screening and processing; obtaining edge-side load based on the security multimodal data; triggering edge-side computing power processing when the monitored edge-side load is greater than the initially set edge-side load; otherwise, initiating the lag degree analysis of computing power allocation and obtaining the allocation lag deviation observation value; if the monitored allocation lag deviation observation value is not greater than 0, initiating the collaborative scheduling lag degree analysis; otherwise, initiating the computing power allocation lag optimization mechanism; obtaining the multimodal scheduling lag deviation observation value through the collaborative scheduling lag degree analysis; when the multimodal scheduling lag deviation observation value is greater than 0, marking the corresponding multimodal scheduling lag deviation observation value as a lag deviation observation value to be classified, initiating the multimodal scheduling lag level classification mechanism; otherwise, transmitting the corresponding security multimodal data to the cloud center.

[0030] In this embodiment, the interaction and interrelation of computing power allocation lag determination, collaborative scheduling lag monitoring, and scheduling lag level classification help to achieve closed-loop management of the entire process, from invalid data filtering to lag status determination, optimization effect verification, and hierarchical transmission handling. This helps to solve core problems in existing technologies such as modal collaborative scheduling lag, insufficient accuracy of computing power allocation, and loss of key information. In turn, it ensures the real-time processing and effective transmission of key modal data in emergency scenarios such as sudden sensor warnings, and improves the overall response speed and anomaly identification accuracy of the security monitoring system.

[0031] Further, the security multimodal data is filtered and processed, including the following steps: Step 1: Monitor the reception and triggering of warning signals: Based on the security warning signal strength (monitored via a Tektronix oscilloscope) being greater than the preset warning signal strength set by preset personnel for the duration within a preset time window (monitored via a timer), a security warning signal trigger value is output. When the monitored security warning signal trigger value is greater than the predefined warning signal trigger value, the security multimodal data is filtered; otherwise, a scheduling qualification prompt is sent, and the security multimodal data continues to be monitored. Among these, the predefined warning... The signal trigger value is represented by the average value of security warning signal trigger values ​​over a historical time period. In the second processing step, security multimodal data is filtered to remove invalid data and reduce the computational burden on security monitoring scenarios. A lightweight edge-side filtering method is used to filter qualified security multimodal data, and the filtered data is then transmitted to edge nodes for multimodal analysis. Qualified security multimodal data includes qualified security video data, qualified security audio data, and qualified security sensor data. The specific method for obtaining qualified security video data is as follows: within a preset time window, data is collected using a video quality analyzer. The maximum peak signal-to-noise ratio (PSNR) of the target motion trajectory video stream is used as the screening value for quantifying the passability of security video data. Security video data with a screening value greater than the initial screening value (such as continuous tracking frames of people or vehicles, superimposed frames of target motion trajectories, etc.) are marked as qualified security video data. Security video data represents the video stream data used to reflect the target motion trajectory in security monitoring scenarios. The initial security video data screening value is represented by the average value of security video data screening values ​​over a historical time period. Qualified security audio data is specifically obtained through... The method is as follows: obtain the maximum signal-to-noise ratio corresponding to the audio signal monitored by the audio analyzer within a preset time window, and use it as the security audio data screening value to quantify the qualification of security audio data. Security audio data with a security audio data screening value greater than the initial security audio data screening value (such as the decibel of the equipment fault alarm sound, the decibel of the prompt sound after the computing power scheduling is triggered, etc.) are marked as qualified security audio data. Security audio data represents the data used to reflect the characteristics of audio signals in the security monitoring scenario. The initial security audio data screening value is represented by the average value of the security audio data screening values ​​over a historical time period.The specific method for obtaining qualified security sensor data is as follows: Within a preset time window, obtain the maximum signal strength of the sensor signal monitored by a Tektronix oscilloscope. Use this as the screening value for quantifying the qualification of the security sensor data. Security sensor data whose screening value is greater than the initially set screening value (such as the trigger signal strength of an infrared sensor, the vibration amplitude of a vibration sensor, etc.) are marked as qualified security sensor data. Security sensor data represents data reflecting the characteristics of sensor signals in a security monitoring scenario. The initially set security sensor data screening value is represented by the average value of security sensor data screening values ​​over a historical time period. Step three involves performing multimodal security monitoring. Edge-side load reduction is implemented to decrease the load on security multimodal data during multimodal analysis at edge nodes. Specifically, it works as follows: Based on the timer monitoring of security multimodal data, the duration from triggering multimodal analysis to the start of analysis is calculated. The edge-side load level is then output, and the decision to trigger edge-side computing power processing is based on this load level. If the edge-side load level exceeds the initial set limit, an instruction to trigger edge-side computing power processing is output, and processing is initiated within the next preset time window. This allows for lag analysis of computing power allocation to be performed under load-appropriate conditions. Conversely, if the load level is below the initial limit, lag analysis of computing power allocation is initiated based on the corresponding security multimodal data. The initial edge-side load level is represented by the average edge-side load level over a historical time period.

[0032] By filtering security multimodal data when the security warning signal trigger value exceeds a predefined warning signal trigger value, it helps to accurately extract key multimodal data in emergency warning scenarios and avoid scheduling delays caused by non-critical data occupying computing power. By acquiring security video data when the security video data filtering value exceeds the initial security video data filtering value, security audio data when the security audio data filtering value exceeds the initial security audio data filtering value, and security sensor data when the security sensor data filtering value exceeds the initial security sensor data filtering value, and by reducing the edge-side burden of security multimodal data based on the acquired data, it helps to accurately control edge-side computing power consumption, reduce redundant computing power occupation while ensuring the integrity of key modal data, and reserve sufficient computing power for critical tasks such as audio depth analysis during sudden sensor warnings.

[0033] Specifically, the process of triggering edge-side computing power processing is as follows: First, the edge-side load, security multimodal data, and the corresponding data volume of security multimodal data monitored by network traffic monitoring tools (such as Wireshark) are used as optimization parameter sets and input into a pre-built video frame rate optimization set, outputting a predefined video frame rate reading value. Second, within a predefined video frame rate range set by preset personnel, a video frame rate reduction operation is performed on the security multimodal data, using the amplitude corresponding to the predefined video frame rate reading value as the adjustment step size. Third, the edge-side load at the initial state of triggering edge-side computing power processing and the edge-side load after performing a video frame rate reduction operation on the security multimodal data are obtained, and the difference between the two is used as the video frame rate reduction qualification assessment value. Fourth, if the video frame rate reduction qualification assessment value is not greater than the predefined video frame rate reduction qualification assessment value, a video frame rate reduction alarm is sent; if the video frame rate reduction qualification assessment value is greater than the predefined video frame rate reduction qualification assessment value, an alarm is sent. The process begins with reducing the acceptable evaluation value, followed by step five. Here, the predefined acceptable evaluation value for decreasing video frame rate is represented by the average of acceptable evaluation values ​​over a historical time period. Step five involves performing a preset number of video frame rate reduction operations on the security multimodal data. If the newly acquired edge-side load is not greater than the initially set edge-side load, a lag analysis of computing power allocation is initiated based on the corresponding security multimodal data; otherwise, an edge-side computing power processing alarm is triggered. By using the predefined amplitude corresponding to the video frame rate reading value as the adjustment step size when the acceptable evaluation value for decreasing video frame rate is greater than the predefined acceptable evaluation value, and performing a preset number of video frame rate reduction operations on the security multimodal data, the extraction accuracy of target features (such as individuals in a restricted area power distribution room) in the security multimodal data is improved. This achieves a dynamic balance between edge-side computing power consumption and video processing accuracy, providing high-quality video data support for multimodal collaborative analysis and enhancing the security monitoring system's ability to accurately identify abnormal events.

[0034] In the implementation scheme of this application, pre-built tables obtained from the database are used, including video frame rate optimization set, first-level scheduling related adjustment table, first-level scheduling number related adjustment table, cache cleanup frequency related table, etc. These built tables have multiple mapping functions. On the one hand, they can realize a precise one-to-one mapping relationship between individual parameters; on the other hand, they also support many-to-one mapping between multiple parameters and individual parameters.

[0035] This can be achieved by inputting various key information collected by pre-defined personnel within a specific historical time period. This information includes, but is not limited to, edge load, optimization parameter sets for security multimodal data and its volume, combinations of first-order scheduling lag control values ​​and the volume of security multimodal data, combinations of first-order scheduling lag pass inspection values ​​and the volume of security multimodal data, and combinations of second-order scheduling lag control values ​​and the memory usage of security multimodal data. These collected key information are then input into a machine learning model capable of revealing the criticality of each feature. Taking a decision tree model as an example, the model's own feature splitting mechanism, based on predefined mapping relationships, extracts results that can be used as weight values ​​or other relevant data information. These results include, but are not limited to, predefined video frame rate readout values, combinations of predefined video frame rate control values ​​and predefined resolution control values, predefined first-order scheduling lag adjustment times, and predefined cache clearing frequency adjustment values. This information is then used to analyze the historical time period model. The acquired data is associated and matched with corresponding weights or data information to generate a video frame rate optimization set, a first-level scheduling related adjustment table, a first-level scheduling frequency related adjustment table, and a cache clearing frequency related table. Information such as the real-time collected edge-side load, the set of optimization parameters for the security multimodal data and its data volume, the combination of first-level scheduling lag control values ​​and the data volume of security multimodal data, the combination of first-level scheduling lag pass inspection values ​​and the data volume of security multimodal data, and the combination of second-level scheduling lag control values ​​and the memory of security multimodal data are input into the pre-constructed tables, including the video frame rate optimization set, the first-level scheduling related adjustment table, the first-level scheduling frequency related adjustment table, and the cache clearing frequency related table. This allows the retrieval of predefined video frame rate reading values, combinations of predefined video frame rate control values ​​and predefined resolution control values, predefined first-level scheduling lag adjustment times, and predefined cache clearing frequency adjustment values.

[0036] In this embodiment, the edge-side load is obtained by filtering and processing security multimodal data. When the edge-side load exceeds the initially set edge-side load, edge-side computing power processing is triggered. This helps to achieve accurate perception of the edge-side load status and proactive intervention in computing power scheduling, avoiding multimodal data processing lag caused by continuous overload. Through the interrelation and connection between the security multimodal data filtering and processing and the triggering of edge-side computing power processing, it helps to achieve the linkage optimization of invalid data filtering and load reduction and overload-limited computing power adjustment, thereby realizing the elastic adaptation and efficient utilization of edge node computing power resources.

[0037] Furthermore, the specific process for analyzing the lag in computing power allocation is as follows: The time elapsed from the start of computing power resource configuration monitored by a timer to the completion of parameter initialization for security multimodal data is obtained and marked as the computing power allocation lag observation value, used to quantify the lag when allocating computing power based on security multimodal data; the difference between the computing power allocation lag observation value and a predefined computing power allocation lag observation value is obtained and marked as the allocation lag deviation observation value, used to quantify the lag deviation when allocating computing power, wherein the predefined computing power allocation lag observation value is represented by the average value of computing power allocation lag observation values ​​over historical time periods; based on the allocation lag deviation observation value, it is determined whether to activate the computing power allocation lag optimization mechanism. Specifically, if the allocation lag deviation observation value is not greater than 0, a computing power allocation qualified prompt is sent and collaborative scheduling lag analysis is activated; otherwise, a computing power allocation lag optimization mechanism that increases task switching frequency is activated within the next preset time window; the computing power allocation lag optimization mechanism is used to accurately reduce the lag in computing power allocation for multimodal data processing.

[0038] It should be added that the computing power allocation lag optimization mechanism is as follows: Predict the execution time slice adjustment step size for security multimodal data: Input the observed allocation lag deviation and the resource pool size monitored by the network storage monitoring tool (Zabbix) into a preset machine learning-execution time slice related model, outputting a preset execution time slice adjustment step size and a preset number of execution time slice adjustment reductions; use the preset execution time slice adjustment step size as the baseline step size for the execution time slice adjustment reduction operation, and use the preset number of execution time slice adjustment reductions as the baseline reduction number for the execution time slice adjustment reduction operation. This execution time slice adjustment reduction operation helps improve the accuracy and efficiency of security multimodal data time slice allocation, quickly releases redundant computing power occupied by non-critical modalities such as video, and avoids computing power contention. This leads to a lag in the processing of key modalities, thereby enabling the targeted allocation of computing resources to core tasks such as sensor-related early warning data and audio depth analysis, ensuring the comprehensive analysis accuracy and emergency response capability of security monitoring under limited computing power. The system reacquires the allocation lag deviation observations. If the allocation lag deviation observation is not greater than 0, and the difference between the initial and final allocation lag deviation observations of the computing power allocation lag optimization mechanism (i.e., the difference between the corresponding data allocation lag deviation observation in the final state) is within the predefined allocation improvement range set by preset personnel, a computing power allocation qualification prompt is sent, and collaborative scheduling lag degree analysis is performed. Otherwise, computing power resources are re-allocated, i.e., a prompt is sent to preset personnel to reconfigure computing power resources for security multimodal data. Computing power resources include CPU (Central Processing Unit), GPU (Graphics Processing Unit), and memory.

[0039] Specifically, in the implementation scheme of this application, the data set of allocation lag bias observations, resource pool size, preset execution time slice adjustment step size and preset execution time slice adjustment decrement number are randomly divided into training sets and input into a preset machine learning model, such as a decision tree model, to obtain a preset machine learning-execution time slice related model. The combination of allocation lag bias observations and resource pool size is then input into the preset machine learning-execution time slice related model to output the preset execution time slice adjustment step size and preset execution time slice adjustment decrement number.

[0040] In this embodiment, the lag analysis of computing power allocation is used to obtain the observed value of allocation lag deviation. When the observed value of allocation lag deviation is not greater than 0, the collaborative scheduling lag analysis is initiated; otherwise, the computing power allocation lag optimization mechanism is initiated. This helps to achieve accurate quantification and differentiated handling of the lag state of computing power allocation, avoiding the waste of computing power or the omission of lag problems that lead to scheduling delays due to blindly initiating the optimization mechanism. This ensures the dynamic adaptation of computing power resource allocation to the needs of multimodal data processing. Through the interaction and interconnection of the lag analysis of computing power allocation, the collaborative scheduling lag analysis, and the computing power allocation lag optimization mechanism, it helps to achieve closed-loop management of the entire process from accurate lag identification to targeted optimization intervention and then to collaborative effect verification. This solves the core pain points of existing technologies, such as the lack of feedback verification in single optimization and the imbalance of modal collaboration. In turn, it improves the real-time response speed of multimodal data processing and ensures the timely analysis of key audio and sensor data in emergency scenarios such as sudden sensor warnings.

[0041] Furthermore, the allocation of computing resources also includes: obtaining a computing resource allocation value, which reflects the allocation of computing resources for security multimodal data during sudden loads; using the value corresponding to the computing resource allocation value as the computing resource allocation benchmark for security multimodal data; and sending a prompt to preset personnel to configure the corresponding computing power for security multimodal data based on the computing resource allocation benchmark.

[0042] The computing power resource allocation value is obtained through the following method:

[0043] ;

[0044] In the formula, C zong K represents the computing resource allocation value, k represents the modality type identifier, and in this embodiment, the modality type identifier for security video data is set to 1, the modality type identifier for security audio data is set to 2, and the modality type identifier for security sensor data is set to 3. k D represents the basic computing power coefficient of the k-th mode. k W represents the real-time data volume of the k-th mode. k C represents the real-time priority weight of the k-th modality. yuThis indicates the amount of emergency computing power reserved.

[0045] The unit of computing power resource allocation is FLOPS, the unit of basic computing power coefficient is FLOPS / (MB / s), which is represented by the computing power consumption per unit time monitored by the data collector, the unit of real-time data volume is MB / s, which is monitored by network traffic monitoring tools (such as Wireshark), the unit of emergency computing power reserve is FLOPS, and the emergency computing power reserve and real-time priority weight are set by preset personnel.

[0046] In this embodiment, when there is a sudden load, such as edge nodes simultaneously handling data processing for multiple monitoring areas, other non-security tasks (such as equipment status monitoring) temporarily occupying computing power, or instantaneous performance degradation of node hardware (such as CPU frequency reduction due to excessively high temperature), the emergency computing power reserve can be used as a computing power buffer to prioritize the processing resources of core security modalities (sensors, audio). Based on the computing power resource allocation value, the corresponding computing power is configured for security multimodal data, which helps to achieve a dynamic balance between computing power supply and demand and alleviate the instantaneous computing power pressure on nodes. This enables the complete capture and real-time processing of key information such as sensor warning signals and audio anomaly features. It also helps to strengthen the anti-interference capability of the security monitoring system in the face of complex load fluctuations, thereby ensuring the timeliness and accuracy of abnormal event identification in emergency scenarios.

[0047] Furthermore, the specific process for analyzing the lag degree of collaborative scheduling is as follows: Based on the time consumed from the completion of computing resource allocation monitored by the timer to the initiation of in-depth analysis based on security multimodal data, obtain multimodal scheduling lag observation values ​​to quantify the lag situation when performing collaborative computing resource scheduling based on security multimodal data; based on the difference between the multimodal scheduling lag observation values ​​and predefined multimodal scheduling lag observation values, output multimodal scheduling lag deviation observation values ​​to characterize the lag deviation situation when performing collaborative computing resource scheduling of security multimodal data. The predefined multimodal scheduling lag observation values ​​are represented by the average value of multimodal scheduling lag observation values ​​over historical time periods; initiate a judgment mechanism based on the multimodal scheduling lag deviation observation values, specifically as follows: if the multimodal scheduling lag deviation observation value is greater than 0, mark the corresponding multimodal scheduling lag deviation observation value as a lag deviation observation value to be classified, and initiate the multimodal scheduling lag level classification mechanism in the next preset time window; otherwise, transmit the corresponding security multimodal data to the cloud center.

[0048] Figure 3 shows a schematic diagram of the multimodal scheduling lag level classification mechanism of the multimodal computing power optimization method for security monitoring scenarios provided in the embodiments of this invention. As shown in Figure 3, the multimodal scheduling lag level classification mechanism is as follows: The observed lag deviation to be classified is compared with a predefined scheduling lag deviation range set by preset personnel; the observed lag deviation to be classified that is greater than the upper limit of the predefined scheduling lag deviation range is marked as a first-level scheduling lag control value, and the first-level scheduling lag control is initiated to quickly release redundant computing power of edge nodes; the predefined... The observed values ​​of the lag deviation to be divided within the scheduling lag deviation range are marked as second-level scheduling lag control values, and second-level scheduling lag control is initiated. Second-level scheduling lag control is used to release redundant memory resources of edge nodes and ensure efficient data transmission to the cloud center. The observed values ​​of the lag deviation to be divided that are less than the lower limit of the predefined scheduling lag deviation range are marked as third-level scheduling lag control values, and the corresponding security multimodal data is transmitted to the cloud center. The scheduling lag deviation degree corresponding to the first-level scheduling lag control value, the second-level scheduling lag control value, and the third-level scheduling lag control value decreases in that order.

[0049] By marking the observed values ​​of lag deviations to be divided that exceed the upper limit of the predefined scheduling lag deviation range as first-level scheduling lag control values ​​and initiating first-level scheduling lag control, it helps to achieve rapid release of computing power in severe lag scenarios. Through the dual-dimensional reduction of video frame rate and resolution, it efficiently alleviates the pressure on core modal computing power and solves the problem of key information loss caused by severe lag. Marking the observed values ​​of lag deviations to be divided that fall within the predefined scheduling lag deviation range as second-level scheduling lag control values ​​and initiating second-level scheduling lag control releases redundant space by adjusting the cache cleanup frequency, accurately alleviating scheduling lag caused by non-computing power bottlenecks. Marking the observed values ​​of lag deviations to be divided that fall below the lower limit of the predefined scheduling lag deviation range as third-level scheduling lag control values ​​and transmitting the corresponding security multimodal data to the cloud center helps to facilitate efficient data flow in slightly lagging fields, avoids accuracy loss or computing power waste caused by over-control, and improves overall scheduling efficiency.

[0050] In this embodiment, multimodal scheduling lag deviation observations are obtained through collaborative scheduling lag degree analysis, and multimodal scheduling lag deviation observations greater than 0 are marked as lag deviation observations to be classified. The multimodal scheduling lag level classification mechanism is activated, which helps to achieve accurate identification and classification of collaborative scheduling lag problems, providing clear targets for differentiated regulation. The collaborative scheduling lag degree analysis and the multimodal scheduling lag level classification mechanism are mutually supportive, which helps to achieve a seamless connection from common lag quantification to level definition, avoiding missed judgment of lag problems or insufficient regulation targeting.

[0051] Furthermore, the specific process of first-level scheduling lag control is as follows: The first-level scheduling lag control value and the amount of security multimodal data monitored by network traffic monitoring tools (such as Wireshark) are input into a pre-constructed first-level scheduling related adjustment table. Predefined video frame rate control values ​​and predefined resolution control values ​​for the security multimodal data are read. Using the division corresponding to the predefined video frame rate control values ​​and predefined resolution control values ​​as the benchmark adjustment step size, a video frame rate reduction operation and a resolution reduction operation for the security multimodal data are performed simultaneously. The corresponding multimodal scheduling lag deviation observation value is obtained. The multimodal scheduling lag deviation observation value of the initial state of first-level scheduling lag control and its difference are used as the first-level scheduling lag pass inspection value. It is then determined whether the first-level scheduling lag pass inspection value passes the pre-defined adjustment table. If the predefined first-level scheduling lag compliance range is not set by the personnel, a first-level scheduling lag step size alarm is sent. If it is set, the corresponding first-level scheduling lag compliance test value and the amount of security multimodal data are input into the pre-built first-level scheduling frequency related adjustment table, and the predefined first-level scheduling lag adjustment frequency is read. The predefined first-level scheduling lag adjustment frequency is used as the adjustment frequency, and the amplitudes corresponding to the predefined video frame rate control value and the predefined resolution control value are used as the adjustment step size of the corresponding decrement operation. After executing the corresponding decrement operation, the multimodal scheduling lag deviation observation value is reacquired. If the reacquired multimodal scheduling lag deviation observation value is still greater than 0, a first-level scheduling lag control alarm is sent; otherwise, the corresponding security multimodal data is transmitted to the cloud center.

[0052] It should be added that the specific process of second-level scheduling lag control is as follows: Input the second-level scheduling lag control value and the memory of the security multimodal data monitored by the resource monitor (such as Resource Monitor) into the pre-built cache clearing frequency related table, and read the predefined cache clearing frequency adjustment value; send a prompt to the preset personnel, and adjust the cache clearing frequency to the predefined cache clearing frequency adjustment value based on the initial cache clearing frequency of the security multimodal data; obtain the multimodal scheduling lag deviation observation value after the cache clearing frequency is adjusted to the predefined cache clearing frequency adjustment value. If the multimodal scheduling lag deviation observation value is still greater than 0, send a second-level scheduling lag control alarm prompt; otherwise, transmit the corresponding security multimodal data to the cloud center.

[0053] Figure 4 shows a schematic diagram of the structure of a multimodal computing power optimization system for security monitoring scenarios provided in an embodiment of this invention. The system utilizes a multimodal computing power optimization method for security monitoring scenarios, including: a computing power allocation lag determination module, a collaborative scheduling lag monitoring module, and a scheduling lag level classification module. The computing power allocation lag determination module is used to filter out invalid security multimodal data for a specified security scenario. After the filtering process is successful, it performs a lag degree analysis of computing power allocation and determines whether to activate the computing power allocation lag optimization mechanism based on the analysis results to improve performance. The switching frequency of high multimodal computing power allocation tasks; the collaborative scheduling lag monitoring module is used to perform collaborative scheduling lag degree analysis to quantify the lag situation when computing power is collaboratively scheduled based on security multimodal data if it is not started, otherwise it will perform collaborative scheduling lag degree analysis after the computing power allocation lag optimization mechanism ends; the scheduling lag level classification module is used to determine whether to start the modal scheduling lag level classification mechanism based on the execution results corresponding to the collaborative scheduling lag degree analysis to classify the computing power scheduling lag degree level of security multimodal data, thereby transmitting the corresponding security multimodal data to the cloud center.

[0054] In this embodiment, when the first-level scheduling lag qualification test value is within the predefined first-level scheduling lag qualification range, the amplitudes corresponding to the predefined video frame rate control value and the predefined resolution control value are used as the adjustment step size of the corresponding decreasing operation, and the corresponding decreasing operation is executed simultaneously. This helps to improve the compression efficiency of video data computing power consumption, quickly release redundant computing power to tilt towards core modalities such as audio and sensors, and thus achieve rapid matching of computing power supply and demand in severe lag scenarios, ensuring real-time response to emergency events. By adjusting the cache clearing frequency to the predefined cache clearing frequency adjustment value based on the initial cache clearing frequency of security multimodal data, it helps to realize the dynamic release of edge-side memory resources, reduce scheduling delays caused by invalid cache occupation, and ensure the smoothness of multimodal data processing and transmission.

[0055] In summary, the embodiments of this invention perform screening and processing of security multimodal data, and after the screening and processing is qualified, perform a lag analysis of computing power allocation. Based on the analysis results, it determines whether to activate the computing power allocation lag optimization mechanism. This helps to achieve accurate allocation and dynamic adaptation of computing power resources, and reduce scheduling redundancy caused by invalid data occupying computing power. If not activated, a collaborative scheduling lag analysis is performed; otherwise, after the computing power allocation lag optimization mechanism ends, a collaborative scheduling lag analysis is performed. This helps to achieve time synchronization and efficient collaboration in multimodal data processing, and reduce scheduling lag caused by single-modal triggering dependencies. Based on the execution results corresponding to the collaborative scheduling lag analysis, it determines whether to activate the modal scheduling lag level classification mechanism. If activated, after the modal scheduling lag level classification mechanism ends, the corresponding security multimodal data is transmitted to the cloud center; otherwise, the corresponding security multimodal data is directly transmitted to the cloud center. This helps to improve the real-time performance and accuracy of computing power optimization data under security monitoring, and solves the problem of low real-time performance and accuracy of computing power optimization data under security monitoring in the prior art.

[0056] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A multimodal computing power optimization method for security monitoring scenarios, characterized in that, The method includes: performing security multimodal data filtering on a specified security scenario to filter out invalid security multimodal data; after the filtering process is qualified, performing a lag analysis of computing power allocation, obtaining the time elapsed from the start of computing power resource configuration to the completion of parameter initialization of the security multimodal data, marking this as a computing power allocation lag observation value, obtaining the difference between the computing power allocation lag observation value and a predefined computing power allocation lag observation value, marking this as an allocation lag deviation observation value, used to quantify the lag deviation during computing power allocation, and determining whether to initiate computing power allocation based on the analysis results. A lag optimization mechanism is implemented to improve the switching frequency of multimodal computing power allocation tasks. The security multimodal data refers to the data set acquired in the security monitoring scenario to reflect the characteristics of video streams, audio signals, and sensor signals. If not activated, a collaborative scheduling lag degree analysis is performed to quantify the lag situation when performing collaborative scheduling of computing power based on security multimodal data. Conversely, after the computing power allocation lag optimization mechanism ends, a collaborative scheduling lag degree analysis is performed. Based on the execution results corresponding to the collaborative scheduling lag degree analysis, it is determined whether to activate the modal scheduling lag level classification mechanism to classify the computing power scheduling lag degree level of security multimodal data, thereby transmitting the corresponding security multimodal data to the cloud center. The computing power allocation lag optimization mechanism is as follows: the allocation lag deviation observation value and the resource pool size are input into a preset machine learning-execution time slice correlation model, and the preset execution time slice adjustment step size and preset execution time slice adjustment decrement number are output. The preset execution time slice adjustment step size is used as the base step size for the execution time slice adjustment decrement operation, and the preset execution time slice adjustment decrement number is used as the execution time slice adjustment... The baseline decrement count for the decrement operation is used to perform time-slice adjustment decrement operations. The allocation lag deviation observation value is reacquired. If the allocation lag deviation observation value is not greater than 0, and the difference between the allocation lag deviation observation value of the initial state and the corresponding observation value of the final state in the computing power allocation lag optimization mechanism is within the predefined allocation improvement qualified range, a computing power allocation qualified prompt is sent and a collaborative scheduling lag degree analysis is performed. Otherwise, the computing power resources are re-allocated, that is, a prompt is sent to the preset personnel to reconfigure computing power resources for security multimodal data. The computing power resources include CPU, GPU and memory.

2. The multimodal computing power optimization method for security monitoring scenarios according to claim 1, characterized in that, The process of filtering and processing security multimodal data includes the following steps: Monitoring the reception and triggering of early warning signals: Based on the duration for which the security early warning signal strength exceeds a preset early warning signal strength within a preset time window, outputting a security early warning signal trigger value. When the monitored security early warning signal trigger value exceeds a predefined early warning signal trigger value, filtering of security multimodal data is performed; otherwise, a scheduling qualification prompt is sent, and monitoring of security multimodal data continues. Filtering of security multimodal data involves using a lightweight edge-side filtering method to filter qualified security multimodal data, obtaining the filtered security multimodal data, and transmitting it to edge nodes for multimodal analysis. The qualified security multimodal data... The data includes qualified security video data, qualified security audio data, and qualified security sensor data. The qualified security video data is specifically acquired by: obtaining the maximum peak signal-to-noise ratio (PSNR) of the target motion trajectory video stream within a preset time window, using this as a screening value to quantify the qualification level of the security video data; and marking security video data with a screening value greater than the initially set screening value as qualified security video data. The security video data represents the video stream data used to reflect the target motion trajectory in a security monitoring scenario. The qualified security audio data is specifically acquired by: obtaining the signal-to-noise ratio (SNR) of the audio signal within a preset time window. The maximum value of the signal strength is used as the screening value for quantifying the passability of security audio data. Security audio data with a screening value greater than the initial screening value are marked as qualified security audio data. The security audio data refers to data used to reflect the characteristics of audio signals in a security monitoring scenario. The qualified security sensor data is obtained by: obtaining the maximum signal strength of the sensor signal within a preset time window, using it as the screening value for quantifying the passability of security sensor data, and marking security sensor data with a screening value greater than the initial screening value as qualified security sensor data. Sensor data, specifically security sensor data, refers to data used to reflect sensor signal characteristics in security monitoring scenarios. Edge-side load reduction for security multimodal data is performed as follows: Based on the time elapsed from triggering multimodal analysis to the start of multimodal analysis, the edge-side load level is output. The edge-side computing power processing is then determined based on the edge-side load level. If the edge-side load level is greater than the initially set edge-side load level, an instruction to trigger edge-side computing power processing is output, and edge-side computing power processing is triggered within the next preset time window to initiate lag analysis of computing power allocation under the premise of load adaptation. Conversely, lag analysis of computing power allocation is initiated based on the corresponding security multimodal data.

3. The multimodal computing power optimization method for security monitoring scenarios according to claim 2, characterized in that, The specific process of triggering edge-side computing power processing is as follows: First, the edge-side load, security multimodal data, and corresponding data volume are used as an optimization parameter set and input into a pre-constructed video frame rate optimization set, outputting a predefined video frame rate read value. Second, within the predefined video frame rate range, the amplitude corresponding to the predefined video frame rate read value is used as the adjustment step size to perform a video frame rate reduction operation on the security multimodal data. Third, the edge-side load in the initial state of triggering edge-side computing power processing and the edge-side load after performing one video frame rate reduction operation on the security multimodal data are obtained, and... The difference between the two is used as the video frame rate reduction qualification assessment value; in the fourth step, if the video frame rate reduction qualification assessment value is not greater than the predefined video frame rate reduction qualification assessment value, a video frame rate reduction alarm is sent; if the video frame rate reduction qualification assessment value is greater than the predefined video frame rate reduction qualification assessment value, the fifth step is executed; in the fifth step, after performing the video frame rate reduction operation of the security multimodal data a preset number of times, if the re-acquired edge-side load is not greater than the initially set edge-side load, the lag analysis of computing power allocation is initiated based on the corresponding security multimodal data; otherwise, an alarm is sent to trigger edge-side computing power processing.

4. The multimodal computing power optimization method for security monitoring scenarios according to claim 3, characterized in that, The specific process for analyzing the lag in computing power allocation is as follows: Based on the observed value of allocation lag deviation, it is determined whether to activate the computing power allocation lag optimization mechanism. Specifically, if the observed value of allocation lag deviation is not greater than 0, a computing power allocation qualified prompt is sent and the collaborative scheduling lag analysis is activated; otherwise, the computing power allocation lag optimization mechanism that increases the task switching frequency is activated within the next preset time window.

5. The multimodal computing power optimization method for security monitoring scenarios according to claim 1, characterized in that, The process of allocating computing resources further includes: obtaining a computing resource allocation value, which is used to reflect the computing resource allocation of security multimodal data during sudden loads; using the value corresponding to the computing resource allocation value as the computing resource allocation benchmark for security multimodal data; and sending a prompt to preset personnel to configure the corresponding computing power for security multimodal data based on the computing resource allocation benchmark.

6. The multimodal computing power optimization method for security monitoring scenarios according to claim 1, characterized in that, The specific process of the collaborative scheduling lag analysis is as follows: Based on the time taken from the completion of computing resource allocation to the initiation of in-depth analysis based on security multimodal data, obtain multimodal scheduling lag observation values; based on the difference between the multimodal scheduling lag observation values ​​and predefined multimodal scheduling lag observation values, output multimodal scheduling lag deviation observation values ​​to characterize the lag deviation situation when security multimodal data is used for computing resource collaborative scheduling; initiate a judgment mechanism based on the multimodal scheduling lag deviation observation values, specifically as follows: if the multimodal scheduling lag deviation observation value is greater than 0, the corresponding multimodal... The observed values ​​of the lag deviation in the current scheduling are marked as lag deviation observation values ​​to be classified, and the multimodal scheduling lag level classification mechanism is initiated in the next preset time window. Otherwise, the corresponding security multimodal data is transmitted to the cloud center. The multimodal scheduling lag level classification mechanism is as follows: the observed values ​​of the lag deviation to be classified are compared with the predefined scheduling lag deviation range; the observed values ​​of the lag deviation to be classified that are greater than the upper limit of the predefined scheduling lag deviation range are marked as first-class scheduling lag control values, and first-class scheduling lag control is initiated to quickly release the redundant computing power of edge nodes. The observed values ​​of the lag deviation to be divided within the predefined scheduling lag deviation range are marked as second-level scheduling lag control values, and second-level scheduling lag control is initiated. The second-level scheduling lag control is used to release the redundant memory resources of the edge nodes. The observed values ​​of the lag deviation to be divided that are less than the lower limit of the predefined scheduling lag deviation range are marked as third-level scheduling lag control values, and the corresponding security multimodal data is transmitted to the cloud center. The scheduling lag control values ​​for the first-level scheduling lag control value, the second-level scheduling lag control value, and the third-level scheduling lag control value correspond to a decreasing degree of scheduling lag deviation in that order.

7. The multimodal computing power optimization method for security monitoring scenarios according to claim 6, characterized in that, The specific process of the first-level scheduling lag control is as follows: input the first-level scheduling lag control value and the amount of security multimodal data into the pre-constructed first-level scheduling related adjustment table, and read the predefined video frame rate control value and predefined resolution control value of the security multimodal data; Using the predefined video frame rate control value and the predefined resolution control value as the corresponding division as the benchmark adjustment step size, and simultaneously performing a video frame rate reduction operation and a resolution reduction operation on the security multimodal data, the corresponding multimodal scheduling lag deviation observation value is obtained. The multimodal scheduling lag deviation observation value of the initial state of the first-level scheduling lag control and its difference are used as the first-level scheduling lag qualification check value. It is determined whether the first-level scheduling lag qualification check value is within the predefined first-level scheduling lag qualification range. If not, a first-level scheduling lag step size alarm prompt is sent. If present, input the corresponding first-level scheduling lag qualification test value and the amount of security multimodal data into the pre-built first-level scheduling number related adjustment table, and read the predefined first-level scheduling lag adjustment number. The number of predefined first-order scheduling lag adjustments is used as the adjustment number. The amplitudes corresponding to the predefined video frame rate control value and the predefined resolution control value are used as the adjustment step size of the corresponding decrement operation. After the corresponding decrement operation is executed, if the newly acquired multimodal scheduling lag deviation observation value is still greater than 0, a first-order scheduling lag control alarm is sent. Otherwise, the corresponding security multimodal data is transmitted to the cloud center.

8. The multimodal computing power optimization method for security monitoring scenarios according to claim 6, characterized in that, The specific process of the second-level scheduling lag control is as follows: Input the second-level scheduling lag control value and the memory of the security multimodal data into the pre-built cache clearing frequency related table, and read the predefined cache clearing frequency adjustment value; send a prompt to the preset personnel, and adjust the cache clearing frequency to the predefined cache clearing frequency adjustment value based on the initial cache clearing frequency of the security multimodal data; obtain the multimodal scheduling lag deviation observation value after the cache clearing frequency is adjusted to the predefined cache clearing frequency adjustment value. If the multimodal scheduling lag deviation observation value is still greater than 0, send a second-level scheduling lag control alarm prompt; otherwise, transmit the corresponding security multimodal data to the cloud center.

9. A multimodal computing power optimization system for security monitoring scenarios, employing the multimodal computing power optimization method for security monitoring scenarios as described in any one of claims 1-8, characterized in that, include: The module includes a computing power allocation lag determination module, a collaborative scheduling lag monitoring module, and a scheduling lag level classification module. The computing power allocation lag determination module is used to filter security multimodal data for a specified security scenario to remove invalid security multimodal data. After the filtering process is qualified, it performs a lag analysis of computing power allocation, obtains the time taken from the start of computing power resource configuration to the completion of parameter initialization of security multimodal data, and marks it as a computing power allocation lag observation value. It also obtains the difference between the computing power allocation lag observation value and a predefined computing power allocation lag observation value and marks it as an allocation lag deviation observation value. This is used to quantify the lag deviation during computing power allocation. Based on the analysis results, it determines whether to activate the computing power allocation lag optimization mechanism to improve performance. The switching frequency of high multimodal computing power allocation tasks; the collaborative scheduling lag monitoring module is used to perform collaborative scheduling lag degree analysis to quantify the lag situation when computing power is collaboratively scheduled based on security multimodal data if it is not started, otherwise it will perform collaborative scheduling lag degree analysis after the computing power allocation lag optimization mechanism ends; the scheduling lag level classification module is used to determine whether to start the modal scheduling lag level classification mechanism to classify the computing power scheduling lag degree level of security multimodal data based on the execution results corresponding to the collaborative scheduling lag degree analysis, so as to transmit the corresponding security multimodal data to the cloud center.

Images