Intelligent video plan generation method and system based on multi-dimensional data fusion

Abstract

The invention relates to the field of intelligent monitoring, in particular to an intelligent video recording plan generation method and system based on multi-dimensional data fusion, and the method comprises the steps: obtaining historical alarm data of equipment from an Internet of Things platform, obtaining video point position viewing behavior data of a user from a video monitoring platform, obtaining pre-configured spatial association relationship data between the equipment and the video point location; calculating a risk probability that the specified equipment gives an alarm in a future set time period; calculating a user attention score of each video point location in a future set time period, and calculating an alarm association importance score of each video point location based on historical alarm data and the spatial association relationship; integrating the risk probability of the alarm, the attention score and the alarm association importance score, and generating a video recording value score of each video point location in each time period; dynamically generating an intelligent video recording plan by adopting a greedy algorithm based on the video recording value score and the system resource constraint; and the intelligent level of the whole monitoring system is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent monitoring, in particular to an intelligent recording plan generation method and system based on multi-dimensional data fusion. BACKGROUND

[0002] The integration of the Internet of Things and video monitoring technology has become a core trend in the development of industrial intelligence. The Internet of Things platform realizes real-time collection and monitoring alarm of production data by accessing sensors, PLCs and other industrial equipment. In typical industrial scenarios such as smart water and smart mines, video monitoring as a key regulatory means, the scientific nature of its recording strategy directly determines the level of operational efficiency, safety compliance and resource cost control.

[0003] Traditional recording schemes have limitations when dealing with complex and variable industrial environments: Current video monitoring platforms mostly use the mainstream architecture of "front-end storage + cloud-side review". This architecture deploys video recorders or smart cameras with storage functions at the front end, and the cloud platform manages them through a video gateway, retrieving front-end recordings as needed to optimize bandwidth usage. Although this architecture can to some extent distribute the storage pressure of the central platform, its reliability is highly dependent on the stability of the front-end devices. The complex industrial site environment (high temperature, high humidity, dust, vibration) easily leads to device failure, storage medium damage or network interruption. In addition, the variety of brands and models of connected devices, and the uneven local recording capabilities of some devices, make this scheme inherently unstable.

[0004] Recording all monitoring points 7x24 hours without interruption can theoretically achieve event backtracking without dead angles, but actual deployment faces great challenges. The massive amount of video data causes network bandwidth congestion and storage resource explosion, resulting in high operating costs, and most of the recorded content is invalid data, with very low resource utilization.

[0005] Recording is triggered by Internet of Things alarms, achieving on-demand recording and effectively reducing redundant recordings. However, it has inherent lag: recording starts after the alarm is triggered, inevitably missing the video recording of the key process before the alarm occurs (such as device abnormal vibration, parameter gradual change, etc.). In industrial scenarios, the state characteristics before a fault occurs are crucial for root cause analysis (for example, abnormal vibration of a water pump for several minutes before it fails). Missing the pre-event footage not only reduces troubleshooting efficiency, but also requires reverse retrieval of possibly ineffective front-end device recordings, further increasing operational complexity. SUMMARY

[0006] To solve the above problems, the application provides a kind of intelligent video recording plan generation method and system based on multi-dimensional data fusion, by fusing Internet of Things alarm data, user behavior data and device-video space correlation, algorithm model is constructed, realize the change from passive response to active prediction of video recording plan.

[0007] In the first aspect, the technical scheme of the application provides an intelligent video recording plan generation method based on multi-dimensional data fusion, comprising the following steps: S1, collect historical data, and obtain the spatial correlation data of pre-configured devices and video points;The historical data includes the historical alarm data of the device obtained from the Internet of Things platform, and the viewing behavior data of the user to the video point obtained from the video monitoring platform; S2, based on the historical alarm data, the risk probability of alarm of the specified device in the future set time period is calculated by the alarm risk prediction module; S3, based on the viewing behavior data of the user, the user attention score of each video point in the future set time period is calculated by the video point value evaluation module, and the alarm correlation importance score of each video point is calculated based on the historical alarm data and the spatial correlation; S4, the recording value score of each video point in each time period is generated by comprehensively considering the risk probability of alarm, the attention score and the alarm correlation importance score;Based on the recording value score and system resource constraint, the intelligent recording plan is dynamically generated by using the greedy algorithm;The plan includes recording point, recording time period, recording definition and pre-recording time length parameters; S5, the intelligent recording plan is sent to the video monitoring system for execution, and when the system receives the real-time alarm event sent by the Internet of Things platform, an emergency recording task with high priority is generated;According to the task priority and the current resource constraint, the recording task in execution is dynamically adjusted, and the emergency recording task is started immediately.

[0008] The dynamic adjustment of the recording task in execution includes: If system resources are sufficient, intelligent recording plan tasks and emergency recording tasks are executed in parallel; If system resources are insufficient, the task with the lowest recording value score in the intelligent recording plan is suspended, and the resource is released to ensure the execution of the emergency recording task.

[0009] As a further limitation of the technical scheme of the application, in S2, the processing logic of the alarm risk prediction module includes: S21, the historical alarm data of the specified device is discretized by hour granularity to form a binary time sequence X, wherein if there is an alarm in a certain hour, the sequence value corresponding to the hour is 1, otherwise it is 0; S22, calculate the ACF value of the time sequence X, i.e. the autocorrelation function ρ k identify the significant period T existing in the sequence; the significant period T is determined by judging whether the ACF value at lag k order exceeds the significance threshold value or not; autocorrelation function

[0010] , ,

[0011] In the formula, indicates that the alarm condition occurs at the first hour, indicates that the alarm condition occurs at the first hour, indicates that the alarm condition occurs at the first hour, n is the length of the time sequence X; S23, Fourier transform the time sequence X to obtain the frequency domain representation , and calculate the normalized amplitude of the frequency component corresponding to the significant period T as the period intensity of the significant period T; the calculation formula of the period intensity S(T) is:

[0012] wherein, the amplitude indicates the intensity of the period component corresponding to the frequency in the sequence; S24, within the significant period T, the historical alarm frequency of each hour segment is counted, and the continuous time segment with the highest historical alarm frequency is marked as a high-risk interval; for a certain set time period in the future, if the time period is located in the high-risk interval, the risk probability value of the time period is the period intensity; if the time period is not located in any high-risk interval, the risk probability value of the time period is 0, and the risk probability, i.e. the periodic risk score, is output.

[0013] As a further limitation of the technical scheme of the application, in S3, the processing logic of the video point value evaluation module comprises: S31, for each video point, the viewing frequency, the average viewing time, and the preset specific time period viewing frequency are extracted from the historical user viewing behavior data of the video point; S32, the extracted viewing frequency, average viewing time, and specific time period viewing frequency are input into the user attention evaluation model to obtain an attention score; The user attention evaluation model is as follows: Attention score = w1 x viewing frequency + w2 x average viewing time + w3 x specific time period viewing frequency Wherein, w1, w2, w3 are weight coefficients determined by training in advance, and w1+ w2+ w3= 1.​

[0014] As a further limitation of the technical solution of the application, the processing logic of the video point value evaluation module further comprises: S33, for each video point, based on the spatial correlation, obtain all Internet of Things devices associated therewith; count each alarm occurring in a historical time window of the Internet of Things device; according to the alarm level and alarm type of each alarm, set the corresponding weight, and calculate the weighted sum of all alarms to obtain the weighted alarm number of the video point; S34, normalize the calculated weighted alarm number of all video points so that the value falls within the interval of 0 to 1, and obtain the alarm correlation importance score of each video point; .

[0015] As a further limitation of the technical solution of the application, the step S4 comprises: The risk probability, attention score and alarm correlation importance score of the alarm are input into the intelligent recording decision module to generate a recording value score of each video point in each time period; and based on the recording value score and system resource constraints, a greedy algorithm is used to dynamically generate an intelligent recording plan.

[0016] As a further limitation of the technical solution of the application, the intelligent recording decision module calculates the recording value score by the following formula: Recording value score(i, t) = α × risk probability(i, t) + β × attention score(i, t) + γ × alarm correlation importance score(i) Wherein, i represents a video point, t represents time, and α, β, γ are weight coefficients determined by historical data training.

[0017] As a further limitation of the technical solution of the application, the intelligent recording decision module further sets a recording value score threshold; based on the comparison result of the recording value score and the threshold, different levels of recording strategies are triggered; the levels of the recording strategies at least include: High value level: recording value score > T_high, triggering high-definition recording and starting pre-recording and delay recording; Medium value level: T_medium < recording value score ≤ T_high, triggering standard-definition recording and starting pre-recording and delay recording; Low value level: T_base < recording value score ≤ T_medium, triggering standard-definition recording or key frame snapshot.

[0018] As a further limitation of the technical solution of the application, the step of the intelligent recording decision module using a greedy algorithm to dynamically generate an intelligent recording plan comprises: S421. Sort all candidate recording tasks in descending order according to their recording value score to form a task queue; the recording task is defined as recording a specific video location within a set time period. S422. Starting from the top of the sorted task queue, select the recording tasks in sequence; S423. Determine whether the total resource consumption of the currently selected tasks does not exceed the system resource constraint; the system resource constraint is the maximum number of concurrent recording channels; (e.g., a maximum of 20 channels for 1080P format and a maximum of 30 channels for 720P format, which can be adjusted according to the system configuration). S424. If the time limit is not exceeded, the current task will be included in the final smart recording plan; if the time limit is exceeded, the task will be skipped and the next task in the queue will be checked. S425. Repeat sub-steps S422 to S424 until the entire task queue has been traversed.

[0019] The method also includes a candidate task supplementation mechanism: For recording tasks whose video value score exceeds the candidate threshold but are not included in the plan, they are listed as candidate tasks; when the system releases resources due to the completion of the task, it automatically selects the task with the highest score from the candidate tasks to supplement the execution queue.

[0020] As a further limitation of the technical solution of the present invention, step S5 is followed by: S6. Collect newly generated alarm data and user viewing behavior data of video points after the execution plan, and use them as new training samples to retrain and optimize the parameters of the alarm risk prediction module, video point value assessment module and intelligent recording decision module, so as to achieve adaptive iteration of the system.

[0021] Secondly, the present invention also provides an intelligent video recording plan generation system based on multi-dimensional data fusion, comprising: The data acquisition module is used to collect historical alarm data and user viewing behavior data from the IoT platform and video surveillance platform, and to obtain pre-configured spatial correlation data between devices and video points; The alarm risk prediction module is used to calculate the probability of a specified device triggering an alarm within a set time period in the future, based on the historical alarm data. The video location value assessment module is used to calculate the user attention score of each video location based on user viewing behavior data, and to calculate the alarm association importance score of each video location based on the historical alarm data and spatial correlation. The intelligent recording decision module is used to generate a recording value score by comprehensively considering the risk probability, attention score and alarm correlation importance score, and dynamically generate an intelligent recording plan based on the score and system resource constraints using a greedy algorithm. The plan execution and response module is used to send the intelligent recording plan to the video surveillance system for execution, and generate a high-priority emergency recording task when a real-time alarm event is received, dynamically adjust the tasks in execution and start the emergency task. The closed-loop optimization module is used to collect newly generated alarm data and user behavior data, and to retrain and optimize the parameters of the alarm risk prediction module, video location value assessment module and intelligent recording decision module.

[0022] As a further limitation of the technical solution of the present invention, the alarm risk prediction module is configured to perform the following operations: Discretize the historical alarm data of the specified device into a binary time sequence with hourly granularity; Calculate the autocorrelation function (ACF) value of the time series to identify significant periods; Perform a Fourier transform on the time series and calculate the normalized periodic intensity of significant periods; High-risk zones are marked based on historical alarm frequencies, and risk probability values ​​for future time periods are output.

[0023] As a further limitation of the technical solution of the present invention, the video location value assessment module includes: The user attention evaluation submodule is used to calculate the attention score based on the weighted sum of viewing frequency, average viewing duration, and viewing frequency in a specific time period. The alarm association assessment submodule is used to calculate the alarm association importance score based on the weighted sum of the alarm level weight and type weight of the associated devices.

[0024] As a further limitation of the technical solution of the present invention, the intelligent video recording decision module is configured as follows: The video recording value score is calculated using a linear weighted formula. Set a threshold for recording value scoring and trigger different recording strategies based on the scoring level; A greedy algorithm is used to generate the optimal recording plan under system resource constraints.

[0025] As a further limitation of the technical solution of the present invention, the system adopts a microservice architecture, in which each module is encapsulated as an independent microservice that is called via API and deployed on the cloud or edge computing nodes.

[0026] As can be seen from the above technical solutions, this application has the following advantages: By integrating IoT alarm data and video user behavior data, it achieves a fundamental transformation from indiscriminate full recording and delayed alarm recording to value recording based on multi-dimensional data prediction. This directly overcomes the three core problems of existing methods: resource waste, omission of key information, and delayed response.

[0027] By predicting high-risk periods and high-value locations using algorithms, high-quality recording is initiated only when necessary, and the pre-recording duration is dynamically allocated. This significantly reduces the cost of network bandwidth and storage resources, while ensuring complete recording of the events before and after critical events, thus maximizing the value of the recorded data.

[0028] Upgrading the decision-making process from manual pre-setting or simple rules to a data-driven automated model reduces reliance on human experience, enables the system to adapt to complex industrial environment changes, and improves the intelligence level and scientific nature of the entire monitoring system.

[0029] The intelligent decision-making of this method is completed in the cloud, which overcomes the shortcomings of existing front-end storage solutions, such as poor reliability and insufficient stability caused by harsh industrial environments and heterogeneous equipment, and forms a more stable and reliable centralized intelligent management capability. Attached Figure Description

[0030] To more clearly illustrate the technical solution of this application, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a flowchart illustrating the method provided in an embodiment of the present invention.

[0032] Figure 2 A block diagram of a system provided in an embodiment of the present invention.

[0033] Figure 3 This is a system architecture diagram provided for an embodiment of the present invention.

[0034] Figure 4 The flowchart of the algorithm layer is provided for an embodiment of the present invention. Detailed Implementation

[0035] To make the purpose, features, and advantages of this application more apparent and understandable, specific embodiments and accompanying drawings will be used to clearly and completely describe the technical solution protected by this application. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0036] Unless otherwise defined, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in this application and in the specification of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0037] The core objective of the intelligent video recording plan is to solve the problems of resource waste and missing key information in traditional video surveillance systems, including the following aspects: (1) Maximizing the value of recording, that is, ensuring that the recorded video content can serve the greatest extent possible for post-event problem tracing, on-site situation analysis and operational efficiency improvement. This requires the system to be able to accurately identify and record video surveillance that is highly related to equipment abnormalities, production accidents or safety incidents, rather than recording all video streams indiscriminately; (2) Minimizing resource consumption, including network bandwidth, server storage space, etc. By recording on demand and intelligent pre-recording, the system avoids continuous recording of monitoring areas that have no abnormalities or operations for a long time without any distinction, thereby reducing the operating costs of the system; (3) Improving the automation and intelligence level of the system and reducing the reliance on human subjective judgment and manual operation. By automatically learning and optimizing recording strategies through algorithms, the system can adapt to dynamically changing business scenarios, from passive response to proactive prediction, providing strong support for safe production and refined management in industries such as water and mining. Specifically, such as Figure 1 As shown, this embodiment of the invention provides an intelligent video recording plan generation method based on multi-dimensional data fusion, including the following steps: S1. Collect historical data and obtain pre-configured spatial correlation data between devices and video points; the historical data includes historical alarm data of devices obtained from the Internet of Things platform and user viewing behavior data of video points obtained from the video surveillance platform. The algorithm design of this application's intelligent video recording plan relies on three key data inputs: historical alarm data, the correlation between devices and video locations, and user viewing behavior data. These three data types together form the basis of the algorithm's decision-making. The data types are shown in Table 1.

[0038] Table 1: Data Input Types

[0039] Among them, (1) historical alarm data: is the core data source for the algorithm to perform pattern learning and risk prediction. The data content should include the timestamp of the alarm occurrence, device ID, alarm type, alarm level, alarm content description, etc. By performing statistical analysis (such as calculating the alarm frequency of each device in different time periods) and pattern mining (such as using periodic detection algorithms to find the high-incidence period of alarms) on historical alarm data, important basis can be provided for predicting future alarm risks.

[0040] (2) The association between devices and video monitoring points: This is the spatial basis for realizing alarm and video linkage. This data defines which video monitoring points should be retrieved when an IoT device triggers an alarm. This association is usually established based on physical spatial location. For example, a water plant's dosing pump (IoT device) is associated with two cameras (video monitoring points) in its pump room. This association needs to be pre-configured in the system and serves as an important input for algorithm decision-making. When the algorithm predicts that a device has a high probability of triggering an alarm, it will prioritize recommending recording video monitoring points associated with that device.

[0041] (3) User viewing behavior patterns: This is an important indicator for measuring the importance of video points. It reflects which video frames receive the most attention during actual operation and maintenance. The data should include the ID of the real-time monitoring point accessed by the user, the viewing time period, the viewing duration, and the viewing frequency. By analyzing user behavior data, key monitoring areas and critical time periods in the system can be identified. For example, if the data shows that maintenance personnel frequently view the monitoring video of a key production process between 9:00 AM and 11:00 AM on weekdays, the algorithm can infer that the point has high monitoring value during that time period, and thus tend to formulate a recording plan for it during that time period, even if there are no related equipment alarms recently. This user behavior-based feedback mechanism enables the recording plan to better fit the actual business needs and avoids the deviation that may be caused by simply relying on alarm data. In addition, the ID of the video recording point accessed by the user, the recording information, the viewing frequency, and other data can also be used as the basis for recording effectiveness, for feedback optimization, and for formulating recording clearing strategies.

[0042] In this embodiment of the invention, the core modules of the intelligent recording plan algorithm design include an alarm risk prediction module, a video location value assessment module, and an intelligent recording decision module; S2. Based on the historical alarm data, calculate the probability of a specified device triggering an alarm within a future set time period using the alarm risk prediction module. The core task of the alarm risk prediction module is to predict the probability of a device triggering an alarm at a future point in time based on historical alarm data. The output of this module is one of the key inputs for calculating the video recording value score. The alarm risk prediction module employs a periodic detection model based on time series analysis. Many industrial devices' abnormal alarms are not entirely random but exhibit certain periodic patterns. For example, some devices may frequently alarm during peak hours on weekdays due to excessive load, or malfunction during specific seasons due to changes in environmental temperature and humidity. Time series analysis can uncover these potential alarm patterns, thus providing forward-looking guidance for video recording planning.

[0043] The periodicity detection algorithm based on time series analysis employs the autocorrelation function (ACF) and Fourier transform. The autocorrelation function measures the correlation between a time series and itself at different time lags. By calculating the ACF, significant periods in alarm data can be identified. For example, if alarm data shows a significant positive correlation peak at every 24-hour (day) lag point, it indicates that the alarms exhibit diurnal periodicity. The Fourier transform converts the time series from the time domain to the frequency domain, identifying the periodic components that constitute the series.

[0044] The inputs and outputs of the alarm risk prediction module are as follows: Input: Historical alarm data, time characteristics (hour / weekday / holiday) Output: The risk probability (0-1, the higher the value, the higher the risk) of each device in future time periods (every hourly granularity).

[0045] The processing logic of the alarm risk prediction module includes: S21. Discretize the historical alarm data of the specified device by hourly granularity to form a binary time sequence X. If an alarm occurs within a certain hour, the sequence value corresponding to that hour is 1; otherwise, it is 0. S22. Calculate the ACF value of the time series X, i.e., the autocorrelation function. ρ ( k Identify significant periods T in the sequence (e.g., a peak at a 24-hour lag means that the alarm behavior at 09:00 today is very similar to that at 09:00 yesterday and the day before yesterday; this is the daily cycle). The ACF is calculated as follows: (1) Using the historical alarm time series of the equipment as input, the historical alarm data of the equipment (alarm records, timestamps) is discretized at the hourly granularity to form a time series X=[x1,x2,...,x n ], where x t =1 indicates that the alarm occurred in hour t, x t =0 indicates that no alarm occurred (t is the hour number, such as t=1 corresponding to 00:00-01:00 on the first day, t=24 corresponding to 23:00-24:00 on the first day), taking data from the past 3 months (approximately 2160 hours, i.e., n=2160). (2) Calculate the mean of the sequence: Calculate the mean of the above time series. The mean reflects the average level of alarm occurrences;

[0046] (3) Calculate the autocovariance and variance of lag k, where k is the lag, i.e., the time interval in hours. For example, k=24 means an interval of 1 day. For each possible lag k, calculate its autocovariance and variance: Autocovariance:

[0047] variance:

[0048] Among them, the autocovariance measures the correlation between the sequence and its own volatility at a lag of k, and the variance is the autocovariance when the lag is 0.

[0049] (4) Calculate the ACF value, i.e., the autocorrelation coefficient of lag k, and the autocorrelation function.

[0050] In the formula, Indicates the first An alarm occurred within hours. Indicates the first An alarm is triggered every hour, where n is the length of the time series X. ρ ( k The value range of ) is [-1, 1]. The larger the absolute value, the stronger the autocorrelation under the lag, which is used to identify significant periods.

[0051] (5) Determine significant periods examine The peak value, is there? (Combined with significance test: 95% confidence interval, threshold value is...) ); If so, then the lag k is considered to have significant autocorrelation, and the corresponding period may be k hours. For example, if a significant peak appears at k=24, it indicates that there is a significant 24-hour (day) period.

[0052] S23. Perform a Fourier transform on the time series X to obtain its frequency domain representation. And calculate the normalized amplitude of the frequency component corresponding to the significant period T, as the period intensity of the significant period T (the period intensity is the normalized amplitude in the spectrum; the larger the amplitude, the more obvious the period corresponding to this frequency). The periodic intensity is determined by the following steps: (1) Apply Fourier transform to the alarm sequence to convert it to the frequency domain. Perform Fast Fourier Transform on the preprocessed time sequence X to obtain the frequency domain representation. ,in Frequency (unit: cycles / hour) represents the number of cyclical fluctuations per hour, with the cycle being... ,like Corresponding period Hour.

[0053] (2) Calculate the spectral amplitude and Fourier transform results. It is a complex number, and its amplitude is... This indicates the intensity of the periodic component corresponding to that frequency in the sequence. The larger the amplitude, the more significant the influence of that period on the sequence.

[0054] (3) Normalization processing: Normalize the amplitude of all frequencies so that its value falls between [0,1] to obtain the period intensity. ;

[0055] For example, by calculation, the amplitude of the daily cycle (24 hours) may be 0.8 and the amplitude of the weekly cycle (168 hours) may be 0.3. Then the intensity of the daily cycle is 0.8 and the intensity of the weekly cycle is 0.3, and the daily cycle is more significant.

[0056] (4) Extract the periodic intensity and find the frequency corresponding to the significant period (e.g., 24 hours) identified by ACF. (For example =1 / 24), and its normalized amplitude is the periodic intensity.

[0057] S24. Within the significant period T, the historical alarm frequency of each hour segment is statistically analyzed, and the continuous period with the highest historical alarm frequency is marked as a high-risk interval. For a future set time period, if the time period is located within the high-risk interval, its risk probability value is the period intensity. If the time period is not located within any high-risk interval, its risk probability value is 0, and the risk probability is output.

[0058] High-risk periods refer to time periods within a significant cycle (such as a daily cycle) where the frequency of alarms is significantly higher than the average level. The steps for defining these periods are as follows: (1) Lock in significant cycles.

[0059] By combining the results of ACF and Fourier transform, the main significant periods are determined. For example, if the normalized amplitude of the Fourier transform exceeds the threshold of 0.5, and ACF shows that k=24 is significant, and the Fourier transform shows that the 24-hour period has the highest intensity, then the daily period is identified.

[0060] (2) Alarm frequency for each time period within the statistical period.

[0061] For significant periods (e.g., 24 hours), divide the time periods by hour (0-1 AM, 1-2 AM, ..., 11 PM-1 AM), and calculate the historical alarm frequency for each time period:

[0062] (3) Define the high-risk zone.

[0063] Select the consecutive time periods with the highest probability of alarm occurrence, and take the top 20% of hourly intervals; this can be further refined by combining time characteristics (holidays, weekday peak hours, weekend off-peak hours), marking these time periods as high-risk intervals. .

[0064] Calculate the periodic risk score: For a future hour t, for example, 10:00 tomorrow, if t falls within a certain high-risk range... Then the risk score ,otherwise The maximum value is taken for high-risk periods that belong to multiple cycles (such as daily cycles and weekly cycles).

[0065] use The periodic risk score represents the probability of risk for each device in each future hour.

[0066] S3. Based on user viewing behavior data, calculate the user attention score of each video point within a future set time period through the video point value assessment module, and calculate the alarm association importance score of each video point based on the historical alarm data and the spatial correlation. The video location value assessment module is responsible for quantifying the importance of each video location within a set timeframe. This importance depends not only on the alarm risk of its associated devices but also on user attention and the severity of historical alarms. This module calculates an attention score based on user behavior and an importance score based on alarm association.

[0067] User behavior can be seen as a key indicator of the business value or importance of video surveillance locations. A frequently viewed video surveillance location usually indicates that the monitored area is a core business area, a high-risk area, or an area with frequent recent issues—a key area of ​​focus. Therefore, by analyzing users' historical viewing records, a weighted statistical model can be used to calculate a focus score for each video surveillance location. User behavior data includes: Viewing frequency: Statistics on the number of times each video location was viewed by different users within a set time period (the more views, the greater the weight). Average viewing time: The average time users spend viewing each video location (a longer viewing time may indicate that the situation in that area is complex and requires a longer observation period, so it can be given a higher weight). View time period distribution: Analyze the time distribution of user viewing behavior (if a certain location is frequently viewed during specific time periods, such as night shift or shift handover time, it indicates that the time period is crucial to that location). The processing logic of the video location value assessment module includes: S31. For each video location, extract the viewing frequency, average viewing duration, and preset viewing frequency for specific time periods from its historical user viewing behavior data. S32. Input the extracted viewing frequency, average viewing duration, and viewing frequency in a specific time period into the user attention evaluation model to obtain the attention score; The user attention evaluation model is as follows: Attention Score = w1 × Viewing Frequency + w2 × Average Viewing Duration + w3 × Viewing Frequency in a Specific Time Period Where w1, w2, and w3 are weight coefficients determined in advance through training, and w1 + w2 + w3 = 1.

[0068] The importance of video feed points is also reflected in their correlation with critical equipment. If a video feed point is associated with equipment that frequently generates high-level alarms, then the importance of that feed point is naturally higher. Therefore, based on equipment alarm data, a weighted statistical model can be used to calculate the alarm correlation importance score for each video feed point. The alarm data includes: Associated Alarm Frequency: This counts the total number of alarms generated by all devices associated with the video location over a given period. The more frequent the alarms, the higher the score.

[0069] Associating Alarm Levels: Different levels of alarms should be assigned different weights. For example, a critical alarm can have a much higher weight than a minor alarm.

[0070] Associated alarm types: Certain types of alarms (such as pressure over-limit and temperature over-limit) may have higher urgency and importance, and can be assigned higher weights.

[0071] The processing logic of the video location value assessment module also includes: S33. For each video point, based on the spatial association, obtain all IoT devices associated with it; count each alarm that occurred by the IoT device within the historical time window; set corresponding weights according to the alarm level and alarm type of each alarm, and calculate the weighted sum of all alarms to obtain the weighted alarm count for that video point. S34. Normalize the weighted alarm counts of all video points so that the values ​​fall within the range of 0 to 1, and obtain the alarm correlation importance score of each video point. .

[0072] Here, This represents the weighted number of alerts. A 30-day time window is used, and normalization is applied to obtain the alert correlation importance score.

[0073] S4. Based on the combined risk probability of the alarm, the attention score, and the alarm correlation importance score, generate a recording value score for each video location in each time period; based on the recording value score and system resource constraints, dynamically generate an intelligent recording plan using a greedy algorithm; the intelligent recording plan includes recording location, recording time period, recording clarity, and pre-recording duration parameters (the pre-recording duration is determined according to the alarm risk probability; 10 minutes of pre-recording when the risk probability > 0.7, 5 minutes of pre-recording when 0.5 < risk probability ≤ 0.7, and no pre-recording when 0.3 < risk probability ≤ 0.5). In this step, the risk probability, attention score, and alarm correlation importance score of the alarm are input into the intelligent recording decision module to generate a recording value score for each video location in each time period.

[0074] The intelligent recording decision module is the final execution stage of the algorithm design. It is responsible for integrating the outputs from the alarm risk prediction module and the video location value assessment module. Based on a comprehensive score, it decides when and which video locations to record, makes the optimal choice, and generates the final intelligent recording plan.

[0075] The intelligent recording decision module takes as input the outputs of the aforementioned modules: the risk probability from the alarm risk prediction module, the attention score and alarm correlation importance score from the video location value assessment module, and system resource constraints (maximum number of concurrent recording channels). These are then comprehensively scored and thresholded to form a single recording value score.

[0076] The intelligent video recording decision module calculates the video recording value score using the following formula: Video recording value score (i,t) = α × risk probability (i,t) + β × attention score (i,t) + γ × alarm correlation importance score (i) Where i represents the video location, t represents time, and α, β, γ are weight coefficients determined through training with historical data.

[0077] Here, the risk probability (i,t) is the output of the alarm risk prediction module, representing the probability that a device associated with video point i will trigger an alarm at time point t; the attention score (i,t) is calculated based on user behavior data, representing the attention level of video point i at time point t; and the alarm association importance score (i) is calculated based on historical alarm data, representing the alarm association importance score of video point i. α, β, and γ are weighting coefficients used to balance the importance of different factors. These coefficients can be trained and optimized using historical data.

[0078] After calculating the recording value score for each video location at each time point, one or more thresholds need to be set to determine whether recording should be initiated. A base threshold is set: T_base=0.3, T_medium=0.5, T_high=0.7. Based on these thresholds, corresponding recording rules are enabled, such as: High-value recording (rating > T_high): Start high definition (1080P), pre-record for 10 minutes, record for 10 minutes after alarm; Medium-value recording (T_medium < rating <= T_high): Start standard definition (720P), pre-record for 5 minutes, record for 5 minutes after alarm; Low-value recording (T_base < rating <= T_medium): Start standard definition (720P), record only after alarm or capture keyframes only.

[0079] Furthermore, after calculating the recording value score of all video locations at all time points, the intelligent recording decision module needs to select a set of recording tasks to maximize the total value while satisfying the total resource constraints. The resource allocation algorithm adopts a greedy algorithm, and its process is as follows: (a) Sort all locations and time periods in descending order of video recording value score; (b) Select tasks sequentially. If the total resource consumption (bandwidth / storage) after adding them does not exceed the constraint (maximum 20 1080P channels), then include them in the plan; (c) High-potential tasks that were not selected (score > 0.6) are listed as candidates and will be automatically added if resources for selected tasks are released.

[0080] S5. The intelligent recording plan is sent to the video surveillance system for execution. When the system receives a real-time alarm event sent by the IoT platform, a high-priority emergency recording task is generated. The recording task in progress is dynamically adjusted according to the task priority and current resource constraints, and the emergency recording task is started immediately.

[0081] The dynamically adjusted recording tasks include: If system resources are sufficient, the intelligent recording plan task and the emergency recording task will be executed in parallel. If system resources are insufficient, the task with the lowest recording value score in the intelligent recording plan will be suspended, and resources will be released to prioritize the execution of the emergency recording task.

[0082] It should be noted that the intelligent recording decision module also adopts a dynamic adjustment mechanism, which needs to be run regularly every day to generate a new recording plan. In addition, if a sudden alarm occurs (real-time alarm level ≥ severe), the associated points will be immediately included in the high-value recording and given priority in resource allocation.

[0083] The method also includes a candidate task supplementation mechanism: For recording tasks whose video value score exceeds the candidate threshold but are not included in the plan, they are listed as candidate tasks; when the system releases resources due to the completion of the task, it automatically selects the task with the highest score from the candidate tasks to supplement the execution queue.

[0084] In some embodiments, step S5 is followed by: S6. Collect newly generated alarm data and user viewing behavior data of video points after the execution plan, and use them as new training samples to retrain and optimize the parameters of the alarm risk prediction module, video point value assessment module and intelligent recording decision module, so as to achieve adaptive iteration of the system.

[0085] The training process for each module is as follows: 1. Data Acquisition and Preprocessing (1) Data collection includes: IoT data: Real-time alarm event reporting data from devices is collected via IoT platform API or message queue (MQTT); historical alarm records are collected via API or database.

[0086] Video surveillance metadata: This includes basic information about video points (such as ID, location, and model) and user operation logs (such as access records and PTZ control records). This data is typically stored in the database of the video monitoring sub-platform and can be obtained via API or direct database connection.

[0087] Data Association: During the data acquisition phase, IoT device IDs are associated with video location IDs, and IoT business areas are associated with video location IDs to prepare for subsequent linkage analysis. The association relationships are stored in the configuration database.

[0088] (2) The data preprocessing process includes: Data cleaning: handling missing values, outliers, and duplicate data. For example, removing abnormal alarms and marking missing user behavior as 0; Feature engineering: Extracting desired features from raw data; Time characteristics: Extract the hour, day of the week, and whether it is a holiday from the timestamp; Statistical characteristics: Number of alarms from the device in the past 1 / 6 / 24 hours, and frequency of user checks in the past 7 / 30 days; Data conversion: Categorical variables such as alarm type and device model are encoded using One-Hot Encoding, and alarm levels are mapped using numerical values; Data integration: Data from different sources (IoT platforms, video platforms) are aligned and integrated according to device ID and timestamp to form a unified dataset for subsequent algorithm model training.

[0089] The processed dataset is divided into a training set, a validation set, and a test set. The training set accounts for 70% and is used for model training; the validation set accounts for 15% and is used for parameter tuning during model training; and the test set accounts for 15% and is used to evaluate the final performance of the model.

[0090] 2. Model parameter initialization and baseline training include: By learning from historical data, each core module can accurately output key indicators, enabling the intelligent recording decision module to generate recording plans that meet actual business needs. Based on the algorithm module design described above, training is also conducted in modules.

[0091] I. Alarm Risk Prediction Module Training The training objective of this module is to enable the model to accurately identify the periodic patterns of device alarms and improve the accuracy of risk probability prediction. It relies primarily on historical alarm data and time characteristics.

[0092] (1) Training data preparation and feature engineering enhancement From the preprocessed historical alarm data, select valid data from the past 12 months (including complete seasonal cycles), remove false alarms (such as repeated alarms of the same type on the same device within 10 seconds) and invalid alarms (records without a clear device ID or timestamp), and finally retain device data with at least 300 alarm records (to ensure the validity of the periodic analysis).

[0093] Group by device ID, and discretize the alarm data of each device into a binary time-series sequence with hourly granularity (e.g., sequence X=[x1,x2,...,x...] for device A). 2160 ], x t =1 indicates an alarm at hour t, x t =0 indicates no alarms, and t is taken as 2160 hours in the past 3 months.

[0094] Based on the basic time features (hours, days of the week, holidays), new time period type features (such as weekday morning peak 8:00-10:00, night shift 0:00-6:00) and alarm accumulation features (number of alarms in the previous 1 hour / 6 hours / 24 hours) are added. The categorical time features are converted into numerical features through One-Hot encoding and concatenated with the time series to form training samples.

[0095] (2) Parameter initialization and baseline training a. ACF parameter initialization. Set the search range for the lag order k to 1-72 hours (covering 1 / 3 of the daily 24-hour cycle and the weekly 168-hour cycle to avoid computational redundancy), and set the initial significance test threshold to a 95% confidence interval. (n=2160, threshold approximately 0.043).

[0096] b. Fourier Transform Parameter Initialization: Fast Fourier Transform (FFT) is used, with the sampling interval of frequency ω set to 1 / 2160 (ensuring the frequency resolution can identify the periodic differences within a 1-hour interval). The amplitude normalization function is initially set to linear normalization.

[0097] c. Baseline model training: Use time-series samples from 70% of the devices as the training set, input them into the ACF model (calculation formula) to identify significant cycles (e.g., at k=24 hours, ρ(k)=0.6>0.043, which is determined to be a daily cycle), and then calculate the cycle intensity through Fourier transform (e.g., daily cycle intensity S(T)=0.8), and output the periodic risk score as the initial risk probability; use the validation set (15% of the device samples) to calculate MSE (mean squared error). If MSE>0.1 (baseline threshold), then enter the parameter tuning stage.

[0098] (3) Parameter tuning and period identification optimization a. Optimization of ACF lag order. Using a grid search method, within the range of k=1-72 hours, the ACF value and significant period are recalculated for each hour of adjustment. The optimal k is determined by minimizing the validation set MSE (e.g., for device B, the optimal k=24 hours, and the MSE decreases from 0.12 to 0.07).

[0099] b. Fourier transform frequency resolution optimization: If the period (168 hours) identification is not significant (e.g., period intensity <0.3), reduce the frequency sampling interval to 1 / 4320 (covering nearly 6 months of data) to enhance the identification ability of low-frequency periods (long periods) until the period identification accuracy on the validation set (compared with manually labeled alarm high-incidence periods) is ≥85%.

[0100] c. Optimize the threshold for high-risk intervals: Initially, high-risk intervals are defined based on the top 20% of alarm frequency periods. If the actual alarm rate during the verified high-risk period is less than 60%, the top ratio is adjusted (e.g., increased to the top 25%). Combined with time characteristics (e.g., extending the high-risk period during holidays by 2 hours), the alarm coverage of high-risk intervals is ensured to be ≥70%.

[0101] (4) Model generalization verification and solidification The trained model is applied to different equipment types (such as water pumps and dosing pumps in water plants, and conveyor belts and fans in mines). If the MSE (Generalization Threshold) of a certain type of equipment is greater than 0.08, the model is retrained using two months of historical data to ensure consistent generalization across similar equipment. When the training set MSE is less than or equal to 0.06, the validation set MSE is less than or equal to 0.08, and the periodicity recognition accuracy is greater than or equal to 88%, the ACF lag order, Fourier transform frequency parameters, and high-risk interval division rules are fixed, and the final risk probability prediction model is output.

[0102] II. Training for Video Location Value Assessment Module The training objective of this module is to ensure that the attention score and the alarm correlation importance score can accurately reflect the actual value of the location. The training process is divided into two parts: attention score training and alarm correlation score training. (1) Training of attention score based on user behavior First, training data labeling and weight initialization are performed. From user behavior data, valid viewing records from the past 30 days are filtered (excluding erroneous records, such as those with a viewing duration of <10 seconds). These records are grouped by video location ID, and the viewing frequency (e.g., location 1 was viewed 20 times in the past 30 days), average viewing duration (e.g., location 1 was viewed an average of 5 minutes), and viewing frequency during specific time periods (e.g., location 1 was viewed 8 times during the morning peak hours of 9:00-11:00) are calculated. Initial weights are set as w1=0.5 (viewing frequency), w2=0.3 (average viewing duration), and w3=0.2 (frequency during specific time periods). An initial attention score is calculated using linear weighting (e.g., location 1 score = 0.5×20 + 0.3×5 + 0.2×8 = 12.1).

[0103] Secondly, user feedback annotation and loss function definition were performed. Five senior operations and maintenance personnel were invited to manually annotate the attention scores of the validation set (15% of video points) (1-5 points, with 5 points indicating extremely high attention). The MAE (mean absolute error) of the model score and the manual annotation was calculated, and the loss function was defined as L=MAE (for example, if the model gives point 2 a score of 10 and the manual gives a score of 4, the MAE=6).

[0104] Finally, iterative optimization of the weights is performed. Gradient descent is used to minimize MAE, adjusting w1, w2, and w3 (with a total weight constraint of 1). For example, if the difference between the manually labeled data and the model score for a specific viewing frequency is large (MAE=4), then w3 is increased to 0.3, w1 is decreased to 0.4, and the score is recalculated; when the validation set MAE ≤ 0.8 (acceptable error), the weights are fixed (e.g., final weights w1=0.45, w2=0.3, w3=0.25).

[0105] (2) Training based on importance score of alarm association First, initialize alarm weights and calculate base scores. Based on business rules, initially set alarm level weights (Severe = 5, Moderate = 2, Minor = 1) and alarm type weights (Pressure Exceeds Limit = 4, Temperature Exceeds Limit = 3, Communication Interruption = 2). By video point ID, count the weighted alarm counts of associated devices over the past 30 days (e.g., if pump B associated with point 3 experiences 2 severe pressure exceedance alarms, the weighted count = 2 × 5 × 4 = 40). Normalize the counts (score = weighted count / maximum weighted count over 30 days) to obtain the initial alarm association importance score (e.g., point 3 score = 40 / 100 = 0.4).

[0106] Secondly, conduct business importance labeling and verification: the business department provides the equipment importance level (e.g., water pump B is core equipment, level 5; valve C is ordinary equipment, level 2), calculate the average importance level of the equipment associated with the video point (e.g., the average level of the equipment associated with point 3 = 5), and conduct correlation analysis between the model score and the average equipment level (expected correlation coefficient ≥ 0.7).

[0107] Finally, the weights are solidified through weight adjustment and score calibration. If the correlation coefficient is <0.7 (e.g., due to unreasonable weighting of alarm types causing deviation), the weights are adjusted (e.g., increasing the weight of high temperature from 3 to 4), and the weighted alarm count is recalculated. If the score of a certain point deviates significantly from the importance level of the equipment (e.g., the score of the point associated with core equipment = 0.3, and the score of the point associated with ordinary equipment = 0.6), a correlation confidence correction factor is introduced (the closer the distance, the larger the correction factor; the confidence of point 3 associated with water pump B = 0.9, and the corrected score = 0.4 × 0.9 = 0.36), until the correlation coefficient is ≥0.75, and the alarm weights and correction rules are solidified.

[0108] III. Training of the Intelligent Video Recording Decision Module The training objective of this module is to enable video recording value scoring to balance three major factors: alarm risk, user attention, and alarm correlation, while maximizing total value under resource constraints.

[0109] (1) Training of comprehensive scoring weights (α, β, γ) a. Training Sample Construction and Labeling. From historical video recording data, select "location-time period" recording tasks from the past 3 months (e.g., the recording task of location 1 from 9:00 to 10:00 on May 10, 2024), and extract the input features of each task: the risk probability output by the alarm risk prediction module (e.g., 0.8), the attention score output by the video location value assessment module, and the alarm association importance score; invite operation and maintenance personnel to label the validity of each recording task (1=valid, 0=invalid, valid is defined as the recording containing alarms or critical operations), forming training sample labels.

[0110] b. Weight Initialization and Loss Function Definition. Initial weights are set as α=0.4 (risk probability), β=0.3 (attention score), and γ=0.3 (alarm correlation importance score). A comprehensive score is calculated (e.g., Task 1 score = 0.4×0.8 + 0.3×12.1 + 0.3×0.4 = 4.27). A logistic regression model is used, with the comprehensive score as input and the validity label as output. The loss function is defined as cross-entropy loss.

[0111] Where y is the label and p is the effective prediction probability.

[0112] c. Weight optimization and determination of recording thresholds. A logistic regression model is trained using 70% of the samples. α, β, and γ are optimized using the F1 score (combining precision and recall) of the validation set (15% of the samples). A grid search method is used (weight range 0.1-0.6, step size 0.05) to select the weight combination with the largest F1 score (e.g., optimal weights α=0.45, β=0.25, γ=0.3). Simultaneously, recording thresholds are determined (T_base=0.3, T_medium=0.5, T_high=0.7) to ensure a task effectiveness rate ≥90% for high-value scores (>0.7) and ≥75% for medium-value scores (0.5-0.7).

[0113] (2) Verification and optimization of resource allocation algorithm a. Construct resource-constrained scenarios to simulate system resource constraints (maximum concurrent recording channels: 20 channels for 1080P and 30 channels for 720P). Randomly select 500 location-time period tasks from historical tasks (covering high, medium, and low value scores) to construct scenarios with different resource stress levels (e.g., resource utilization rates of 60%, 80%, and 100%).

[0114] b. Determine the validation metrics for the greedy algorithm. In each scenario, execute resource allocation according to the greedy algorithm process (select tasks in descending order → judge resources → include in the plan), calculate the total recording value (sum of comprehensive scores of all selected tasks), resource utilization rate (actual number of channels occupied / maximum number of channels), and high-value task coverage rate (number of selected high-value tasks / total number of high-value tasks). The expected metrics are: total recording value ≥ 90% of the theoretical maximum value (sum of the values ​​of all tasks), resource utilization rate ≥ 85%, and high-value task coverage rate ≥ 95%.

[0115] c. Algorithm optimization: If the coverage of high-value tasks is less than 95% (e.g., resources are occupied by low- and medium-value tasks), increase the priority coefficient of high-value tasks during the sorting stage (e.g., the sorting weight of tasks with a score > 0.7 is multiplied by 1.2); if the resource utilization rate is less than 85%, reduce the threshold of candidate tasks (e.g., from 0.6 to 0.55) to supplement low-value but potentially high-value tasks; when all scenarios meet the expected indicators, solidify the greedy algorithm rules.

[0116] IV. Multi-module collaborative verification and overall optimization The three modules are connected in series, and the test set data (15% of devices + 15% of video points + 15% of recording tasks) is input to simulate the actual business scenario: the alarm risk prediction module outputs the risk probability → the video point value assessment module outputs the two-dimensional score → the intelligent recording decision module generates the recording plan and calculates the overall indicators: recording effectiveness (number of effective recordings / total number of recordings) ≥ 80%, resource utilization ≥ 85%, and key alarm recording coverage (recording coverage of associated points when an alarm occurs) ≥ 98%.

[0117] If the overall performance of the test set fails to meet the standards, each module is backtracked; if the risk probability prediction of the alarm risk prediction module has a large deviation, the module is retrained; if the weights of the intelligent recording decision module are unreasonable, the weights are re-optimized; until the overall performance meets the standards.

[0118] Finally, record the final parameters of each module (e.g., ACF lag order k=24, attention score weight w1=0.45, comprehensive score weight α=0.45), training metrics (e.g., alarm risk prediction MSE=0.06, recording effectiveness rate=82%), and save the optimal parameters and weights as a configuration file. Based on the above training process, the evaluation metrics used in the module are summarized as follows: For the alarm risk prediction module, the accuracy of risk probability prediction is evaluated using indicators such as mean squared error (MSE) and mean absolute error (MAE).

[0119] For the video location value assessment module, the reasonableness of the attention score and alarm correlation importance score is evaluated by comparing them with the perceived importance of video locations in actual business.

[0120] For the intelligent recording decision-making module, indicators such as total recording value and resource utilization rate are used to evaluate the effectiveness of the recording plan and the rationality of resource allocation.

[0121] The trained modules are deployed to the actual system by encapsulating each algorithm module as a microservice (RESTful API) and deploying them on edge nodes or in the cloud. Through the microservice approach, they are connected with data acquisition modules, recording execution modules (video monitoring systems), etc., to realize the generation and execution of intelligent recording plans.

[0122] In the early stages of a project, models trained on limited historical data may not accurately reflect complex real-world situations. As the system operates, new alarm and user behavior data will continuously accumulate. It is necessary to periodically (monthly or quarterly) retrain and evaluate the modules using this new data. Furthermore, when new alarm patterns or changes in user behavior are detected, the weight coefficients in the modules should be updated promptly to ensure that system decisions align with actual business needs.

[0123] Integrating the algorithm module of this application into the IoT platform and video surveillance system requires designing a reasonable system architecture and complete API interfaces to decouple the algorithm module from the business system during system integration, so that the algorithm can be iterated and optimized independently without affecting the stability of the entire platform.

[0124] like Figure 2 As shown, this embodiment of the invention also provides an intelligent video recording plan generation system based on multi-dimensional data fusion, comprising: The data acquisition module is used to collect historical alarm data and user viewing behavior data from the IoT platform and video surveillance platform, and to obtain pre-configured spatial correlation data between devices and video points; The alarm risk prediction module is used to calculate the probability of a specified device triggering an alarm within a set time period in the future, based on the historical alarm data. The video location value assessment module is used to calculate the user attention score of each video location based on user viewing behavior data, and to calculate the alarm association importance score of each video location based on the historical alarm data and spatial correlation. The intelligent recording decision module is used to generate a recording value score by comprehensively considering the risk probability, attention score and alarm correlation importance score, and dynamically generate an intelligent recording plan based on the score and system resource constraints using a greedy algorithm. The plan execution and response module is used to send the intelligent recording plan to the video surveillance system for execution, and generate a high-priority emergency recording task when a real-time alarm event is received, dynamically adjust the tasks in execution and start the emergency task. The closed-loop optimization module is used to collect newly generated alarm data and user behavior data, and to retrain and optimize the parameters of the alarm risk prediction module, video location value assessment module and intelligent recording decision module.

[0125] In some embodiments, the alarm risk prediction module is configured to perform the following operations: Discretize the historical alarm data of the specified device into a binary time sequence with hourly granularity; Calculate the autocorrelation function (ACF) value of the time series to identify significant periods; Perform a Fourier transform on the time series and calculate the normalized periodic intensity of significant periods; High-risk zones are marked based on historical alarm frequencies, and risk probability values ​​for future time periods are output.

[0126] In some embodiments, the video location value assessment module includes: The user attention evaluation submodule is used to calculate the attention score based on the weighted sum of viewing frequency, average viewing duration, and viewing frequency in a specific time period. The alarm association assessment submodule is used to calculate the alarm association importance score based on the weighted sum of the alarm level weight and type weight of the associated devices.

[0127] In some embodiments, the intelligent video recording decision module is configured as follows: The video recording value score is calculated using a linear weighted formula. Set a threshold for recording value scoring and trigger different recording strategies based on the scoring level; A greedy algorithm is used to generate the optimal recording plan under system resource constraints.

[0128] In some embodiments, the system adopts a microservice architecture, in which each module is encapsulated as an independent microservice that is called via API and deployed on cloud or edge computing nodes.

[0129] The system architecture adopts a layered design, divided into a data layer, an algorithm layer, and an application layer, such as... Figure 3 As shown, data acquisition, algorithmic decision-making, and business applications are decoupled to improve the system's flexibility and maintainability.

[0130] The data layer forms the foundation of the entire intelligent video recording project, responsible for data storage and management, including raw data, preprocessed data, and model files. Data is typically stored in relational databases (such as MySQL). For historical data requiring large-scale offline analysis and model training, a data lake or data warehouse can be built to uniformly store and manage heterogeneous data from IoT and video platforms, providing comprehensive data support for the algorithm layer.

[0131] Algorithm Layer: The algorithm layer is the core of the intelligent video recording plan, responsible for performing key tasks such as data analysis, model training, and intelligent decision-making. It encapsulates the pre-trained alarm risk prediction module, video location value assessment module, and intelligent video recording decision-making module into independent services (via a RESTful API), providing prediction and assessment capabilities. This layer can be deployed on a separate container or server for easy expansion and maintenance. The flowchart of the algorithm layer is shown below. Figure 4 As shown.

[0132] Application Layer: The application layer is the interface between the intelligent recording plan and the user, connecting the algorithm with specific IoT and video applications. The application layer is responsible for translating the algorithm layer's decisions into concrete system operations, i.e., calling algorithm services, executing business logic such as generating recording plans and handling real-time alarms, and providing management and query functions to the user.

[0133] The interfaces that need to be designed and used in system integration include: Data query interface: Provides an interface for algorithm services to query historical and real-time data.

[0134] Model prediction interface: The main interface provided by the algorithm service, which receives input features (such as device ID, time, etc.) and returns prediction results (such as alarm probability, location value score). Video recording plan management interface: The application logic layer interacts with the video monitoring sub-platform through this interface to create, update, delete, and query video recording plans.

[0135] Alarm linkage interface: When the IoT platform generates an alarm, it notifies the application logic layer through this interface to trigger the emergency recording process.

[0136] Intelligent video recording systems place high demands on server computing, memory, and network resources. Therefore, a comprehensive system performance and resource monitoring module is required to monitor CPU and memory usage in real time, ensuring system stability. It should also monitor the network bandwidth used for video stream retrieval and uploading to ensure no impact on normal business operations. Furthermore, it needs to monitor platform storage space usage, set cleanup rules, and remove expired or worthless recording files.

[0137] To assess algorithm module performance, key metrics such as prediction accuracy and recall should be monitored. If performance degradation is detected, the module optimization process should be initiated. Furthermore, a participation mechanism should be established. Users should be allowed to label automatically generated recording plans and saved recordings within those plans as useful or useless. This labeled data can serve as labels for supervised learning and can be directly used to optimize the value assessment module.

[0138] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating intelligent video recording plans based on multi-dimensional data fusion, characterized in that, Includes the following steps: S1. Collect historical data and obtain pre-configured spatial correlation data between devices and video points; the historical data includes historical alarm data of devices obtained from the Internet of Things platform and user viewing behavior data of video points obtained from the video surveillance platform. S2. Based on historical alarm data, calculate the probability of a specified device triggering an alarm within a set future time period using the alarm risk prediction module. S3. Based on user viewing behavior data, calculate the user attention score of each video point within a future set time period through the video point value assessment module, and calculate the alarm correlation importance score of each video point based on historical alarm data and spatial correlation. S4. Based on the risk probability, attention score and alarm correlation importance score of the comprehensive alarm, a recording value score for each video point in each time period is generated. Based on video recording value scoring and system resource constraints, a greedy algorithm is used to dynamically generate an intelligent recording plan; the intelligent recording plan includes parameters such as recording location, recording time period, recording resolution, and pre-recording duration. S5. The intelligent recording plan is sent to the video surveillance system for execution. When the system receives a real-time alarm event sent by the IoT platform, a high-priority emergency recording task is generated. The recording task in progress is dynamically adjusted according to the task priority and current resource constraints, and the emergency recording task is started immediately.

2. The intelligent video recording plan generation method based on multi-dimensional data fusion according to claim 1, characterized in that, In S2, the processing logic of the alarm risk prediction module includes: S21. Discretize the historical alarm data of the specified device by hourly granularity to form a binary time sequence X. If an alarm occurs within a certain hour, the sequence value corresponding to that hour is 1; otherwise, it is 0. S22. Calculate the ACF value of the time series X, i.e., the autocorrelation function. ρ ( k The significant period T in the sequence is identified; the significant period T is determined by judging whether the ACF value exceeds the significance threshold at lag k. Autocorrelation function ， ， In the formula, Indicates the first An alarm occurred within an hour. Indicates the first An alarm occurs within an hour, where n is the length of the time series X. S23. Perform a Fourier transform on the time series X to obtain its frequency domain representation. The normalized amplitude of the frequency component corresponding to the significant period T is calculated, which is taken as the period intensity of the significant period T; the formula for calculating the period intensity S(T) is: Among them, amplitude Indicates the frequency The intensity of the corresponding periodic component in the sequence; S24. Within the significant period T, the historical alarm frequency of each hour segment is statistically analyzed, and the continuous period with the highest historical alarm frequency is marked as a high-risk interval. For a future set period of time, if the period is located within the high-risk interval, its risk probability value is the period intensity. If the period is not located within any high-risk interval, its risk probability value is 0, and the risk probability, i.e., the periodic risk score, is output.

3. The intelligent video recording plan generation method based on multi-dimensional data fusion according to claim 2, characterized in that, In S3, the processing logic of the video location value assessment module includes: S31. For each video location, extract the viewing frequency, average viewing duration, and preset viewing frequency for specific time periods from its historical user viewing behavior data. S32. Input the extracted viewing frequency, average viewing duration, and viewing frequency in a specific time period into the user attention evaluation model to obtain the attention score; The user attention evaluation model is as follows: Attention Score = w1 × Viewing Frequency + w2 × Average Viewing Duration + w3 × Viewing Frequency in a Specific Time Period Where w1, w2, and w3 are weight coefficients determined in advance through training, and w1 + w2 + w3 = 1.

4. The intelligent video recording plan generation method based on multi-dimensional data fusion according to claim 3, characterized in that, The processing logic of the video location value assessment module also includes: S33. For each video point, based on the spatial association, obtain all IoT devices associated with it; count each alarm that occurred by the IoT device within the historical time window; set corresponding weights according to the alarm level and alarm type of each alarm, and calculate the weighted sum of all alarms to obtain the weighted alarm count for that video point. S34. Normalize the weighted alarm counts of all video points so that the values ​​fall within the range of 0 to 1, and obtain the alarm correlation importance score of each video point. 。 5. The intelligent video recording plan generation method based on multi-dimensional data fusion according to claim 4, characterized in that, The steps in S4 include: The risk probability, attention score, and alarm correlation importance score of the alarm are input into the intelligent recording decision module to generate a recording value score for each video location in each time period; and based on the recording value score and system resource constraints, a greedy algorithm is used to dynamically generate an intelligent recording plan.

6. The intelligent video recording plan generation method based on multi-dimensional data fusion according to claim 5, characterized in that, The intelligent video recording decision module calculates the video recording value score using the following formula: Video recording value score (i,t) = α × risk probability (i,t) + β × attention score (i,t) + γ × alarm correlation importance score (i) Where i represents the video location, t represents time, α, β, and γ are weight coefficients determined through training with historical data, the risk probability (i,t) is the probability that a device associated with video location i will trigger an alarm at time t, output by the alarm risk prediction module; the attention score (i,t) is calculated based on user behavior data, representing the attention level of video location i at time t; and the alarm association importance score (i) is calculated based on historical alarm data, representing the alarm association importance score of video location i.

7. The intelligent video recording plan generation method based on multi-dimensional data fusion according to claim 6, characterized in that, The intelligent video recording decision module also sets a threshold for recording value scoring; Based on the comparison result between the video recording value score and the threshold, different levels of recording strategies are triggered; the levels of the recording strategies include at least: High value level: Recording value score > T_high, triggers high-definition recording and starts pre-recording and delayed recording; Medium value level: T_medium < recording value score ≤ T_high, trigger standard definition recording and start pre-recording and delayed recording; Low value level: T_base < video value score ≤ T_medium, triggering standard definition recording or keyframe capture.

8. The intelligent video recording plan generation method based on multi-dimensional data fusion according to claim 7, characterized in that, The intelligent recording decision module uses a greedy algorithm to dynamically generate an intelligent recording plan, and the steps include: S421. Sort all candidate recording tasks in descending order according to their recording value score to form a task queue; a recording task is defined as recording a specific video location within a set time period. S422. Starting from the top of the sorted task queue, select the recording tasks in sequence; S423. Determine whether the total resource consumption of the currently selected tasks does not exceed the system resource constraint; the system resource constraint is the maximum number of concurrent recording channels; S424. If the time limit is not exceeded, the current task will be included in the final smart recording plan; if the time limit is exceeded, the task will be skipped and the next task in the queue will be checked. S425. Repeat sub-steps S422 to S424 until the entire task queue has been traversed.

9. The intelligent video recording plan generation method based on multi-dimensional data fusion according to claim 1, characterized in that, Following steps S5 are: S6. Collect newly generated alarm data and user viewing behavior data of video points after the execution plan, and use them as new training samples to retrain and optimize the parameters of the alarm risk prediction module, video point value assessment module and intelligent recording decision module, so as to achieve adaptive iteration of the system.

10. An intelligent video recording plan generation system based on multi-dimensional data fusion, characterized in that, include: The data acquisition module is used to collect historical data and obtain pre-configured spatial correlation data between devices and video points; the historical data includes historical alarm data of devices obtained from the Internet of Things platform and user viewing behavior data of video points obtained from the video surveillance platform. The alarm risk prediction module is used to calculate the probability of a specified device triggering an alarm within a set time period in the future, based on the historical alarm data and using a time series analysis algorithm. The video location value assessment module is used to calculate the user attention score of each video location within a future set time period based on user viewing behavior data, and to calculate the alarm correlation importance score of each video location based on the historical alarm data and the spatial correlation relationship. The intelligent recording decision module is used to generate a recording value score for each video location in each time period by comprehensively considering the risk probability of the alarm, the attention score, and the alarm correlation importance score; based on the recording value score and system resource constraints, a greedy algorithm is used to dynamically generate an intelligent recording plan; the plan includes parameters such as recording location, recording time period, recording clarity, and pre-recording duration. The plan execution and response module is used to send the intelligent recording plan to the video surveillance system for execution. When the system receives a real-time alarm event sent by the IoT platform, it generates a high-priority emergency recording task. Based on the task priority and current resource constraints, it dynamically adjusts the recording tasks in progress and immediately starts the emergency recording task.

Images