Monitoring data processing method, device and computer storage medium

Abstract

This application discloses a monitoring data processing method, device, and computer storage medium, relating to the field of artificial intelligence technology. The method comprises: obtaining abnormal behavior detection results output by a video analysis module, wherein the abnormal behavior detection results include abnormal behavior descriptions and monitoring data corresponding to the abnormal behavior descriptions; generating a large model input instruction based on the abnormal behavior descriptions; inputting the large model input instructions and the monitoring data into the large model to obtain a review result of the abnormal behavior detection results; and issuing an alarm signal if the review result passes. This application uses a large model to review the abnormal behavior detection results of the video analysis module, thereby improving the accuracy of the early warning and ensuring that only events that truly require attention trigger early warnings.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, and computer storage medium for processing monitoring data. Background Art

[0002] In the security field, after cameras capture image data, they use artificial intelligence (AI) technology to identify abnormal behavior and then trigger an alert. However, in complex monitoring environments, AI's recognition of abnormal behavior is easily affected by occlusions or interference similar to occlusions in the image data, making it difficult to accurately identify abnormal behavior, resulting in false alerts. To avoid unnecessary panic caused by false alerts, related technologies generally send the corresponding clips to a human end after AI identifies abnormal behavior, who then manually verifies whether abnormal behavior is present. This results in a significant lag in the security system's danger warnings. Summary of the Invention

[0003] The main purpose of this application is to provide a monitoring data processing method, device and computer storage medium, aiming to solve the technical problem of insufficient warning accuracy in early warning systems based on artificial intelligence.

[0004] To achieve the above objectives, an embodiment of the present application provides a method for processing monitoring data, the method comprising:

[0005] Obtaining an abnormal behavior detection result output by the video analysis module, wherein the abnormal behavior detection result includes an abnormal behavior description and monitoring data corresponding to the abnormal behavior description;

[0006] Generate a large model input instruction based on the abnormal behavior description;

[0007] Inputting the large model input instruction and the monitoring data into the large model to obtain a review result of the abnormal behavior detection result;

[0008] If the review result is passed, an alarm signal is issued.

[0009] In one embodiment, before the step of obtaining the abnormal behavior detection result output by the video analysis module, wherein the abnormal behavior detection result includes an abnormal behavior description and monitoring data corresponding to the abnormal behavior description, the following steps are included:

[0010] Performing frame segmentation on the video file to extract video frames of the video file;

[0011] Performing target detection on the video frame, identifying the target object in the video frame, and extracting feature parameters of the target object;

[0012] Comparing the characteristic parameters with the characteristic template to determine whether the target object has abnormal behavior;

[0013] According to the comparison result, an abnormal behavior description and monitoring data corresponding to the abnormal behavior description are generated.

[0014] In one embodiment, the step of comparing the feature parameters with a feature template to determine whether the target object has abnormal behavior includes:

[0015] Defining a set of abnormal behavior characteristics of the target object;

[0016] Comparing the characteristic parameters of the target object with the characteristic template corresponding to the target object in the abnormal behavior characteristic set to determine the matching degree between the characteristic parameters and the characteristic template;

[0017] If the matching degree reaches or exceeds a preset threshold, the target object is marked as having abnormal behavior.

[0018] In one embodiment, before the step of generating a large model input instruction based on the abnormal behavior description, the following steps are included:

[0019] Identifying abnormal behavior and determining a category of the abnormal behavior based on the abnormal behavior description;

[0020] Based on the categories, determining the priority and accuracy requirements of the abnormal behavior detection results;

[0021] If the priority of the abnormal behavior detection result is high, performing a large model review on the abnormal behavior detection result;

[0022] If the priority of the abnormal behavior detection result is low and the accuracy requirement is high, performing large model and manual double verification on the abnormal behavior detection result;

[0023] If the large model is in a busy state, the abnormal behavior detection result is reviewed according to the priority sorting.

[0024] In one embodiment, the step of generating a large model input instruction based on the abnormal behavior description includes:

[0025] Extracting instruction parameters based on the abnormal behavior description, the instruction parameters including a target object, the abnormal behavior of the target object, and the time and location of occurrence of the abnormal behavior;

[0026] Selecting a corresponding instruction template from preset instruction templates according to the category of the abnormal behavior;

[0027] Fill the instruction parameters into the instruction template to generate the large model input instruction.

[0028] In one embodiment, before the step of inputting the large model input instruction and the monitoring data into the large model and obtaining the review result of the abnormal behavior detection result, the following steps are included:

[0029] Collect historical monitoring data and annotate abnormal behaviors in the historical monitoring data to form a labeled training data set;

[0030] Inputting the training data set into the large model, training the large model, and obtaining abnormal behavior prediction results of the training data set;

[0031] Comparing the abnormal behavior prediction result with the corresponding label and calculating a loss function;

[0032] Based on the loss function, the learning parameters of the large model are adjusted.

[0033] In one embodiment, the step of inputting the large model input instruction and the monitoring data into the large model and obtaining a review result of the abnormal behavior detection result includes:

[0034] Sending the large model input instruction and the monitoring data to the large model through the API interface;

[0035] receiving a review result returned by the large model, wherein the review result is used to indicate whether the abnormal behavior detection result is correct;

[0036] The review result is verified. If the verification result indicates that the review result fails, the generation parameters of the large model input instruction are adjusted according to the verification result.

[0037] In one embodiment, the step of issuing an alarm signal if the review result passes further includes:

[0038] Verifying the review result, and if the verification result indicates that the review result passes, determining a warning level based on the abnormal behavior detection result;

[0039] According to the warning level, a corresponding alarm signal is sent.

[0040] An embodiment of the present application also provides a monitoring data processing device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the monitoring data processing method described above.

[0041] An embodiment of the present application further provides a computer storage medium, which is a computer-readable storage medium. A computer program is stored on the computer storage medium. When the computer program is executed by a processor, the steps of the monitoring data processing method described above are implemented.

[0042] The present application discloses a method for processing monitoring data. The method comprises obtaining abnormal behavior detection results output by a video analysis module, wherein the abnormal behavior detection results include a description of the abnormal behavior and the monitoring data corresponding to the abnormal behavior description; generating a large model input instruction based on the abnormal behavior description; inputting the large model input instruction and the monitoring data into the large model to obtain a review result of the abnormal behavior detection result; and issuing an alarm signal if the review result passes. This application uses the large model to review the abnormal behavior detection results of the video analysis module, thereby improving the accuracy of the early warning and ensuring that only events that truly require attention trigger the early warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a flowchart of a first embodiment of a method for processing monitoring data according to an embodiment of the present application;

[0044] Figure 2 This is a flow chart of a second embodiment of a method for processing monitoring data according to an embodiment of the present application;

[0045] Figure 3 This is a flow chart of a third embodiment of a method for processing monitoring data according to an embodiment of the present application;

[0046] Figure 4 This is a flow chart of a fourth embodiment of a method for processing monitoring data according to an embodiment of the present application;

[0047] Figure 5 This is a flowchart of a fifth embodiment of a method for processing monitoring data according to an embodiment of the present application;

[0048] Figure 6 This is a flowchart of a sixth embodiment of a method for processing monitoring data according to an embodiment of the present application;

[0049] Figure 7 This is a schematic diagram of the structure of the monitoring data processing equipment for this application.

[0050] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0051] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0052] In the security field, after cameras capture image data, they use artificial intelligence (AI) technology to identify abnormal behavior and then trigger an alert. However, in complex monitoring environments, AI's recognition of abnormal behavior is easily affected by occlusions or interference similar to occlusions in the image data, making it difficult to accurately identify abnormal behavior, resulting in false alerts. To avoid unnecessary panic caused by false alerts, related technologies generally send the corresponding clips to a human end after AI identifies abnormal behavior, who then manually verifies whether abnormal behavior is present. This results in a significant lag in the security system's danger warnings.

[0053] To address the above-mentioned deficiencies in the related art, an embodiment of the present application proposes a method for processing monitoring data. The method obtains abnormal behavior detection results output by a video analysis module, wherein the abnormal behavior detection results include abnormal behavior descriptions and monitoring data corresponding to the abnormal behavior descriptions; generates a large model input instruction based on the abnormal behavior descriptions; inputs the large model input instructions and the monitoring data into the large model to obtain a review result of the abnormal behavior detection results; and issues an alarm signal if the review result passes. The present application uses a large model to review the abnormal behavior detection results of the video analysis module, thereby improving the accuracy of the early warning and ensuring that only events that truly require attention trigger early warnings.

[0054] It should be noted that the execution subject of this embodiment can be a monitoring data processing system, a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, mobile phone, etc., or a monitoring data processing device capable of performing the above functions. Below, this embodiment and the following embodiments are described using a monitoring data processing system as an example (hereinafter referred to as the "system").

[0055] The method for processing monitoring data of the first embodiment proposed in this application is described in detail in Figure 1 The method includes steps S10 to S40:

[0056] Step S10: Acquire an abnormal behavior detection result output by the video analysis module, wherein the abnormal behavior detection result includes an abnormal behavior description and monitoring data corresponding to the abnormal behavior description.

[0057] In this embodiment, the system first creates or generates a task for warning conditions in a specified area. This task is sent to the video analysis module through the API (Application Programming Interface), which is responsible for analyzing the video files of the specified area. The abnormal behavior detection result refers to the potential risk or abnormal behavior description detected by the video analysis module, such as the category (such as smoke, sparks, etc.) and location information of the target object, as well as the monitoring data corresponding to the abnormal behavior description. Monitoring data refers to visual data containing the target object. The target object refers to the object identified as abnormal behavior or abnormal event in the abnormal behavior detection result, such as a person smoking or a fireworks display.

[0058] Optionally, the system establishes a communication connection with the API provided by the video analysis module to exchange data. When the video analysis module detects abnormal behavior or an abnormal event in a received video file, it immediately pushes the abnormal behavior detection results, including a description of the abnormal behavior and corresponding monitoring data, through the API. The system monitors the API in real time and, upon receiving the abnormal behavior detection results, parses them to extract the abnormal behavior description and corresponding monitoring data.

[0059] It should be noted that video files may come from a variety of sources, including surveillance cameras, mobile devices, drone monitoring, and remote servers. These video files can be real-time video streams transmitted by surveillance cameras, video files uploaded by users via mobile devices, real-time footage from drones in mid-air, and video files stored on remote servers.

[0060] Optionally, multiple video analysis algorithms are preconfigured in the system, each targeting a specific category of target objects. The configuration information for these video analysis algorithms is sent to the video analysis module via an API interface. The module then analyzes the received video files for abnormal behavior based on the configured video analysis algorithms. When a video file enters the video analysis module, it first uses a preliminary target detection algorithm to quickly identify potential target objects within the video file. Once the target object is identified, the module selects an appropriate video analysis algorithm based on the target object's category to perform abnormal behavior analysis.

[0061] Specifically, after receiving a video file, the video analysis module performs preliminary target detection, which aims to identify various potential target objects in the video file and determine the target object's category. Next, based on the target object's category determined by the preliminary detection results, the video analysis module selects a video analysis algorithm suitable for that category from a preconfigured library of video analysis algorithms. For example, if the preliminary detection identifies the target object as "smoke," the video analysis module will invoke a video analysis algorithm specifically designed for smoke detection; if the target object is "smoking," the video analysis module will invoke a video analysis algorithm specifically designed for behavior recognition.

[0062] After selecting a video analysis algorithm, the video analysis module conducts an in-depth analysis of the target object and generates anomaly detection results containing a detailed description of the abnormal behavior and the corresponding monitoring data. These anomaly detection results include not only the target object, the description of the target object's abnormal behavior, and the corresponding monitoring data, but also the identifier of the video analysis algorithm used to analyze the corresponding video file.

[0063] It should be noted that the identifier of a video analysis algorithm is a specific code or label used to distinguish different video analysis algorithms. The identifier can be a name, a number, or a metadata structure containing version, parameter, and configuration information. A variety of video analysis algorithms are integrated into the system. Each video analysis algorithm may have different recognition accuracy, reliability, or applicability in specific scenarios for different target object categories, such as smoke detection, face recognition, or vehicle recognition. The video analysis module receives the video analysis algorithms configured by the system through the API interface and uses these video analysis algorithms to analyze video files.

[0064] In a feasible implementation manner, steps S11 to S14 may be further included before step S10:

[0065] Step S11 , performing frame segmentation on the video file to extract video frames of the video file.

[0066] It should be noted that frame segmentation is the process of extracting individual video frames from a continuous video file in chronological order. This is equivalent to breaking down a continuous dynamic image into a series of static images. Each video frame captures the visual content of the video file at a specific point in time. Video frames are the basic building blocks of video files.

[0067] Optionally, after receiving a video file, the video analysis module extracts individual video frames from the video file at a specific frame rate using a video processing library (such as OpenCV). Specifically, the video analysis module initializes the video decoder, which decodes the compressed data stream in the video file into processable video frames. Next, it sets a frame extraction interval, which determines how frequently video frames are extracted from the video file. It then loops through each frame in the video file, each read yielding a video frame containing the visual information at that specific moment in the video file. After extracting the video frames, they are stored as image data structures for processing, such as cv2.Mat objects in OpenCV.

[0068] It should be noted that the cv2.Mat object is a matrix-based data structure used to store pixel data of an image.

[0069] Step S12: performing target detection on the video frame, identifying the target object in the video frame, and extracting feature parameters of the target object.

[0070] In this embodiment, the video analysis module passes the extracted video frames as input to the configured video analysis algorithms. These video analysis algorithms can be traditional computer vision algorithms, such as the SIFT (Scale-invariant feature transform) algorithm, which can identify key points in the image and describe them, and can remain invariant even when the image size, rotation and illumination changes; or the HOG (Histogram of Oriented Gradients) combined with the SVM (Support Vector Machine) algorithm. The HOG algorithm constructs features by calculating and counting the gradient direction histograms of local areas of the image, while SVM is a machine learning algorithm for classification, which can effectively classify features.

[0071] In addition, the video analysis module may also use a deep learning model, such as the YOLO (You Only LookOnce) algorithm, which is an object detection algorithm that can simultaneously predict multiple bounding boxes and category probabilities in a single forward propagation, thereby achieving rapid identification of target objects in video frames.

[0072] During video frame processing, the video frames undergo image preprocessing, including grayscale conversion, denoising, and brightness and contrast adjustments to reduce the impact of ambient lighting variations and camera noise. After image preprocessing, feature extraction techniques such as edge detection, color analysis, and shape recognition are used to detect objects within the video frames, identifying them such as people, vehicles, and smoke.

[0073] Once the target object is detected, its feature parameters are further extracted. These include bounding box coordinates, shape, color histogram, motion trajectory, and contour changes. Furthermore, by quantifying these parameters, corresponding attribute values ​​can be calculated. Attribute values ​​can include the target object's size, speed, color, or more complex attributes such as smoke density.

[0074] For example, when analyzing the target object "smoke," the smoke concentration can be estimated by analyzing the color intensity or transparency of the smoke area, or by calculating the pixel values ​​of the area covered by the smoke. The spread of the smoke can also be determined by calculating the area of ​​the smoke's bounding box or tracking its changes over time. Furthermore, by comparing changes in the shape and position of the smoke in consecutive frames, the speed of the smoke's dynamic changes can be determined.

[0075] Step S13: Compare the feature parameters with the feature template to determine whether the target object has abnormal behavior.

[0076] Step S14: Generate an abnormal behavior description and monitoring data corresponding to the abnormal behavior description according to the comparison result.

[0077] It's important to note that a feature template is a predefined set of parameters that represent abnormal behavior or states. These parameters reflect the characteristic behavior of a target object under abnormal circumstances. Feature templates are built based on extensive historical data analysis and relevant domain knowledge, ensuring they accurately reflect the target object's behavioral patterns or characteristic patterns under abnormal circumstances. Feature templates can include information such as the target object's speed range, shape characteristics, color distribution, and motion trajectory.

[0078] During the comparison process, the degree of match between the feature parameters and the feature template is calculated. Based on the comparison results, it is determined whether the target object's behavior matches the feature template. If the feature parameters closely match the feature template, the target object is considered to have abnormal behavior; if the degree of match is low, the target object's behavior is considered normal.

[0079] The video analysis module sets a matching threshold, which defines the minimum acceptable similarity between the feature parameters and the feature template. If the calculated matching threshold exceeds this threshold, meaning the similarity between the feature parameters and the abnormal behavior feature template is high, the video analysis module will determine that the target object has abnormal behavior. This is because a high matching degree means that the target object's feature parameters are very close to the abnormal behavior feature template, indicating that the target object's behavior is similar to the predefined abnormal behavior.

[0080] On the contrary, if the matching degree is lower than this threshold, that is, the similarity between the feature parameters and the abnormal behavior feature template is low, the video analysis module will consider the behavior of the target object to be normal because it is quite different from the abnormal behavior feature template and does not meet the characteristics of abnormal behavior.

[0081] For example, if the shape features or visual features of the target object are slightly different from the shape features in the feature template, that is, the matching degree exceeds a preset threshold, the video analysis module will deem the behavior of the target object abnormal.

[0082] In a feasible implementation, step S13 may further include steps S131 to S133:

[0083] Step S131: defining a set of abnormal behavior features of the target object.

[0084] In this embodiment, an abnormal behavior feature set is a collection of multiple target objects and their characteristic parameters, which together describe the characteristic behavior of the target objects when they exhibit abnormal behavior. These feature sets are typically constructed based on the definition of the target objects' abnormal behavior and reflect the target objects' behavioral patterns or characteristic patterns under abnormal circumstances. The characteristic parameters may include the target objects' shape, color, motion trajectory, speed, etc., or they may include more complex attributes such as how the target objects interact with other objects and their behavioral patterns.

[0085] In practice, the first step is to collect and organize data on both normal and abnormal behavior related to the target object. Then, through data analysis methods, key features that distinguish normal from abnormal behavior are extracted. These key features are used to construct a feature template, which serves as the basis for determining whether the target object has exhibited abnormal behavior.

[0086] Step S132: comparing the characteristic parameters of the target object with the characteristic template corresponding to the target object in the abnormal behavior characteristic set, and determining the matching degree between the characteristic parameters and the characteristic template.

[0087] Step S133: If the matching degree reaches or exceeds a preset threshold, the target object is marked as having abnormal behavior.

[0088] After extracting video frames, the video analysis module uses feature extraction techniques to identify the target object from these frames. Once the target object is identified, the video analysis module calculates its feature parameters, such as bounding box coordinates, shape, and color histogram. These feature parameters are then compared with a predefined set of abnormal behavior signatures. If the target object's feature parameters closely match the parameters of the corresponding feature template in the abnormal behavior signature set—that is, if the match exceeds a preset threshold—the video analysis module determines that the target object has exhibited abnormal behavior.

[0089] Step S20: Generate a large model input instruction based on the abnormal behavior description.

[0090] In this embodiment, the system first extracts key information from the abnormal behavior description, such as the target object's category (e.g., person, smoke), location (e.g., coordinates and name of the monitored area), and behavior description (e.g., smoking, fire). Next, based on this key information and combined with pre-set instruction templates, the system automatically generates a large-scale model input instruction describing the target object's behavior. For example, if the target object is identified as "a person smoking," the system uses the corresponding instruction template to generate a large-scale model input instruction, such as "a person smoking in a no-smoking area has been detected."

[0091] Optionally, the system can automate the generation of large model input instructions using pre-defined instruction templates and a rules engine. The instruction templates contain standardized sentence structures for describing different categories of target objects and behaviors. The rules engine then selects and populates the corresponding instruction templates based on pre-defined rules, the target object attributes, and other parameters in the abnormal behavior description, generating specific large model input instructions.

[0092] For example, if the abnormal behavior description indicates that the detected target object is "smoke" and is located in area A, and the location and visual characteristics of the smoke are displayed in the monitoring data, the system can generate the following large model input instructions: "A high concentration of smoke is detected in area A. The smoke characteristics are bright flames accompanied by dot-shaped rising smoke, which may indicate a fire and trigger an early warning. Please check whether the early warning result is correct."

[0093] Step S30: inputting the large model input instruction and the monitoring data into the large model to obtain a review result of the abnormal behavior detection result.

[0094] In this embodiment, a large, pre-trained model is constructed. This model, equipped with data processing and pattern recognition capabilities, can be used to verify the abnormal behavior detection results output by the video analysis module. This large model is typically based on a deep learning framework and trained with a large amount of annotated data, enabling it to accurately identify and classify various abnormal behaviors.

[0095] Before using the big model for review, to accurately verify abnormal behavior detection results, a structured input format, such as a JSON object, is constructed to integrate the big model input instructions and the corresponding monitoring data. The monitoring data is input into the big model as visual data, while the big model input instructions serve as supplementary contextual information.

[0096] It's important to note that large models are typically based on deep learning frameworks, such as convolutional neural networks (CNNs) for image processing and recurrent neural networks (RNNs) or Transformer models for text processing. These models can identify the characteristics of target objects from monitoring data and, combined with contextual information in the large model's input instructions, review descriptions of abnormal behavior to determine whether they are false positives.

[0097] Specifically, the system encodes monitoring data into a format recognizable by the large model, such as a Base64-encoded string, and stores it as a field in a JSON object. Simultaneously, the large model's input instructions are embedded in the same JSON object and converted into vector representations through word embedding, allowing the large model to understand the semantic content of the text. This structured input ensures that the large model can consider both visual and textual information, resulting in a more comprehensive assessment of the accuracy of abnormal behavior descriptions.

[0098] It should be noted that Base64 is a method of representing binary data based on 64 printable characters. The purpose of Base64 encoding is to convert binary data into a text format so that it can be transmitted through text transmission protocols such as email without compromising data integrity.

[0099] After receiving the structured input, the large model extracts the characteristic data of the monitoring data and the large model input instructions. Based on this characteristic data, the large model generates a verification result, which indicates whether the abnormal behavior detection result of the input is a true warning or a false alarm.

[0100] Step S40: If the review result is passed, an alarm signal is issued.

[0101] In this embodiment, the system determines whether an alert is necessary based on the review results. If the review indicates that the abnormal behavior detected is genuine and requires an early warning, the system immediately triggers the early warning mechanism and issues an alert. This alert can be communicated to relevant personnel in a variety of ways, such as through the system interface display, SMS or email notification, or by triggering an audible or visual alarm. This allows relevant personnel to receive timely early warning of abnormal behavior and take appropriate measures.

[0102] Please refer to Figure 2 The monitoring data processing method of the second embodiment proposed in this application further includes steps S201 to S205 before step S20:

[0103] Step S201: Identify abnormal behavior and determine the category of the abnormal behavior based on the abnormal behavior description.

[0104] The system first analyzes the abnormal behavior descriptions received from the video analysis module. These descriptions typically include key information such as the target object category (such as person, vehicle, smoke), location coordinates, and behavioral characteristics. By analyzing the abnormal behavior descriptions, the system can identify specific abnormal behaviors and classify them into predefined abnormal behavior categories.

[0105] Step S202: Based on the category, determine the priority and accuracy requirements of the abnormal behavior detection result.

[0106] Step S203: If the priority of the abnormal behavior detection result is high, a large model review is performed on the abnormal behavior detection result.

[0107] Based on the identified abnormal behavior categories, the system can set priorities and accuracy requirements for the corresponding abnormal behavior detection results. Different abnormal behavior categories may correspond to different security risk levels, so the review priority should be determined based on the category. Furthermore, the accuracy requirements can also vary by category to ensure more rigorous review of critical or high-risk abnormal behaviors.

[0108] Optionally, the system can dynamically adjust the priority and accuracy requirements based on actual conditions. In resource-constrained environments, when monitoring resources are tight, the system may prioritize and process abnormal behavior without prioritizing accuracy. In such cases, the system can rely directly on the large model for review, eliminating the need for manual review and improving the efficiency of the early warning process.

[0109] Understandably, for certain specific tasks identified by the system, if there are additional accuracy requirements (i.e., high accuracy requirements), a double-check mechanism will be activated. This is done to ensure the accuracy of alerts and avoid false positives. In this case, in addition to the automatic review by the large model, a manual review process will also be implemented. For general tasks (i.e., those without additional accuracy requirements), the large model will be used for review.

[0110] The purpose of setting priorities is to sort abnormal behaviors according to their urgency and importance when reviewing large models, so as to more effectively allocate review resources and ensure that key events are handled with priority.

[0111] If the abnormal behavior detection result has a higher priority, it will be directly reviewed by the large model. This means that the system will use the trained large model to further verify the abnormal behavior detection result to improve the accuracy and reliability of the result.

[0112] Step S204: If the priority of the abnormal behavior detection result is low and the accuracy requirement is high, the abnormal behavior detection result is subjected to large model and manual double verification.

[0113] For abnormal behavior detection results with lower priority but higher accuracy requirements, the system not only uses a large model for review, but also incorporates a manual review process to ensure the accuracy of the final results. This double-check mechanism is suitable for abnormal behavior that is not the highest priority but whose misjudgment could lead to serious consequences.

[0114] Step S205: If the large model is in a busy state, the abnormal behavior detection result is reviewed according to the priority sorting.

[0115] It should be noted that a busy state means that the large model is currently performing other computing tasks or reviewing other abnormal behavior detection results, which has caused its resources (such as computing power and memory) to be occupied and unable to immediately accept and process new review requests. The system monitors the resource usage of the large model in real time and marks it as busy when it detects that its resources are fully occupied.

[0116] For example, when the large model is busy, there may be multiple abnormal behavior detection results waiting for review. The system will sort them according to their priority to ensure that high-priority abnormal behavior detection results are processed promptly. If the large model is currently unavailable, the system will queue the abnormal behavior detection results in order of priority. Once the large model resources are released, they will be processed in the queue order.

[0117] Please refer to Figure 3In the monitoring data processing method of the third embodiment proposed in this application, step S20 may further include steps S210 to S230:

[0118] Step S210: extracting instruction parameters according to the abnormal behavior description, where the instruction parameters include a target object, the abnormal behavior of the target object, and the time and location of occurrence of the abnormal behavior.

[0119] The system extracts key information from the description of abnormal behavior, which serves as the basis for generating input instructions for the larger model. These instruction parameters typically include the target object (the subject of the abnormal behavior, such as a person, vehicle, or object), the target object's abnormal behavior (specific unusual or illegal actions), and the time and location of the abnormal behavior.

[0120] For example, if the abnormal behavior description states "someone was detected smoking in a no-smoking area in area A", the system will extract the target object as "person", the abnormal behavior as "smoking", the time of occurrence as a certain time, and the location as "area A" and "no-smoking area".

[0121] Step S220: selecting a corresponding instruction template from preset instruction templates according to the category of the abnormal behavior.

[0122] In monitoring systems, multiple pre-set instruction templates are often used to guide large models in reviewing different types of abnormal behavior. These instruction templates are customized based on factors such as the nature of the abnormal behavior, its urgency, and the review requirements, ensuring the accuracy and efficiency of the review process.

[0123] It should be noted that each category of abnormal behavior is associated with one or more specific instruction templates. These instruction templates are pre-defined text structures used to guide the large model on how to understand and process input data.

[0124] Step S230: Fill the instruction parameters into the instruction template to generate the large model input instruction.

[0125] After selecting the appropriate instruction template, the system populates the variables in the instruction template with the extracted instruction parameters, generating the output instructions for the large model. This step enables the system to generate a complete, accurate, and compliant large model input instruction, guiding the large model in reviewing abnormal behavior. During the parameter filling process, the system ensures accuracy and completeness to avoid inaccurate review results due to missing or incorrect information.

[0126] For example, if the instruction template requires the input format of "It is detected that {target object} performed {abnormal behavior} at {location} at {time}, and {abnormal event} may occur, triggering an early warning. Please check whether the early warning result is correct", the system will substitute the extracted instruction parameters into the corresponding positions and generate a large model input instruction such as "It is detected that a person is smoking in area A at the current time, and a fire may occur, triggering an early warning. Please check whether the early warning result is correct."

[0127] Alternatively, the system can use statistical models, such as n-gram models, to predict and generate large-scale input instructions. Statistical models can analyze large amounts of text data, learn the statistical laws of language, and generate instruction content based on these laws.

[0128] Specifically, data preprocessing is performed on historical abnormal behavior detection results and their corresponding correct annotations, including text cleaning, word segmentation, and serialization. This data is then used to train an n-gram model to calculate the probability distribution of sequences of different language units. This allows the next language unit to be predicted based on the known context when generating input instructions for the large model.

[0129] When generating large-model input instructions, the system uses key information extracted from the abnormal behavior detection results as input and iteratively selects the words with the highest probability through the n-gram model to construct a complete sentence. For example, the system might generate the following large-model input instruction: "Smoke was detected at {specific location}, indicating a possible fire source, triggering an alert. Please check whether the alert result is correct." Here, "{specific location}" can be dynamically filled in based on the abnormal behavior detection results, while the structure and content of the entire sentence are generated by the n-gram model based on learned language patterns.

[0130] Optionally, the system can adjust the format and content of the large model input instructions by using custom instruction templates to adapt to different target object categories. This approach involves creating multiple predefined instruction templates, each designed for a specific target object category and containing corresponding descriptions and instructions. When the system receives abnormal behavior detection results and identifies a target object, it selects an appropriate instruction template based on the target object category. It then populates the analyzed instruction parameters into the instruction template to generate the final large model input instruction.

[0131] Furthermore, the system can determine the input instructions for the large model based on the identification of the video analysis algorithm, ensuring that the large model input instructions are consistent with the target object and the analysis technology used. When the system receives abnormal behavior detection results, it identifies the algorithm identification that generated these abnormal behavior detection results. This identification helps the system understand which specific video analysis algorithm was used by the video analysis module to process the video file and generate the abnormal behavior detection results. This allows the system to select the appropriate instruction template based on the characteristics of the corresponding video analysis algorithm, thereby adjusting the tone and precision of the large model input instructions to more accurately reflect the analysis results and maintain the clarity and practicality of the information.

[0132] For example, the system maintains a library of predefined instruction templates for different target object categories. For example, for smoke, a command template might include "Smoke has been detected at {specific location}, and the smoke concentration has reached {warning data}, triggering an alert. Please confirm whether the alert result is correct." This template clearly conveys the alert result, allowing the recipient to quickly understand the situation and respond. The "{specific location}" and "{warning data}" in the instruction template can be replaced based on the actual description of the abnormal behavior, ensuring the accuracy and real-time performance of the input instructions for the large model.

[0133] Please refer to Figure 4 The monitoring data processing method of the fourth embodiment proposed in this application may further include steps S301 to S304 before step S30:

[0134] Step S301: historical monitoring data is collected, and abnormal behaviors in the historical monitoring data are annotated to form a labeled training data set.

[0135] In this embodiment, historical monitoring data covers monitoring records stored in the system, and also includes abnormal behavior-related information from other channels, such as pictures, videos, dynamic images, and independent video frames. Historical monitoring data contains activities in a specific area over a period of time. The labeling process can be completed by professionals or automated labeling tools. The purpose is to identify and mark abnormal behaviors in historical monitoring data. These abnormal behaviors are given specific labels, such as "fire", "smoking", etc. The labeling information usually includes the timestamp, type, and target object involved of the abnormal behavior.

[0136] Step S302: input the training data set into the large model, train the large model, and obtain the abnormal behavior prediction result of the training data set.

[0137] During training, the large model receives a labeled training dataset containing historical monitoring data and corresponding labels for abnormal behavior. The large model processes this input data through its internal neural network structure, learning how to extract features from the historical monitoring data and predict abnormal behavior. The prediction result is the large model's verification of the labels for abnormal behavior in the training dataset or the output of a probability of abnormal behavior.

[0138] Step S303: Compare the abnormal behavior prediction result with the corresponding label and calculate the loss function.

[0139] In this embodiment, the loss function is a metric that measures the difference between the model's predictions and the actual labels. Common loss functions include cross-entropy loss and mean squared error. In this step, the system compares the model's predictions with the actual labels in the training dataset and calculates a loss value, which reflects the performance of the large model on the current training dataset.

[0140] For classification tasks, commonly used loss functions are cross entropy or binary cross entropy. For regression tasks, commonly used loss functions are mean squared error (MSE) or mean absolute deviation (MAD).

[0141] Step S304: Based on the loss function, adjust the learning parameters of the large model.

[0142] In this embodiment, to update the weights of the large model, the gradient of the loss function with respect to the model parameters is calculated. Then, using a gradient descent algorithm or other optimization algorithm, such as the Adam optimization algorithm, the weights and biases of the large model are updated to reduce prediction error. This process is repeated until the loss function of the large model falls within a preset range or the preset number of training rounds is reached, thus completing the training of the large model.

[0143] Please refer to Figure 5 In the monitoring data processing method of the fifth embodiment proposed in this application, step S30 may further include steps S310 to S330:

[0144] Step S310: Send the large model input instruction and the monitoring data to the large model through the API interface.

[0145] Step S320: receiving the review result returned by the large model, wherein the review result is used to indicate whether the abnormal behavior detection result is correct.

[0146] Step S330: verifying the review result. If the verification result indicates that the review result fails, adjusting the generation parameters of the large model input instruction according to the verification result.

[0147] In this embodiment, after receiving input instructions and monitoring data from the large model, the large model uses its trained model structure and model parameters to process and analyze the monitoring data to determine whether abnormal behavior exists. It also verifies the abnormal behavior detection results and generates corresponding verification results. The verification result is the large model's evaluation of the abnormal behavior detection results, indicating whether the abnormal behavior detection results are correct.

[0148] It should be noted that the review result can be a probability value, indicating the confidence level that the abnormal behavior detection result is a false positive; or it can be a binary classification result, directly indicating whether the abnormal behavior description is a true positive or a false positive. In addition, the review result can also include additional information, such as a confidence score and specific judgment basis.

[0149] Exemplarily, the review result may be expressed in the form of a Boolean value, where True indicates that the abnormal behavior detection result is correct, and False indicates that the abnormal behavior detection result is incorrect.

[0150] The verification process primarily involves manual inspection of the review results. If the review results contain incorrect judgments about abnormal behavior detection, the review is considered failed. Based on the verification results, the system can adjust the generation parameters of the large model input instructions, such as modifying the instruction template, adjusting the rules of the rule engine, or modifying the parameters of the large model call, to improve the accuracy of the next review.

[0151] Please refer to Figure 6 In the monitoring data processing method of the sixth embodiment proposed in this application, step S40 may further include steps S410 to S420:

[0152] Step S410: verifying the review result. If the verification result indicates that the review result passes, determining the warning level based on the abnormal behavior detection result.

[0153] Step S420: Send a corresponding alarm signal according to the warning level.

[0154] In this embodiment, the purpose of verifying the review results is to ensure that the review results output by the large model are accurate and reliable. After the review, the system will determine the warning level based on the priority of the abnormal behavior detection results. The warning level can be customized, for example, divided into low, medium, and high, and each warning level corresponds to different response measures and urgency. The verification process can include manual review, in which professional personnel review the review results output by the large model to confirm their correctness.

[0155] When the verification results indicate that the large model's accuracy has reached the preset standard, the large model training is complete. When the system performs a review based on the large model, if the large model outputs a review result indicating that the abnormal behavior detection result is correct, the warning level can be directly determined and an alarm signal can be issued. In this case, subsequent manual verification steps are no longer necessary.

[0156] It should be noted that the system can distinguish warning levels by the color, sound or other visual and auditory elements of the alarm signal to facilitate users to quickly identify and process it.

[0157] The present application provides a monitoring data processing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the monitoring data processing method in the above-mentioned embodiment one.

[0158] Reference below Figure 7 , which shows a schematic diagram of the structure of a monitoring data processing device suitable for implementing the embodiment of the present application. The monitoring data processing device in the embodiment of the present application may include various hardware and software components for implementing the scheduling method of the inspection task. Figure 7 The monitoring data processing device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0159] like Figure 7As shown, the monitoring data processing device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. RAM 1004 also stores various programs and data required for the operation of the monitoring data processing device. Processing device 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems may be connected to I / O interface 1006: input devices 1007, such as a touchscreen, touchpad, or keyboard; output devices 1008, such as a liquid crystal display (LCD), speaker, or vibrator; storage device 1003, such as a magnetic tape or hard disk; and communication devices 1009. The communication device 1009 can allow the monitoring data processing device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a monitoring data processing device with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems can be implemented or provided instead.

[0160] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0161] The monitoring data processing device provided in this application, employing the monitoring data processing device method described in the aforementioned embodiment, can solve the technical problem of improving the accuracy of video analysis. Compared to the prior art, the monitoring data processing device provided in this application has the same beneficial effects as the monitoring data processing method described in the aforementioned embodiment. Other technical features of the monitoring data processing device are the same as those disclosed in the aforementioned embodiment and are not further elaborated here.

[0162] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0163] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0164] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, and the computer-readable program instructions are used to execute the monitoring data processing method in the above-mentioned embodiment.

[0165] The computer-readable storage medium provided herein may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including, but not limited to, wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0166] The computer-readable storage medium may be included in the monitoring data processing device; or may exist independently without being assembled into the monitoring data processing device.

[0167] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the processing device of the monitoring data, the processing device of the monitoring data: obtains the abnormal behavior detection result output by the video analysis module, wherein the abnormal behavior detection result includes the abnormal behavior description and the monitoring data corresponding to the abnormal behavior description; generates a large model input instruction based on the abnormal behavior description; inputs the large model input instruction and the monitoring data into the large model to obtain the review result of the abnormal behavior detection result; and if the review result passes, issues an alarm signal.

[0168] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0169] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0170] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0171] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned surveillance data processing method, thereby solving the technical problem of improving the accuracy of video analysis. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are similar to those of the surveillance data processing method provided in the aforementioned embodiments, and are not further elaborated here.

[0172] An embodiment of the present application provides a computer program product, including a computer program, which implements the steps of the monitoring data processing method as described above when the computer program is executed by a processor.

[0173] The computer program product provided in this application can solve the technical problem of how to improve the accuracy of video analysis. Compared with the prior art, the beneficial effects of the computer program product provided in the embodiments of this application are the same as the beneficial effects of the monitoring data processing method provided in the above embodiments, and will not be repeated here.

[0174] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.

[0175] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0176] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method.

[0177] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for processing monitoring data, characterized in that: The monitoring data processing method includes: Receive a video file, and perform preliminary target detection using a video analysis module to identify a target object in the video file and a category of the target object; According to the identified category, the video analysis module selects the video analysis algorithm applicable to the category from a preconfigured video analysis algorithm library to analyze the video file to obtain an abnormal behavior detection result; Obtaining an abnormal behavior detection result output by the video analysis module, wherein the abnormal behavior detection result includes an abnormal behavior description and monitoring data corresponding to the abnormal behavior description; Identifying abnormal behavior and determining a category of the abnormal behavior based on the abnormal behavior description; Based on the categories, determining the priority and accuracy requirements of the abnormal behavior detection results; If the priority of the abnormal behavior detection result is high, performing a large model review on the abnormal behavior detection result; If the priority of the abnormal behavior detection result is low and the accuracy requirement is high, performing large model and manual double verification on the abnormal behavior detection result; If the large model is in a busy state, reviewing the abnormal behavior detection result according to the priority sorting; When conducting a large model review, a large model input instruction is generated based on the abnormal behavior description; Inputting the large model input instruction and the monitoring data into the large model to obtain a review result of the abnormal behavior detection result; If the review result is passed, an alarm signal is issued; Generating a large model input instruction according to the abnormal behavior description includes: Obtaining an algorithm identifier of the video analysis algorithm used to generate the abnormal behavior detection result; Determine a corresponding instruction template according to the algorithm identifier; The large model input instruction is generated based on the instruction template and the abnormal behavior description.

2. The method for processing monitoring data according to claim 1, wherein: Before the step of obtaining the abnormal behavior detection result output by the video analysis module, wherein the abnormal behavior detection result includes the abnormal behavior description and the monitoring data corresponding to the abnormal behavior description, the following steps are included: Performing frame segmentation on the video file to extract video frames of the video file; Performing target detection on the video frame, identifying the target object in the video frame, and extracting feature parameters of the target object; Comparing the characteristic parameters with the characteristic template to determine whether the target object has abnormal behavior; According to the comparison result, an abnormal behavior description and monitoring data corresponding to the abnormal behavior description are generated.

3. The method for processing monitoring data according to claim 2, wherein: The step of comparing the characteristic parameters with the characteristic template to determine whether the target object has abnormal behavior includes: Defining a set of abnormal behavior characteristics of the target object; Comparing the characteristic parameters of the target object with the characteristic template corresponding to the target object in the abnormal behavior characteristic set to determine the matching degree between the characteristic parameters and the characteristic template; If the matching degree reaches or exceeds a preset threshold, the target object is marked as having abnormal behavior.

4. The method for processing monitoring data according to claim 1, wherein: The step of generating a large model input instruction according to the abnormal behavior description includes: Extracting instruction parameters based on the abnormal behavior description, the instruction parameters including a target object, the abnormal behavior of the target object, and the time and location of occurrence of the abnormal behavior; Selecting a corresponding instruction template from preset instruction templates according to the category of the abnormal behavior; Fill the instruction parameters into the instruction template to generate the large model input instruction.

5. The method for processing monitoring data according to claim 1, wherein: Before the step of inputting the large model input instruction and the monitoring data into the large model and obtaining the review result of the abnormal behavior detection result, the method includes: Collect historical monitoring data and annotate abnormal behaviors in the historical monitoring data to form a labeled training data set; Inputting the training data set into the large model, training the large model, and obtaining abnormal behavior prediction results of the training data set; Comparing the abnormal behavior prediction result with the corresponding label and calculating a loss function; Based on the loss function, the learning parameters of the large model are adjusted.

6. The method for processing monitoring data according to claim 1, wherein: The step of inputting the large model input instruction and the monitoring data into the large model to obtain the review result of the abnormal behavior detection result includes: Sending the large model input instruction and the monitoring data to the large model through the API interface; receiving a review result returned by the large model, wherein the review result is used to indicate whether the abnormal behavior detection result is correct; The review result is verified. If the verification result indicates that the review result fails, the generation parameters of the large model input instruction are adjusted according to the verification result.

7. The method for processing monitoring data according to claim 1, wherein: If the review result is passed, the step of issuing an alarm signal further includes: Verifying the review result, and if the verification result indicates that the review result passes, determining a warning level based on the abnormal behavior detection result; According to the warning level, a corresponding alarm signal is sent.

8. A monitoring data processing device, characterized in that: The monitoring data processing device includes: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the monitoring data processing method according to any one of claims 1 to 7.

9. A computer storage medium, characterized in that The computer storage medium is a computer-readable storage medium, and a computer program is stored on the computer storage medium. When the computer program is executed by a processor, the steps of the monitoring data processing method according to any one of claims 1 to 7 are implemented.

Images