Monitoring data processing method and device and computer storage medium

Abstract

The invention discloses a monitoring data processing method and device and a computer storage medium, and relates to the technical field of artificial intelligence, the method comprises the steps that an abnormal behavior detection result output by a video analysis module is acquired, and the abnormal behavior detection result comprises abnormal behavior description and monitoring data corresponding to the abnormal behavior description; generating a large model input instruction according to the abnormal behavior description; inputting the large model input instruction and the monitoring data into a large model, and obtaining a recheck result of the abnormal behavior detection result; and if the re-checking result is passed, sending out an alarm signal. According to the method, the abnormal behavior detection result of the video analysis module is rechecked through the large model, so that the early warning accuracy is improved, and it is ensured that early warning can be triggered only by events which really need to be concerned.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method and device for processing monitoring data and a computer storage medium. Background Art

[0002] In the security field, after the camera collects image data, it uses artificial intelligence (AI) technology to identify abnormal behaviors, and then triggers an early warning after the abnormal behavior is identified. However, in a complex monitoring environment, the results of AI's recognition of abnormal behaviors are easily affected by occlusions in the image data or interference similar to occlusions, and it is unable to accurately identify abnormal behaviors, which leads to false early warnings. In related technologies, in order to avoid unnecessary panic caused by false early warnings, generally after AI identifies abnormal behavior, the corresponding fragments of the abnormal behavior are first sent to the manual end to manually review whether there is abnormal behavior. This results in a serious lag in the danger warning of the security system. Summary of the invention

[0003] The main purpose of this application is to provide a monitoring data processing method, device and computer storage medium, aiming to solve the technical problem of insufficient warning accuracy in early warning systems based on artificial intelligence.

[0004] To achieve the above purpose, the present invention provides a method for processing monitoring data, the method comprising: Obtaining an abnormal behavior detection result output by a video analysis module, wherein the abnormal behavior detection result includes an abnormal behavior description and monitoring data corresponding to the abnormal behavior description; Generate a large model input instruction according to the abnormal behavior description; Inputting the large model input instruction and the monitoring data into the large model to obtain a review result of the abnormal behavior detection result; If the review result is passed, an alarm signal is issued.

[0005] In one embodiment, before the step of obtaining the abnormal behavior detection result output by the video analysis module, wherein the abnormal behavior detection result includes an abnormal behavior description and monitoring data corresponding to the abnormal behavior description, the step includes: Performing frame segmentation on the video file to extract video frames of the video file; Performing target detection on the video frame, identifying the target object in the video frame, and extracting feature parameters of the target object; Comparing the characteristic parameters with the characteristic template to determine whether the target object has abnormal behavior; According to the comparison result, an abnormal behavior description and monitoring data corresponding to the abnormal behavior description are generated.

[0006] In one embodiment, the step of comparing the feature parameter with the feature template to determine whether the target object has abnormal behavior includes: Defining a set of abnormal behavior characteristics of the target object; Comparing the characteristic parameters of the target object with the characteristic template corresponding to the target object in the abnormal behavior characteristic set to determine the matching degree between the characteristic parameters and the characteristic template; If the matching degree reaches or exceeds a preset threshold, the target object is marked as having abnormal behavior.

[0007] In one embodiment, before the step of generating a large model input instruction according to the abnormal behavior description, the step includes: According to the abnormal behavior description, identifying the abnormal behavior and determining the category of the abnormal behavior; Based on the categories, determining the priority and accuracy requirements of the abnormal behavior detection results; If the priority of the abnormal behavior detection result is high, a large model review is performed on the abnormal behavior detection result; If the priority of the abnormal behavior detection result is low and the accuracy requirement is high, the abnormal behavior detection result is subjected to large model and manual double re-verification; If the large model is in a busy state, the abnormal behavior detection result is reviewed according to the priority sorting.

[0008] In one embodiment, the step of generating a large model input instruction according to the abnormal behavior description includes: Extracting instruction parameters according to the abnormal behavior description, wherein the instruction parameters include a target object, the abnormal behavior of the target object, and the time and location of occurrence of the abnormal behavior; According to the category of the abnormal behavior, a corresponding instruction template is selected from preset instruction templates; Fill the instruction parameters into the instruction template to generate the large model input instruction.

[0009] In one embodiment, before the step of inputting the large model input instruction and the monitoring data into the large model and obtaining the review result of the abnormal behavior detection result, the step includes: Collecting historical monitoring data and marking abnormal behaviors in the historical monitoring data to form a labeled training data set; Inputting the training data set into the large model, training the large model, and obtaining abnormal behavior prediction results of the training data set; Comparing the abnormal behavior prediction result with the corresponding label, and calculating a loss function; Based on the loss function, the learning parameters of the large model are adjusted.

[0010] In one embodiment, the step of inputting the large model input instruction and the monitoring data into the large model to obtain the review result of the abnormal behavior detection result includes: Sending the large model input instructions and the monitoring data to the large model through an API interface; Receiving a review result returned by the large model, wherein the review result is used to indicate whether the abnormal behavior detection result is correct; The review result is verified, and if the verification result shows that the review result fails, the generation parameters of the large model input instruction are adjusted according to the verification result.

[0011] In one embodiment, the step of issuing an alarm signal if the review result passes further includes: Verifying the review result, and if the verification result shows that the review result passes, determining a warning level based on the abnormal behavior detection result; According to the warning level, a corresponding alarm signal is sent.

[0012] An embodiment of the present application also provides a monitoring data processing device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the monitoring data processing method as described above.

[0013] An embodiment of the present application further provides a computer storage medium, which is a computer-readable storage medium. A computer program is stored on the computer storage medium. When the computer program is executed by a processor, the steps of the monitoring data processing method described above are implemented.

[0014] The embodiment of the present application discloses a method for processing monitoring data, which obtains the abnormal behavior detection result output by the video analysis module, wherein the abnormal behavior detection result includes the abnormal behavior description and the monitoring data corresponding to the abnormal behavior description; generates a large model input instruction according to the abnormal behavior description; inputs the large model input instruction and the monitoring data into the large model to obtain the review result of the abnormal behavior detection result; and issues an alarm signal if the review result passes. The present application reviews the abnormal behavior detection result of the video analysis module through the large model, thereby improving the accuracy of the early warning and ensuring that only events that really need attention will trigger the early warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flowchart of a first embodiment of a method for processing monitoring data involved in an embodiment of the present application; Figure 2 This is a flow chart of a second embodiment of a method for processing monitoring data according to an embodiment of the present application; Figure 3 This is a flow chart of a third embodiment of a method for processing monitoring data according to an embodiment of the present application; Figure 4 This is a flow chart of a fourth embodiment of a method for processing monitoring data according to an embodiment of the present application; Figure 5 This is a flow chart of a fifth embodiment of a method for processing monitoring data according to an embodiment of the present application; Figure 6 This is a flow chart of a sixth embodiment of a method for processing monitoring data according to an embodiment of the present application; Figure 7 This is a schematic diagram of the structure of the monitoring data processing equipment for this application.

[0016] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0017] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0018] In the security field, after the camera collects image data, it uses artificial intelligence (AI) technology to identify abnormal behaviors, and then triggers an early warning after the abnormal behavior is identified. However, in a complex monitoring environment, the results of AI's recognition of abnormal behaviors are easily affected by occlusions in the image data or interference similar to occlusions, and it is unable to accurately identify abnormal behaviors, which leads to false early warnings. In related technologies, in order to avoid unnecessary panic caused by false early warnings, generally after AI identifies abnormal behavior, the corresponding fragments of the abnormal behavior are first sent to the manual end to manually review whether there is abnormal behavior. This results in a serious lag in the danger warning of the security system.

[0019] In order to solve the above-mentioned defects existing in the related art, the embodiment of the present application proposes a method for processing monitoring data, which obtains the abnormal behavior detection result output by the video analysis module, wherein the abnormal behavior detection result includes the abnormal behavior description and the monitoring data corresponding to the abnormal behavior description; generates a large model input instruction according to the abnormal behavior description; inputs the large model input instruction and the monitoring data into the large model to obtain the review result of the abnormal behavior detection result; if the review result passes, an alarm signal is issued. The present application reviews the abnormal behavior detection result of the video analysis module through the large model, thereby improving the accuracy of the early warning and ensuring that only events that really need attention will trigger the early warning.

[0020] It should be noted that the execution subject of this embodiment can be a monitoring data processing system, or a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or a monitoring data processing device capable of realizing the above functions, etc. The following takes the monitoring data processing system as an example (hereinafter referred to as the "system") to illustrate this embodiment and the following embodiments.

[0021] For the method of processing monitoring data of the first embodiment proposed in this application, please refer to Figure 1 The method comprises steps S10 to S40: Step S10: Acquire the abnormal behavior detection result output by the video analysis module, wherein the abnormal behavior detection result includes an abnormal behavior description and monitoring data corresponding to the abnormal behavior description.

[0022] In this embodiment, the system first creates or generates a task for warning the situation in the specified area. This task is sent to the video analysis module through the API (Application Programming Interface), and the video analysis module is responsible for analyzing the video files of the specified area. The abnormal behavior detection result refers to the potential risk or abnormal behavior description detected by the video analysis module, such as the category (such as smoke, sparks, etc.) and location information of the target object, as well as the monitoring data corresponding to the abnormal behavior description. Monitoring data refers to visual data containing the target object. The target object refers to an object identified as abnormal behavior or abnormal event in the abnormal behavior detection result, such as a person smoking or fireworks in progress.

[0023] Optionally, the system exchanges data by establishing a communication connection with the API interface provided by the video analysis module. When the video analysis module detects abnormal behavior or abnormal events in the received video file, it will immediately push the abnormal behavior detection results containing abnormal behavior descriptions and corresponding monitoring data through the API interface. The system monitors the API interface in real time. After receiving the abnormal behavior detection results, the system will parse the received abnormal behavior detection results and extract the abnormal behavior descriptions and corresponding monitoring data.

[0024] It should be noted that the sources of video files may include surveillance cameras, mobile devices, drone monitoring, remote servers, etc. These video files can be video streams transmitted in real time by surveillance cameras, video files uploaded by users through mobile devices, real-time images monitored by drones in the air, and video files stored on remote servers.

[0025] Optionally, multiple video analysis algorithms are preconfigured in the system, and each video analysis algorithm analyzes a specific category of target objects. The configuration information of the video analysis algorithm is sent to the video analysis module through the API interface, and the video analysis module performs abnormal behavior analysis on the received video file according to the configured video analysis algorithm. When a video file enters the video analysis module, it first uses a preliminary target detection algorithm to quickly identify potential target objects in the video file. After the target object is determined, the video analysis module will select an appropriate video analysis algorithm to perform abnormal behavior analysis based on the category of the target object.

[0026] Specifically, after receiving the video file, the video analysis module performs preliminary target detection, which aims to identify various potential target objects in the video file and determine the category of the target object. Next, according to the category of the target object in the preliminary detection result, a video analysis algorithm suitable for the corresponding category is selected from the preconfigured video analysis algorithm library. For example, if the target object is initially detected as "smoke", the video analysis module will call a video analysis algorithm specifically for smoke detection; if the target object is "smoker", the video analysis algorithm for behavior recognition will be called.

[0027] After selecting the video analysis algorithm, the video analysis module conducts in-depth analysis of the target object and generates abnormal behavior detection results containing detailed abnormal behavior descriptions and corresponding monitoring data. These abnormal behavior detection results not only include the target object, the abnormal behavior description of the target object, and the monitoring data corresponding to the abnormal behavior description, but also may include the identification of the video analysis algorithm used to analyze the corresponding video file.

[0028] It should be noted that the identification of the video analysis algorithm is a specific code or label used to distinguish different video analysis algorithms. The identification can be a name, number, or a metadata structure containing version, parameter, and configuration information. There are multiple video analysis algorithms integrated in the system. Each video analysis algorithm may have different recognition accuracy, reliability, or applicability in specific scenarios for different target object categories, such as smoke detection, face recognition, or vehicle recognition. The video analysis module receives the video analysis algorithms configured by the system through the API interface and uses these video analysis algorithms to analyze video files.

[0029] In a feasible implementation manner, before step S10, steps S11 to S14 may also be included: Step S11, performing frame segmentation on the video file to extract video frames of the video file.

[0030] It should be noted that frame segmentation is the process of extracting individual video frames from a continuous video file in chronological order, which is equivalent to decomposing a continuous dynamic image into a series of static images. Each video frame captures the visual content of the video file at a specific time point. Video frame is the basic unit of video files.

[0031] Optionally, after receiving the video file, the video analysis module extracts individual video frames from the video file at a certain frame rate through a video processing library (such as OpenCV). Specifically, the video analysis module initializes the video decoder, which is responsible for decoding the compressed data stream in the video file into processable video frames. Next, the frame extraction interval is set, which determines the frequency of extracting video frames from the video file. Then, each frame in the video file is read in a loop, and each read will obtain a video frame containing the visual information of the video file at a specific moment. After the video frame is extracted, it is stored as an image data structure for processing, such as a cv2.Mat object in OpenCV.

[0032] It should be noted that the cv2.Mat object is a matrix-based data structure used to store pixel data of an image.

[0033] Step S12: performing target detection on the video frame, identifying the target object in the video frame, and extracting feature parameters of the target object.

[0034] In this embodiment, the video analysis module passes the extracted video frames as input to the configured video analysis algorithms, which can be traditional computer vision algorithms, such as the SIFT (Scale-invariant feature transform) algorithm, which can identify and describe key points in an image and remain invariant even when the image size, rotation, and illumination change; or the HOG (Histogram of Oriented Gradients) combined with the SVM (Support Vector Machine) algorithm. The HOG algorithm constructs features by calculating and counting the gradient directional histograms of local areas of the image, while SVM is a machine learning algorithm for classification that can effectively classify features.

[0035] In addition, the video analysis module may also use deep learning-based models, such as the YOLO (You Only LookOnce) algorithm, which is an object detection algorithm that can simultaneously predict multiple bounding boxes and category probabilities in a single forward propagation, thereby achieving rapid identification of target objects in video frames.

[0036] In the process of processing video frames, the video frames are first preprocessed, including grayscale processing, denoising, adjusting brightness and contrast, etc., to reduce the impact of factors such as ambient light changes and camera noise on the results. After completing the image preprocessing, feature extraction techniques such as edge detection, color analysis, shape recognition, etc. are then used to perform target detection on the video frames to identify target objects in the video frames, such as people, vehicles, smoke, etc.

[0037] After the target object is detected, the feature parameters of the target object are further extracted. The feature parameters include bounding box coordinates, shape, color histogram, motion trajectory, contour change, etc. At the same time, by quantifying these parameters, the corresponding attribute values ​​can be calculated. The attribute value can be the size, speed, color or more complex attributes of the target object, such as the density of smoke.

[0038] For example, when analyzing the target object "smoke", the concentration of smoke can be estimated by analyzing the color intensity or transparency of the smoke area, or by calculating the pixel value of the smoke-covered area. At the same time, the spread range can be determined by calculating the area of ​​the smoke boundary box or tracking the change of smoke over time. In addition, the dynamic change speed of smoke can be determined by comparing the changes in the shape and position of smoke in consecutive frames.

[0039] Step S13: compare the feature parameters with the feature template to determine whether the target object has abnormal behavior.

[0040] Step S14: Generate an abnormal behavior description and monitoring data corresponding to the abnormal behavior description according to the comparison result.

[0041] It should be noted that the feature template is a predefined set of parameters that represent abnormal behavior or abnormal state. These parameter sets reflect the characteristic performance of the target object under abnormal conditions. The establishment of the feature template is based on a large amount of historical data analysis and related domain knowledge, ensuring that the feature template can accurately reflect the behavior pattern or characteristic pattern of the target object under abnormal conditions. The feature template can include the speed range, shape characteristics, color distribution, motion trajectory, etc. of the target object.

[0042] During the comparison process, the matching degree between the feature parameters and the feature template is calculated. Based on the comparison results, it is determined whether the behavior of the target object matches the feature template. If the feature parameters highly match the feature template, it is considered that the target object has abnormal behavior; if the matching degree is low, it is considered that the behavior of the target object is normal.

[0043] The video analysis module sets a matching threshold, which defines the minimum acceptable similarity between the feature parameters and the feature template. If the calculated matching exceeds the matching threshold, that is, the similarity between the feature parameters and the abnormal behavior feature template is high, the video analysis module will determine that the target object has abnormal behavior. This is because a high matching degree means that the feature parameters of the target object are very close to the feature template of the abnormal behavior, indicating that the behavior of the target object is similar to the predefined abnormal behavior.

[0044] On the contrary, if the matching degree is lower than this threshold, that is, the similarity between the feature parameters and the abnormal behavior feature template is low, the video analysis module considers that the behavior of the target object is normal because it is quite different from the abnormal behavior feature template and does not meet the characteristics of abnormal behavior.

[0045] Exemplarily, if the shape features or visual features of the target object are slightly different from the shape features in the feature template, that is, the matching degree exceeds a preset threshold, the video analysis module will deem the behavior of the target object abnormal.

[0046] In a feasible implementation manner, step S13 may further include steps S131 to S133: Step S131: defining an abnormal behavior feature set of the target object.

[0047] In this embodiment, the abnormal behavior feature set is a set of multiple target objects and feature parameters of the target objects, which together describe the characteristic performance of the target objects when abnormal behavior occurs. These feature sets are usually constructed based on the definition of abnormal behavior of the target objects, reflecting the behavior mode or feature mode of the target objects under abnormal circumstances. The feature parameters may include the shape, color, motion trajectory, speed, etc. of the target objects, and may also include more complex attributes, such as the interaction mode of the target objects with other objects, the behavior mode of the target objects, etc.

[0048] In the specific implementation, it is necessary to first collect and organize the data of normal and abnormal behaviors related to the target object. Then, through data analysis methods, the key features that can distinguish normal and abnormal behaviors are extracted. These key features are used to construct feature templates, which will serve as the basis for judging whether the target object has abnormal behavior.

[0049] Step S132: comparing the feature parameters of the target object with the feature template corresponding to the target object in the abnormal behavior feature set, and determining the matching degree between the feature parameters and the feature template.

[0050] Step S133: If the matching degree reaches or exceeds a preset threshold, the target object is marked as having abnormal behavior.

[0051] After the video analysis module extracts the video frames, it uses feature extraction technology to identify the target object from these video frames. Once the target object is identified, the video analysis module calculates the feature parameters of the target object, such as bounding box coordinates, shape, color histogram, etc. Then, these feature parameters are compared with the pre-defined abnormal behavior feature set. If the feature parameters of the target object highly match the parameters of the corresponding feature template in the abnormal behavior feature set, that is, if they exceed the preset matching threshold, the video analysis module will determine that the target object has abnormal behavior.

[0052] Step S20: Generate a large model input instruction according to the abnormal behavior description.

[0053] In this embodiment, the system first extracts key information from the abnormal behavior description, such as the category of the target object (such as people, smoke), location (such as the coordinates and name of the monitoring area), behavior description (such as smoking, fire), etc. Then, based on these key information and combined with the preset instruction template, the system automatically generates a large model input instruction that describes the behavior of the target object. For example, if the target object is identified as a "person who is smoking", the system will use the corresponding instruction template to generate a large model input instruction, such as "someone is detected smoking in a no-smoking area."

[0054] Optionally, the system can use preset instruction templates and rule engines to automate the generation of large model input instructions. The instruction templates contain standardized sentence structures for describing different categories of target objects and behaviors, while the rule engine selects and fills in the corresponding instruction templates based on the attributes of the target object and other parameters in the abnormal behavior description according to predefined rules to generate specific large model input instructions.

[0055] For example, if the abnormal behavior description indicates that the detected target object is "smoke" and is located in area A, and the location and visual characteristics of the smoke are displayed in the monitoring data, the system can generate the following large model input instruction: "A high concentration of smoke is detected in area A. The smoke characteristics are bright flames accompanied by dot-shaped rising smoke, which may indicate a fire and trigger an early warning. Please check whether the early warning result is correct."

[0056] Step S30: input the large model input instruction and the monitoring data into the large model to obtain the review result of the abnormal behavior detection result.

[0057] In this embodiment, a trained large model is constructed, which has data processing and pattern recognition capabilities and can be used to review the abnormal behavior detection results output by the video analysis module. The large model is usually based on a deep learning framework and is trained with a large amount of labeled data so that it can accurately identify and classify various abnormal behaviors.

[0058] Before using the big model for review, in order to accurately review the abnormal behavior detection results, a structured input format, such as a JSON object, is constructed to integrate the big model input instructions and the corresponding monitoring data. The monitoring data is input into the big model as visual data, and the big model input instructions are input as supplementary context information.

[0059] It should be noted that large models are usually based on deep learning frameworks, such as Convolutional Neural Networks (CNN) for processing image data, and Recurrent Neural Networks (RNN) or Transformer models for processing text data. These models can identify the characteristics of the target object from the monitoring data, and combine the contextual information in the input instructions of the large model to review the abnormal behavior description to indicate whether the abnormal behavior description is a false alarm.

[0060] Specifically, the system encodes the monitoring data into a format recognizable by the big model, such as a Base64-encoded string, and uses it as a field in a JSON object. At the same time, the big model input instructions are also embedded in the same JSON object and converted into vector representations through word embedding processing so that the big model can understand the semantic content of the text. Such structured input ensures that the big model can consider both visual and textual information, thereby more comprehensively evaluating the accuracy of abnormal behavior descriptions.

[0061] It should be noted that Base64 is a method of representing binary data based on 64 printable characters. The purpose of Base64 encoding is to convert binary data into a text format so that it can be transmitted through text transmission protocols such as email without destroying the integrity of the data.

[0062] After receiving the structured input, the big model extracts the feature data of the monitoring data and the big model input instruction respectively. Based on these feature data, the big model generates a review result, which indicates whether the input abnormal behavior detection result is real and needs to be warned or is a false alarm.

[0063] Step S40: If the review result is passed, an alarm signal is issued.

[0064] In this embodiment, the system will determine whether it is necessary to issue an alarm signal based on the review results. If the review results show that the abnormal behavior detection result is real and an alarm is needed, the system will immediately trigger the alarm mechanism and issue an alarm signal. The alarm signal can be conveyed to relevant personnel in a variety of ways, such as displaying through the system interface, sending SMS or email notifications, triggering sound or light alarms, etc. In this way, relevant personnel can obtain early warnings of abnormal behavior in a timely manner and take appropriate measures to deal with it.

[0065] Please refer to Figure 2 The monitoring data processing method of the second embodiment proposed in the present application, before step S20, further includes steps S201 to S205: Step S201: According to the abnormal behavior description, identify the abnormal behavior and determine the category of the abnormal behavior.

[0066] The system first analyzes the abnormal behavior description received from the video analysis module, which usually contains key information such as the category of the target object (such as people, vehicles, smoke, etc.), location coordinates, behavior characteristics, etc. By analyzing the abnormal behavior description, the system can identify specific abnormal behaviors and classify them into predefined abnormal behavior categories.

[0067] Step S202: Based on the category, determine the priority and accuracy requirement of the abnormal behavior detection result.

[0068] Step S203: If the priority of the abnormal behavior detection result is high, a large model review is performed on the abnormal behavior detection result.

[0069] Based on the determined abnormal behavior categories, the system can set the priority and accuracy requirements for the abnormal behavior detection results. Different abnormal behavior categories may correspond to different security risk levels, so the review priority needs to be determined based on the category. At the same time, the accuracy requirements can also vary according to the category to ensure that critical or high-risk abnormal behaviors can be reviewed more strictly.

[0070] Optionally, the system can dynamically adjust the setting of priority and accuracy requirements according to actual conditions. In the case of limited resources, when monitoring resources are tight, the system may only sort and process abnormal behaviors based on their priority without pursuing accuracy. In this case, the system can directly rely on the big model for review, omitting the dual mechanism of manual review to improve the efficiency of the early warning processing process.

[0071] It is understandable that for certain specific tasks identified by the system, if there are additional requirements for accuracy (i.e., high accuracy is required), a double-check mechanism will be enabled. This is done to ensure the accuracy of the alarm and avoid false alarms. In this case, in addition to the automatic review of the large model, a manual review will also be added. For general tasks (i.e., when there are no additional requirements for accuracy), the large model will be used for review.

[0072] The purpose of setting priorities is to sort abnormal behaviors according to their urgency and importance when reviewing large models, so as to more effectively allocate review resources and ensure that key events are handled with priority.

[0073] If the abnormal behavior detection result has a higher priority, it will be directly reviewed by the big model. This means that the system will use the trained big model to further verify the abnormal behavior detection result to improve the accuracy and reliability of the result.

[0074] Step S204: If the priority of the abnormal behavior detection result is low and the accuracy requirement is high, the abnormal behavior detection result is subjected to a large model and manual double verification.

[0075] For abnormal behavior detection results with lower priority but higher accuracy requirements, the system will not only use the large model for review, but also add a manual review to ensure the accuracy of the final results. This double review mechanism is suitable for those abnormal behaviors that are not the highest priority but whose misjudgment may lead to more serious consequences.

[0076] Step S205: If the large model is in a busy state, the abnormal behavior detection result is reviewed according to the priority sorting.

[0077] It should be noted that the busy state means that the large model is currently performing other computing tasks or processing the review of other abnormal behavior detection results, which causes its resources (such as computing power, memory, etc.) to be occupied and cannot immediately accept and process new review requests. The system will monitor the resource usage of the large model in real time. When it detects that the resources of the large model are fully occupied, the system will mark it as busy.

[0078] For example, when the large model is busy, there may be multiple abnormal behavior detection results waiting for review. The system will sort them according to their priority to ensure that high-priority abnormal behavior detection results can be processed in time. If the large model is not currently available, the system will queue the abnormal behavior detection results in order of priority, and once the large model resources are released, they will be processed in the order of the queue.

[0079] Please refer to Figure 3 In the monitoring data processing method of the third embodiment proposed in the present application, step S20 may further include steps S210 to S230: Step S210: extracting instruction parameters according to the abnormal behavior description, wherein the instruction parameters include a target object, the abnormal behavior of the target object, and the time and location of occurrence of the abnormal behavior.

[0080] The system extracts key information from the abnormal behavior description as the basis for the subsequent generation of large model input instructions. These instruction parameters usually include the target object (i.e. the subject of the abnormal behavior, such as a person, vehicle or object), the abnormal behavior of the target object (i.e. the specific unusual or illegal actions), and the time and place of the abnormal behavior.

[0081] For example, if the abnormal behavior description states that "someone was detected smoking in a no-smoking area in area A", the system will extract the target object as "person", the abnormal behavior corresponds to "smoking", the time of occurrence is a certain current time, and the location is "area A" and "no-smoking area".

[0082] Step S220: Select a corresponding instruction template from preset instruction templates according to the category of the abnormal behavior.

[0083] In monitoring systems, multiple instruction templates are usually preset to guide large models to review different types of abnormal behaviors. These instruction templates are customized based on factors such as the nature, urgency, and review requirements of the abnormal behavior to ensure the accuracy and efficiency of the review process.

[0084] It should be noted that each category of abnormal behavior is associated with one or more specific instruction templates. These instruction templates are pre-defined text structures used to guide the large model on how to understand and process input data.

[0085] Step S230: Fill the instruction parameters into the instruction template to generate the large model input instruction.

[0086] After selecting the appropriate instruction template, the system will use the extracted instruction parameters to fill in the variable part of the instruction template to generate a large model output instruction. Through this step, the system can generate a complete, accurate and compliant large model input instruction to guide the large model to review abnormal behavior. In the process of filling in parameters, the system will ensure the accuracy and completeness of the parameters to avoid inaccurate review results due to missing or incorrect information.

[0087] For example, if the instruction template requires the input format of "It is detected that {target object} performed {abnormal behavior} at {location} at {time}, and {abnormal event} may occur, triggering an early warning. Please check whether the early warning result is correct", the system will substitute the extracted instruction parameters into the corresponding positions to generate a large model input instruction such as "It is detected that a person is smoking in area A at the current time, and a fire may occur, triggering an early warning. Please check whether the early warning result is correct."

[0088] Optionally, the system can use statistical models, such as n-gram models, to predict and generate large model input instructions. Statistical models can learn the statistical laws of language by analyzing large amounts of text data, and generate instruction content based on these laws.

[0089] Specifically, the historical abnormal behavior detection results and their corresponding correct annotations are preprocessed, including text cleaning, word segmentation, and serialization. Then, these data are used to train the n-gram model and calculate the probability distribution of different language unit sequences, so that the next language unit can be predicted based on the known context when generating large model input instructions.

[0090] When generating large model input instructions, the system uses the key information extracted from the abnormal behavior detection results as input, and iteratively selects the words with the highest probability through the n-gram model to construct a complete sentence. For example, the system may generate the following large model input instruction: "Smoke was detected at {specific location}, and there may be a fire source, triggering an early warning. Please check whether the early warning result is correct." Here, "{specific location}" can be dynamically filled according to the abnormal behavior detection results, and the structure and content of the entire sentence are generated by the n-gram model based on the learned language rules.

[0091] Optionally, the system can adjust the format and content of the large model input instructions by customizing the instruction templates to adapt to different target object categories. The implementation of this method involves creating multiple predefined instruction templates, each of which is designed for a specific target object category and contains corresponding descriptions and instructions. When the system receives the abnormal behavior detection results and identifies the target object, it selects the appropriate instruction template based on the target object category, and then fills the analyzed instruction parameters into the instruction template to generate the final large model input instruction.

[0092] In addition, the system can also determine the large model input instructions based on the identification of the video analysis algorithm to ensure that the large model input instructions are coordinated with the target object and the analysis technology used. When the system receives abnormal behavior detection results, it will identify the algorithm identification that generated these abnormal behavior detection results. This identification can help the system understand which specific video analysis algorithm is used by the video analysis module to process the video file and generate abnormal behavior detection results, so that the system can select the appropriate instruction template according to the characteristics of the corresponding video analysis algorithm, thereby adjusting the tone and accuracy of the large model input instructions to more accurately reflect the analysis results and keep the information clear and practical.

[0093] Exemplarily, the system maintains a command template library that contains predefined command templates for different categories of target objects. For example, for smoke, the command template may contain "Smoke is detected at {specific location}, the smoke concentration has reached {warning data}, triggering the warning, please confirm whether the warning result is correct". This template clearly conveys the warning result, allowing the recipient to quickly understand the situation and respond. Among them, "{specific location}" and "{warning data}" in the command template can be replaced according to the actual abnormal behavior description to ensure the accuracy and real-time nature of the large model input command.

[0094] Please refer to Figure 4 The monitoring data processing method of the fourth embodiment proposed in the present application may further include steps S301 to S304 before step S30: Step S301: historical monitoring data is collected, and abnormal behaviors in the historical monitoring data are annotated to form a labeled training data set.

[0095] In this embodiment, the historical monitoring data covers the monitoring records stored in the system, and also includes abnormal behavior related information from other channels, such as pictures, videos, dynamic images, and independent video frames. The historical monitoring data contains the activities in a specific area over a period of time. The labeling process can be completed by professionals or automated labeling tools. The purpose is to identify and mark abnormal behaviors in historical monitoring data. These abnormal behaviors are given specific labels, such as "fire", "smoking", etc. The labeling information usually includes the timestamp, type, and target object involved of the abnormal behavior.

[0096] Step S302: input the training data set into the large model, train the large model, and obtain the abnormal behavior prediction result of the training data set.

[0097] During the training process, the big model receives annotated training data sets that contain historical monitoring data and corresponding abnormal behavior labels. The big model processes the input data through its internal neural network structure, learning how to extract features from historical monitoring data and predict abnormal behavior. The prediction result is the big model's review result of the abnormal behavior labels in the training data set or the probability output of abnormal behavior.

[0098] Step S303: Compare the abnormal behavior prediction result with the corresponding label and calculate the loss function.

[0099] In this embodiment, the loss function is an indicator that measures the difference between the model prediction result and the actual label. Commonly used loss functions include cross entropy loss, mean square error, etc. In this step, the system compares the model's prediction result with the actual label in the training data set and calculates the loss value, which reflects the performance of the large model on the current training data set.

[0100] For classification tasks, the commonly used loss functions are Cross Entropy or Binary Cross-Entropy. For regression tasks, the commonly used loss functions are Mean Squared Error (MSE) or Mean Absolute Deviation (MAD).

[0101] Step S304: Based on the loss function, the learning parameters of the large model are adjusted.

[0102] In this embodiment, in order to update the weight of the large model, it is necessary to calculate the gradient of the loss function with respect to the model parameters, and then update the weight and bias of the large model according to the gradient descent algorithm or other optimization algorithms, such as the Adam optimization algorithm, to reduce the prediction error. This process is repeated until the loss function of the large model falls within a preset range or reaches a preset training round, thereby completing the training of the large model.

[0103] Please refer to Figure 5 In the monitoring data processing method of the fifth embodiment proposed in the present application, step S30 may further include steps S310 to S330: Step S310: sending the large model input instruction and the monitoring data to the large model through an API interface.

[0104] Step S320: receiving the review result returned by the large model, wherein the review result is used to indicate whether the abnormal behavior detection result is correct.

[0105] Step S330: verifying the review result. If the verification result indicates that the review result fails, adjusting the generation parameters of the large model input instruction according to the verification result.

[0106] In this embodiment, after receiving the input instructions and monitoring data of the big model, the big model will use its trained model structure and model parameters to process and analyze the monitoring data to determine whether there is abnormal behavior, and at the same time confirm the abnormal behavior detection results and generate corresponding review results. The review result is the evaluation of the abnormal behavior detection results by the big model, which indicates whether the abnormal behavior detection results are correct.

[0107] It should be noted that the review result can be a probability value, indicating the confidence that the abnormal behavior detection result is a false alarm; or it can be a binary classification result, directly indicating whether the abnormal behavior description is a positive alarm or a false alarm. In addition, the review result can also include some additional information, such as confidence score, specific judgment basis, etc.

[0108] Exemplarily, the review result may be expressed in the form of a Boolean value, where True indicates that the abnormal behavior detection result is correct, and False indicates that the abnormal behavior detection result is incorrect.

[0109] The verification process mainly involves manually checking whether the review results are correct. If the judgment of the abnormal behavior detection results in the review results is incorrect, the review results are considered to have failed. At this time, the system can adjust the generation parameters of the large model input instructions based on the verification results, such as modifying the instruction template, adjusting the rules of the rule engine, modifying the parameters of the large model call, etc., to improve the accuracy of the next review.

[0110] Please refer to Figure 6In the monitoring data processing method of the sixth embodiment proposed in the present application, step S40 may further include steps S410 to S420: Step S410: verifying the review result, and if the verification result shows that the review result passes, determining the warning level based on the abnormal behavior detection result.

[0111] Step S420: Send a corresponding alarm signal according to the warning level.

[0112] In this embodiment, the purpose of verifying the review results is to ensure that the review results output by the large model are accurate and reliable. After the review, the system will determine the warning level based on the priority of the abnormal behavior detection results. The warning level can be customized, for example, divided into low, medium, and high levels, and each warning level corresponds to different response measures and urgency. The verification process can include manual review, in which professionals review the review results output by the large model to confirm their correctness.

[0113] When the verification results show that the accuracy of the big model review has reached the preset standard, it means that the big model training is complete. When the system is reviewing based on the big model, if the big model output review result shows that the judgment of the abnormal behavior detection result is correct, the warning level can be directly determined and an alarm signal can be sent. At this time, the subsequent manual verification step may not be required.

[0114] It should be noted that the system can distinguish warning levels by the color, sound or other visual and auditory elements of the alarm signal to facilitate users to quickly identify and process it.

[0115] The present application provides a monitoring data processing device, the monitoring data processing device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the monitoring data processing method in the above-mentioned embodiment one.

[0116] Reference below Figure 7 , which shows a schematic diagram of the structure of a monitoring data processing device suitable for implementing the embodiment of the present application. The monitoring data processing device in the embodiment of the present application may include various hardware and software components for implementing the scheduling method of the inspection task. Figure 7 The monitoring data processing device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0117] like Figure 7As shown, the monitoring data processing device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the monitoring data processing device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the processing device of monitoring data to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows the processing device of monitoring data with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0118] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0119] The monitoring data processing device provided by the present application adopts the monitoring data processing device method in the above embodiment, which can solve the technical problem of how to improve the accuracy of video analysis. Compared with the prior art, the beneficial effects of the monitoring data processing device provided by the present application are the same as the beneficial effects of the monitoring data processing method provided by the above embodiment, and the other technical features in the monitoring data processing device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.

[0120] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0121] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0122] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the monitoring data processing method in the above-mentioned embodiment.

[0123] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0124] The computer-readable storage medium may be included in the monitoring data processing device; or may exist independently without being assembled into the monitoring data processing device.

[0125] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by a processing device for monitoring data, the processing device for monitoring data: obtains an abnormal behavior detection result output by a video analysis module, wherein the abnormal behavior detection result includes an abnormal behavior description and monitoring data corresponding to the abnormal behavior description; generates a large model input instruction according to the abnormal behavior description; inputs the large model input instruction and the monitoring data into the large model to obtain a review result of the abnormal behavior detection result; and issues an alarm signal if the review result passes.

[0126] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0127] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0128] The modules involved in the embodiments described in the present application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0129] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned monitoring data processing method, and can solve the technical problem of how to improve the accuracy of video analysis. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the monitoring data processing method provided in the above-mentioned embodiment, and will not be repeated here.

[0130] An embodiment of the present application provides a computer program product, including a computer program, which implements the steps of the monitoring data processing method as described above when the computer program is executed by a processor.

[0131] The computer program product provided in this application can solve the technical problem of how to improve the accuracy of video analysis. Compared with the prior art, the beneficial effects of the computer program product provided in the embodiment of this application are the same as the beneficial effects of the monitoring data processing method provided in the above embodiment, which will not be repeated here.

[0132] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.

[0133] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0134] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method.

[0135] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for processing monitoring data, characterized in that: The monitoring data processing method includes: Obtaining an abnormal behavior detection result output by a video analysis module, wherein the abnormal behavior detection result includes an abnormal behavior description and monitoring data corresponding to the abnormal behavior description; Generate a large model input instruction according to the abnormal behavior description; Inputting the large model input instruction and the monitoring data into the large model to obtain a review result of the abnormal behavior detection result; If the review result is passed, an alarm signal is issued.

2. The method for processing monitoring data according to claim 1, characterized in that: Before the step of obtaining the abnormal behavior detection result output by the video analysis module, wherein the abnormal behavior detection result includes the abnormal behavior description and the monitoring data corresponding to the abnormal behavior description, the method further includes: Performing frame segmentation on the video file to extract video frames of the video file; Performing target detection on the video frame, identifying the target object in the video frame, and extracting feature parameters of the target object; Comparing the characteristic parameters with the characteristic template to determine whether the target object has abnormal behavior; According to the comparison result, an abnormal behavior description and monitoring data corresponding to the abnormal behavior description are generated.

3. The method for processing monitoring data according to claim 2, characterized in that: The step of comparing the feature parameters with the feature template to determine whether the target object has abnormal behavior includes: Defining a set of abnormal behavior characteristics of the target object; Comparing the characteristic parameters of the target object with the characteristic template corresponding to the target object in the abnormal behavior characteristic set to determine the matching degree between the characteristic parameters and the characteristic template; If the matching degree reaches or exceeds a preset threshold, the target object is marked as having abnormal behavior.

4. The method for processing monitoring data according to claim 1, characterized in that: Before the step of generating a large model input instruction according to the abnormal behavior description, the method includes: According to the abnormal behavior description, identifying the abnormal behavior and determining the category of the abnormal behavior; Based on the categories, determining the priority and accuracy requirements of the abnormal behavior detection results; If the priority of the abnormal behavior detection result is high, a large model review is performed on the abnormal behavior detection result; If the priority of the abnormal behavior detection result is low and the accuracy requirement is high, the abnormal behavior detection result is subjected to large model and manual double re-verification; If the large model is in a busy state, the abnormal behavior detection result is reviewed according to the priority sorting.

5. The method for processing monitoring data according to claim 1, characterized in that: The step of generating a large model input instruction according to the abnormal behavior description comprises: Extracting instruction parameters according to the abnormal behavior description, wherein the instruction parameters include a target object, the abnormal behavior of the target object, and the time and location of occurrence of the abnormal behavior; According to the category of the abnormal behavior, a corresponding instruction template is selected from preset instruction templates; Fill the instruction parameters into the instruction template to generate the large model input instruction.

6. The method for processing monitoring data according to claim 1, characterized in that: Before the step of inputting the large model input instruction and the monitoring data into the large model and obtaining the review result of the abnormal behavior detection result, the method includes: Collecting historical monitoring data and marking abnormal behaviors in the historical monitoring data to form a labeled training data set; Inputting the training data set into the large model, training the large model, and obtaining abnormal behavior prediction results of the training data set; Comparing the abnormal behavior prediction result with the corresponding label, and calculating a loss function; Based on the loss function, the learning parameters of the large model are adjusted.

7. The method for processing monitoring data according to claim 1, characterized in that: The step of inputting the large model input instruction and the monitoring data into the large model to obtain the review result of the abnormal behavior detection result includes: Sending the large model input instruction and the monitoring data to the large model through an API interface; Receiving a review result returned by the large model, wherein the review result is used to indicate whether the abnormal behavior detection result is correct; The review result is verified, and if the verification result shows that the review result fails, the generation parameters of the large model input instruction are adjusted according to the verification result.

8. The method for processing monitoring data according to claim 1, characterized in that: If the review result is passed, the step of issuing an alarm signal also includes: Verifying the review result, and if the verification result shows that the review result passes, determining a warning level based on the abnormal behavior detection result; According to the warning level, a corresponding alarm signal is sent.

9. A monitoring data processing device, characterized in that: The monitoring data processing device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the monitoring data processing method according to any one of claims 1 to 8.

10. A computer storage medium, characterized in that: The computer storage medium is a computer-readable storage medium, and a computer program is stored on the computer storage medium. When the computer program is executed by a processor, the steps of the monitoring data processing method according to any one of claims 1 to 8 are implemented.

Images