Data processing method and device, equipment and medium

Abstract

The invention belongs to the field of artificial intelligence, and relates to a data processing method, which comprises the following steps: acquiring video monitoring data in a preset time period; a preset first sliding window is adopted to segment the video monitoring data to obtain a plurality of video clips, and each video clip comprises a plurality of video frames; instantiation processing is carried out on each video clip, a plurality of instances are generated, and each instance comprises multiple frames of continuous video images; performing anomaly detection on the plurality of examples by adopting a preset anomaly detection model to obtain a target prediction score of the video monitoring data; and if the target prediction score reaches a preset score threshold, triggering a preset alarm mechanism to give an alarm. The invention further provides a data processing device, equipment and a medium. In addition, the invention also relates to a block chain technology, and the video monitoring data and the target prediction score information can be stored in a block chain. The video data detection accuracy can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a data processing method, device, equipment and medium. Background Art

[0002] With the rapid development of the financial industry, the security issues of bank ATMs and bank entrances have become increasingly prominent, and their importance cannot be ignored. Traditional monitoring methods often rely on manual monitoring, which is not only inefficient, but also difficult to achieve immediate response to abnormal events. Therefore, it is particularly important to develop an intelligent monitoring system that can monitor bank ATMs and entrances in real time.

[0003] At present, there are already some intelligent monitoring systems on the market, which can efficiently monitor the security of banking institutions and cash transaction points, and effectively prevent the occurrence of crimes such as theft and robbery. However, these systems still have many shortcomings in the identification of abnormal behavior. First, since the definition of abnormal behavior is quite subjective and may vary from person to person, it is difficult for the system to accurately identify abnormal behavior. Secondly, when marking abnormal situations as 1 / 0 labels, there is also a lack of clear standards, which makes the judgment results of the system uncertain. In addition, due to the scarcity of abnormal samples, the existing methods generally have low model accuracy when abnormal samples are scarce, which is difficult to meet the needs of practical applications.

[0004] In summary, the current intelligent monitoring systems at bank ATMs and bank entrances have insufficient precision in identifying abnormal behaviors in monitoring data and low accuracy in video data detection. Summary of the invention

[0005] The purpose of the embodiments of the present application is to propose a data processing method, device, equipment and medium to solve the problems of insufficient precision and low video data detection accuracy when identifying abnormal behavior in existing monitoring data.

[0006] In order to solve the above technical problems, the present application provides a data processing method, which adopts the following technical solutions:

[0007] The video surveillance data within a preset time period is obtained; a preset first sliding window is used to segment the video surveillance data to obtain a plurality of video segments, each of which includes a plurality of video frames; each video segment is instantiated to generate a plurality of instances, each of which includes a plurality of frames of continuous video images; a preset anomaly detection model is used to perform anomaly detection on the plurality of instances to obtain a target prediction score of the video surveillance data; if the target prediction score reaches a preset score threshold, a preset alarm mechanism is triggered to issue an alarm.

[0008] Furthermore, the step of using a preset first sliding window to segment the video surveillance data to obtain a plurality of video clips specifically includes:

[0009] A preset video frame rate detection algorithm is used to detect the frame rate of the video surveillance data to obtain video frame rate data of the video surveillance data; based on the video frame rate data, a preset window size of the first sliding window and a window sliding step size are used to perform sliding window segmentation on the video surveillance data to obtain multiple video clips.

[0010] Furthermore, each video clip is instantiated to generate multiple instances, specifically including:

[0011] According to the time length of each video clip, the window size and window sliding step of the second sliding window corresponding to each video clip are determined; based on the window size and window sliding step of the second sliding window, each video clip is slid along the time dimension to obtain multiple instances.

[0012] Furthermore, before the step of using a preset anomaly detection model to perform anomaly detection on multiple instances to obtain a target prediction score, the step further includes:

[0013] Historical video data is obtained, and a first sliding window is used to segment the historical video data to obtain a plurality of historical video clip samples; each historical video clip sample is instantiated to generate a plurality of historical instances; a pre-trained three-dimensional convolutional neural network is used to extract features from each historical instance to obtain a historical feature vector for each historical instance; based on the historical feature vector and a preset sorting loss function, a gradient descent algorithm is used to train a preset multi-instance regression learning model to obtain a preset anomaly detection model, and the sorting loss function is used to measure the extent to which the score of the abnormal instance in each historical instance is higher than the score of the normal instance.

[0014] Further, based on the historical feature vector and the preset ranking loss function, the preset multi-instance regression learning model is trained using a gradient descent algorithm to obtain the trained anomaly detection model, specifically including:

[0015] According to the preset sorting loss function, a preset multi-instance regression learning model is constructed; the historical feature vector is input into the multi-instance regression learning model to obtain a prediction score; based on the prediction score and the label information of the historical video data, the loss value of the sorting loss function is calculated; based on the loss value, the gradient of the parameters of the multi-instance regression learning model is calculated, and the parameters of the multi-instance regression learning model are updated using the gradient, and the steps from obtaining the historical video data to calculating the loss value of the sorting loss function are returned to execute until the loss value of the sorting loss function converges to the loss threshold; the multi-instance regression learning model whose loss value converges to the loss threshold is used as the preset anomaly detection model.

[0016] Furthermore, the steps of using a preset anomaly detection model to perform anomaly detection on multiple instances to obtain a target prediction score of the video surveillance data specifically include:

[0017] A pre-trained three-dimensional convolutional neural network is used to extract features from multiple instances to obtain a feature vector for each instance. The feature vector is input into an anomaly detection model to output a target prediction score for the video surveillance data. If the target prediction score reaches a preset score threshold, it is determined that the video surveillance data is abnormal; if the target prediction score does not reach the score threshold, it is determined that the video surveillance data is not abnormal.

[0018] Furthermore, if the target prediction score reaches a preset score threshold, the step of triggering a preset alarm mechanism to issue an alarm specifically includes:

[0019] If the target prediction score reaches the preset score threshold, the video surveillance data is subjected to scene analysis to obtain the scene information of the video surveillance data; based on the preset scene alarm strategy and scene information, the alarm method of the video surveillance data is determined, and an alarm is issued based on the alarm method.

[0020] In order to solve the above technical problems, the embodiment of the present application further provides a data processing device, which adopts the following technical solution:

[0021] An acquisition module is used to acquire video surveillance data within a preset time period;

[0022] A segmentation module, used to segment the video surveillance data using a preset first sliding window to obtain a plurality of video segments, each of which includes a plurality of video frames;

[0023] An instantiation module, used for instantiating each video clip to generate multiple instances, each instance containing multiple frames of continuous video images;

[0024] A detection module, used to perform anomaly detection on multiple instances using a preset anomaly detection model to obtain a target prediction score of the video surveillance data;

[0025] The alarm module is used to trigger the preset alarm mechanism to issue an alarm if the target prediction score reaches the preset score threshold.

[0026] In order to solve the above technical problems, an embodiment of the present application also provides a computer device, including a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the above data processing method are implemented.

[0027] In order to solve the above technical problems, an embodiment of the present application also provides a computer-readable storage medium, which stores computer-readable instructions. The computer-readable instructions can be executed by at least one processor to enable at least one processor to perform the steps of the above data processing method.

[0028] Compared with the prior art, the embodiments of the present application mainly have the following beneficial effects: by acquiring video surveillance data within a preset time period, and using sliding window technology for segmentation and instantiation processing, the key information in the video is effectively extracted. On this basis, a preset anomaly detection model is adopted, and anomaly detection is regarded as a regression problem, wherein the anomaly detection model is trained by a sorting loss function, which can achieve high-precision recognition of abnormal fragments in video surveillance data. This method not only overcomes the problems of subjective definition of abnormal behavior and unclear marking standards, but also can significantly improve the accuracy of video data detection when abnormal samples are scarce. Compared with the traditional video abnormal behavior detection method based on classification problems, this scheme has stronger adaptability and practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0030] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0031] Figure 2 It is a flowchart of a data processing method provided by this application;

[0032] Figure 3 It is a structural schematic diagram of a data processing device provided by the present application;

[0033] Figure 4 It is a structural schematic diagram of a computer device provided by this application. DETAILED DESCRIPTION

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0035] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0036] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0037] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0038] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0039] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0040] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0041] It should be noted that the data processing method provided in the embodiment of the present application is generally executed by a server, and accordingly, the data processing device is generally set in the server.

[0042] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0043] Continue to refer Figure 2 , shows a flow chart of an embodiment of a data processing method according to the present application. The data processing method comprises the following steps:

[0044] Step S201, obtaining video surveillance data within a preset time period.

[0045] In this embodiment, the electronic device (eg, Figure 1 The server shown in the figure) can obtain video surveillance data through a wired connection or a wireless connection. It should be noted that the above wireless connection method may include but is not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0046] The preset time period refers to a time range that is pre-set and used to obtain video surveillance data. It can usually be determined based on monitoring needs and security risk assessment. For example, it can be set to the entire bank's business hours every day to comprehensively monitor the security status of bank ATMs and entrances.

[0047] Video surveillance data refers to continuous image data captured by surveillance cameras, which may contain information such as timestamps and image content for subsequent analysis and identification of abnormal behavior. For example, surveillance videos of bank ATMs and entrances during peak hours.

[0048] Step S202: Use a preset first sliding window to segment the video surveillance data to obtain a plurality of video segments, each of which includes a plurality of video frames.

[0049] The first sliding window refers to a fixed-length time period or frame range for segmenting video surveillance data, and is used to segment long video data into multiple shorter video segments for subsequent processing.

[0050] The multiple video clips refer to multiple parts of the video surveillance data obtained by segmenting the first sliding window, and each clip contains continuous video frames for subsequent instantiation processing and analysis.

[0051] The multiple video frames refer to the continuous images contained in the video clip, which are the basic components of video data and are used to form dynamic videos. For example, a 30-second video clip may contain hundreds of frames of images.

[0052] Step S203: instantiate each video clip to generate multiple instances, each instance including multiple frames of continuous video images.

[0053] Among them, instantiation processing refers to further dividing the video clip into multiple instances containing multiple frames of continuous video images for input into the anomaly detection model.

[0054] The multiple instances refer to multiple units containing multiple frames of continuous video images obtained through instantiation processing, which are used for testing the anomaly detection model.

[0055] The multiple frames of continuous video images refer to a continuous image sequence that constitutes an instance, which is used to capture and display dynamic changes in the video. For example, an instance contains continuous video frames within 5 seconds.

[0056] Step S204, using a preset anomaly detection model to perform anomaly detection on multiple instances to obtain a target prediction score for the video surveillance data.

[0057] Among them, the anomaly detection model refers to a model that has been trained to identify abnormal behaviors in video surveillance data. The prediction score of normal video data output by the anomaly detection model will be lower than the prediction score of abnormal video. In this way, it is possible to judge whether the predicted video data is abnormal based on the numerical value of the output prediction score. The anomaly detection model can be built based on machine learning or deep learning algorithms. For example, the anomaly detection model can be used to identify theft in front of a bank ATM.

[0058] Anomaly detection refers to the process of using anomaly detection models to analyze video surveillance data, identify and mark abnormal behaviors. For example, identifying suspicious people or abnormal behaviors at the entrance of a bank.

[0059] The target prediction score refers to the total prediction score of the video surveillance data that the anomaly detection model outputs by performing anomaly detection on each instance in the video surveillance data. It is used to determine whether there is abnormal behavior.

[0060] Step S205: If the target prediction score reaches a preset score threshold, a preset alarm mechanism is triggered to issue an alarm.

[0061] The score threshold is the critical value used to determine whether the target prediction score reaches the abnormal standard. When the score exceeds this threshold, it is considered that abnormal behavior exists. For example, if it is set to 0.8, an alarm is triggered when the prediction score exceeds 0.8.

[0062] The alarm mechanism refers to an alarm system that is automatically triggered when the target prediction score reaches the score threshold, which is used to promptly notify relevant personnel and take countermeasures. For example, a text message or email is sent to notify security personnel.

[0063] The embodiment of the present application can effectively extract key information from the video by acquiring video surveillance data within a preset time period and using sliding window technology for segmentation and instantiation processing. On this basis, a preset anomaly detection model is used to regard anomaly detection as a regression problem, wherein the anomaly detection model is trained by a sorting loss function, which can achieve high-precision recognition of abnormal segments in video surveillance data. This method not only overcomes the problems of subjective definition of abnormal behavior and unclear marking standards, but also can significantly improve the accuracy of video data detection when abnormal samples are scarce. Compared with the traditional video abnormal behavior detection method based on classification problems, this solution has stronger adaptability and practicality.

[0064] In some optional implementations of this embodiment, step 202, using a preset first sliding window to segment the video surveillance data to obtain multiple video clips, specifically includes the following steps:

[0065] A preset video frame rate detection algorithm is used to detect the frame rate of the video surveillance data to obtain video frame rate data of the video surveillance data; based on the video frame rate data, a preset window size of the first sliding window and a window sliding step size are used to perform sliding window segmentation on the video surveillance data to obtain multiple video clips.

[0066] The video frame rate detection algorithm refers to an algorithm used to detect the video frame rate (the number of frames displayed per second), which is used to determine the playback speed and smoothness of video surveillance data.

[0067] The video frame rate data refers to the number of frames displayed per second in the video obtained by the video frame rate detection algorithm, which is used to guide subsequent video processing and analysis. For example, the frame rate of video surveillance data is 30fps (30 frames per second).

[0068] The window size of the first sliding window refers to the number of video frames or time length contained in the first sliding window, which is used to determine the size of the segmented video segments, for example, set to 600 frames or 30 seconds.

[0069] The window sliding step of the first sliding window refers to the number of frames or the length of time skipped each time the first sliding window is moved, and is used to control the degree of overlap between video clips. For example, it is set to 300 frames or 15 seconds.

[0070] Sliding window segmentation refers to the process of segmenting video surveillance data into multiple video segments using sliding window technology, which is used to reduce the complexity of data processing and improve analysis efficiency. For example, one day's surveillance data is segmented into 30-second windows and 15-second steps.

[0071] In one example, in the financial field, the security monitoring of bank ATM machines is of vital importance. The technical solution proposed in this embodiment can be applied to the video surveillance data analysis of bank ATM machines. First, a video frame rate detection algorithm is used to detect the frame rate of the video surveillance data collected by the ATM machine, and its video frame rate data is obtained, for example, the frame rate is detected to be 25 frames per second. Subsequently, based on the frame rate data, the first sliding window strategy is used to segment the video data. For example, the window size is set to 10 seconds (i.e., 250 frames, considering that the frame rate is 25 frames per second), and the window sliding step is 5 seconds (i.e., 125 frames). Through such sliding window segmentation, the original video surveillance data can be divided into multiple continuous but non-overlapping video clips, each of which contains 10 seconds of monitoring content. For ATM machine monitoring, the time period in which abnormal transaction behaviors (such as multiple attempts to crack passwords, abnormal withdrawals, etc.) occur can be quickly identified, providing accurate positioning for security personnel, shortening response time, and improving the intelligent level of bank security monitoring.

[0072] The embodiment of the present application can accurately and efficiently obtain the frame rate information of the video surveillance data through the video frame rate detection algorithm, providing a reliable data basis for subsequent video processing. Based on the detected video frame rate data, the video surveillance data is segmented using the first sliding window strategy. This step makes full use of the accuracy of the frame rate data to ensure the continuity and integrity of each video segment in time. At the same time, by flexibly setting the window size and sliding step size, the length and overlap of the video segment can be adjusted according to actual needs to adapt to different analysis scenarios. This frame rate-based sliding window segmentation method not only improves the flexibility and efficiency of video data processing, but also helps to improve the accuracy and reliability of subsequent video content analysis.

[0073] In some optional implementations of this embodiment, in step S203, each video clip is instantiated to generate multiple instances, which specifically includes the following steps:

[0074] According to the time length of each video clip, the window size and window sliding step of the second sliding window corresponding to each video clip are determined; based on the window size and window sliding step of the second sliding window, each video clip is slid along the time dimension to obtain multiple instances.

[0075] The time length refers to the time range corresponding to the video frames contained in each video segment, and is used to describe the duration of each video segment. For example, the time length of a video segment is 30 seconds.

[0076] The second sliding window refers to a sliding window used when instantiating the video clip, and its size and step length are determined according to the time length of the video clip, for example, a 5-second window and a 1-second step length.

[0077] The window size of the second sliding window refers to the number of video frames or time length contained in the second sliding window, which is used to determine the size of each instance, for example, set to 100 frames or 5 seconds.

[0078] The window sliding step of the second sliding window refers to the number of frames or the length of time skipped each time the second sliding window is moved, and is used to control the degree of overlap between instances. For example, it is set to 20 frames or 1 second.

[0079] Among them, the time dimension refers to the direction of time flow in video data, which is used to describe the sequence and relationship between video frames.

[0080] Sliding processing refers to the process of segmenting and processing video clips using sliding window technology to extract key information and features. For example, sliding window processing is performed on video clips along the time dimension to generate multiple instances.

[0081] In an example, the technical solution of this embodiment is described in detail by taking the bank ATM surveillance video as an example. After the surveillance video is divided into multiple video clips, for each video clip, the window size and window sliding step of the second sliding window are dynamically determined according to its time length. Based on the determined window size and sliding step, each video clip is slid along the time dimension to obtain multiple instances containing video content within a specific time range. These instances not only retain the key information of the original video, but also reduce the amount of data, which is convenient for subsequent analysis and storage.

[0082] The embodiment of the present application can dynamically determine the window size and sliding step size of the second sliding window that matches each video clip according to the time length characteristics of each video clip. This strategy significantly enhances the flexibility and pertinence of video processing. By sliding the video clips along the time dimension, not only the continuity and integrity of each instance in time are ensured, but also the fine division of the video content is achieved. This fine division helps to more accurately capture the key events and details in the video, and improves the accuracy and efficiency of video content analysis. At the same time, since the size and step size of the sliding window are determined according to the time length of the video clip, the method can adaptively process video clips of different lengths, avoiding the information redundancy or omission problems caused by the fixed window size in the traditional method, thereby optimizing the overall effect of video processing and providing more reliable data support for subsequent applications such as video content analysis and event detection.

[0083] In some optional implementations of this embodiment, in step S204, before using a preset anomaly detection model to perform anomaly detection on multiple instances and obtaining a target prediction score, the following steps are specifically included:

[0084] Historical video data is obtained, and a first sliding window is used to segment the historical video data to obtain a plurality of historical video clip samples; each historical video clip sample is instantiated to generate a plurality of historical instances; a pre-trained three-dimensional convolutional neural network is used to extract features from each historical instance to obtain a historical feature vector for each historical instance; based on the historical feature vector and a preset sorting loss function, a gradient descent algorithm is used to train a preset multi-instance regression learning model to obtain a preset anomaly detection model, and the sorting loss function is used to measure the extent to which the score of the abnormal instance in each historical instance is higher than the score of the normal instance.

[0085] The historical video data refers to the historical video surveillance data used to train the multi-instance regression learning model to obtain the anomaly detection model, including known abnormal video samples and normal video samples. For example, the surveillance video at the entrance of a bank in the past year.

[0086] The multiple historical video clip samples refer to multiple video clip samples obtained by segmenting the historical video data through the first sliding window, which are used to train the multi-instance regression learning model to obtain the anomaly detection model. For example, the historical video data is segmented into hundreds of 30-second segment samples.

[0087] The multiple historical instances refer to multiple instance samples obtained by instantiating the historical video clip samples, which are used to train the multi-instance regression learning model to obtain the anomaly detection model. For example, each clip sample is divided into multiple 5-second instance samples.

[0088] Among them, a 3D convolutional neural network refers to a convolutional neural network that can process video data (including the time dimension) and is used to extract spatiotemporal features in the video (e.g., each historical instance, each instance). For example, it is used to identify actions and events in the video.

[0089] The historical feature vector refers to the vector representation obtained by extracting features from historical instances using a three-dimensional convolutional neural network. For example, each historical instance is extracted into a 128-dimensional feature vector.

[0090] Among them, the ranking loss function refers to the loss function used to train the multi-instance regression learning model, which optimizes the model performance by measuring the degree to which the scores of abnormal instances are higher than the scores of normal instances. For example, the ranking-based loss function can make the model more accurate in identifying abnormal behaviors.

[0091] The gradient descent algorithm is an iterative algorithm used to optimize the parameters of a machine learning model. It minimizes the loss value by calculating the gradient of the loss function with respect to the parameters and updating the parameters in the opposite direction of the gradient. For example, the gradient descent algorithm is used to optimize the model parameters when training a multi-instance regression learning model.

[0092] Among them, the multi-instance regression learning model refers to a machine learning model that can process multiple instances and output regression scores. For example, an anomaly detection model based on the multi-instance regression learning model can identify abnormal behaviors in surveillance video data.

[0093] Among them, abnormal instances refer to instance samples containing abnormal behaviors, which are used as positive samples when training multi-instance regression learning models, such as instances corresponding to theft behaviors in front of bank ATMs.

[0094] Among them, normal instances refer to instance samples containing normal behaviors, which are used as negative samples when training multi-instance regression learning models. For example, the instance corresponding to the normal flow of people at the bank entrance.

[0095] In one example, the technical solution of this embodiment is described in detail by taking the surveillance video of a city street as an example. First, historical video data containing normal and abnormal events are obtained, and these data come from multiple surveillance cameras on city streets. The historical video data is segmented using a preset first sliding window (such as a 10-second window, sliding for 1 second each time) to obtain multiple historical video clip samples. These samples contain both normal events (such as normal walking of pedestrians and normal driving of vehicles) and abnormal events (such as pedestrian running and vehicle collision). Next, each historical video clip sample is instantiated to generate multiple historical instances. Then, a pre-trained three-dimensional convolutional neural network is used to extract features from each historical instance to obtain the historical feature vector of each historical instance. Based on the historical feature vector and the preset sorting loss function, the gradient descent algorithm is used to train the preset multi-instance regression learning model. The sorting loss function is designed to measure the degree to which the score of the abnormal instance in each historical instance is higher than the score of the normal instance, thereby guiding the model to learn the ability to distinguish between normal and abnormal events. During the training process, by continuously adjusting the model parameters, the model's prediction score for abnormal events is gradually higher than that for normal events.

[0096] The embodiment of the present application can effectively segment the historical video data through the first sliding window, obtain a rich sample of historical video clips, and lay a solid foundation for subsequent feature extraction and model training. Each historical video clip sample is instantiated to generate historical instances that can be processed by the deep learning model (multi-instance regression learning model), thereby improving the flexibility and efficiency of data processing. The pre-trained three-dimensional convolutional neural network is used to extract features from historical instances, and a historical feature vector containing spatiotemporal information is obtained, which enhances the model's ability to understand the video content. Based on the historical feature vectors and the sorting loss function, the gradient descent algorithm is used to train the multi-instance regression learning model, and an anomaly detection model is successfully constructed. The model can accurately measure the degree to which the scores of abnormal instances in historical instances are higher than those of normal instances, realize the effective identification of abnormal video events, improve the accuracy and reliability of anomaly detection, and provide strong support for video surveillance and data security.

[0097] In some optional implementations of this embodiment, the step of “based on the historical feature vector and the preset ranking loss function, using the gradient descent algorithm, training the preset multi-instance regression learning model to obtain the trained anomaly detection model” specifically includes the following steps:

[0098] According to the preset sorting loss function, a preset multi-instance regression learning model is constructed; the historical feature vector is input into the multi-instance regression learning model to obtain a prediction score; based on the prediction score and the label information of the historical video data, the loss value of the sorting loss function is calculated; based on the loss value, the gradient of the parameters of the multi-instance regression learning model is calculated, and the parameters of the multi-instance regression learning model are updated using the gradient, and the steps from obtaining the historical video data to calculating the loss value of the sorting loss function are returned to execute until the loss value of the sorting loss function converges to the loss threshold; the multi-instance regression learning model whose loss value converges to the loss threshold is used as the preset anomaly detection model.

[0099] The prediction score refers to the output value of the multi-instance regression learning model to evaluate the abnormality of the input instance, which is used to determine whether the instance contains abnormal behavior. For example, a higher score indicates a greater probability of abnormal behavior in the instance.

[0100] The label information refers to the known abnormal or normal behavior label of each historical instance in the historical video data, which is used to provide supervision information when training the multi-instance regression learning model. For example, each historical instance is labeled as abnormal or normal.

[0101] The loss value refers to the difference between the model prediction score calculated by the ranking loss function and the true label information, which is used to evaluate model performance and guide model optimization. For example, the smaller the loss value, the better the model performance.

[0102] The gradient refers to the partial derivative of the loss function with respect to the model parameters, which indicates the rate and direction at which the loss value changes with the parameters and is used to guide the update of the model parameters. For example, by calculating the gradient, we can determine in which direction the model parameters should be adjusted to reduce the loss value.

[0103] Convergence refers to the state in which the loss value gradually decreases and tends to be stable during model training, indicating that the model has learned the inherent laws and characteristics of the data. For example, when the loss value no longer decreases significantly after multiple iterations, the model is considered to have converged.

[0104] The loss threshold refers to the critical loss value used to determine whether the multi-instance regression learning model has converged. When the loss value is less than this threshold, it is considered that the multi-instance regression learning model has been trained and has reached the expected performance level. For example, when it is set to 0.01, it means that the multi-instance regression learning model is considered to have converged when the loss value drops below 0.01 during the training process.

[0105] In one example, first, a large amount of historical video data is obtained from the monitoring system, and these data should include a variety of normal events and abnormal events. The historical video data is segmented using a preset first sliding window (such as a 5-second window, sliding for 1 second each time) to obtain multiple historical video clip samples. These samples include both normal events (such as normal walking of pedestrians and normal driving of vehicles) and abnormal events (such as pedestrian running, vehicle collision, etc.). Each historical video clip sample is instantiated to generate multiple historical instances. Then, a pre-trained three-dimensional convolutional neural network is used to extract features from each historical instance to obtain a historical feature vector for each historical instance. These feature vectors can capture the spatiotemporal information in the video and provide strong support for subsequent anomaly detection. According to the preset sorting loss function, a multi-instance regression learning model is constructed. The historical feature vector is input into the multi-instance regression learning model to obtain a prediction score. Then, based on the prediction score and the label information (normal or abnormal) of the historical video data, the loss value of the sorting loss function is calculated. Then, based on the loss value, the gradient of the model parameters is calculated using the back propagation algorithm, and the model parameters are updated using the gradient descent algorithm. Repeat the above steps until the loss value of the sorting loss function converges to the preset loss threshold.

[0106] The embodiment of the present application can make full use of the preset ranking loss function to guide the construction and optimization of the multi-instance regression learning model through a carefully designed process. First, the model can accurately predict the score based on the historical feature vector, which benefits from the model's deep mining ability for complex data relationships. Subsequently, by comparing the predicted score with the real label information of the historical video data, the loss value is accurately calculated. This process ensures the quantification and tracking of the model error. Furthermore, the gradient of the model parameters is inferred from the loss value, and the model parameters are iteratively updated accordingly. This refined parameter tuning strategy significantly improves the fitting accuracy and generalization ability of the model. Iterate until the loss value converges to the preset loss threshold, which not only ensures the stability of the training process, but also ensures that the final anomaly detection model has a high degree of accuracy and reliability.

[0107] In some optional implementations of this embodiment, step S204, using a preset anomaly detection model to perform anomaly detection on multiple instances to obtain a target prediction score for the video surveillance data, may specifically include the following steps:

[0108] A pre-trained three-dimensional convolutional neural network is used to extract features from multiple instances to obtain a feature vector for each instance. The feature vector is input into an anomaly detection model to output a target prediction score for the video surveillance data. If the target prediction score reaches a preset score threshold, it is determined that the video surveillance data is abnormal; if the target prediction score does not reach the score threshold, it is determined that the video surveillance data is not abnormal.

[0109] The feature vector refers to the vector representation extracted from the video instance by the feature extraction algorithm, which is used to describe the key information and features in the video instance. For example, each video instance is extracted into a 128-dimensional vector containing spatiotemporal features.

[0110] In one example, a pre-trained three-dimensional convolutional neural network (3D Convolutional Neural Network, 3D-CNN) is used to extract features from multiple video instances. 3D-CNN has significant advantages in the field of video processing because it can capture the temporal and spatial features of video data at the same time. Specifically, multiple instances can be input into the pre-trained 3D-CNN model in sequence, and deep features are extracted for each instance through the network's convolutional layer, pooling layer, and fully connected layer structures, and finally a feature vector for each instance is obtained. Next, the extracted feature vector is input into the anomaly detection model, and the target prediction score is output. Then, the target prediction score is compared with the preset score threshold to determine whether the video surveillance data is abnormal. If the target prediction score reaches or exceeds the score threshold, it is determined that the video surveillance data is abnormal; if the target prediction score is lower than the score threshold, it is determined that the video surveillance data is not abnormal.

[0111] The embodiment of the present application can extract features from multiple instances through a pre-trained three-dimensional convolutional neural network, which can make full use of the spatiotemporal information of video data to generate accurate feature vectors rich in key information. These feature vectors, as inputs to the anomaly detection model, can enable the model to more accurately understand the content of the video surveillance data and output a prediction score closely related to abnormal behavior. By setting a reasonable score threshold, it is possible to accurately determine the abnormal behavior of video surveillance data, effectively distinguish normal behavior from abnormal behavior, and reduce the occurrence of false positives and false negatives.

[0112] In some optional implementations of this embodiment, step S205, if the target prediction score reaches a preset score threshold, triggering a preset alarm mechanism to issue an alarm, specifically includes the following steps:

[0113] If the target prediction score reaches the preset score threshold, the video surveillance data is subjected to scene analysis to obtain the scene information of the video surveillance data; based on the preset scene alarm strategy and scene information, the alarm method of the video surveillance data is determined, and an alarm is issued based on the alarm method.

[0114] Scene parsing refers to the process of in-depth analysis of video surveillance data to extract scene information and contextual relationships, which is used to improve the accuracy and robustness of anomaly detection. For example, scene parsing can identify different areas and activity patterns of people at the entrance of a bank.

[0115] The scene information refers to the scene description and contextual relationship information in the video surveillance data obtained through scene analysis, which is used to assist in the anomaly detection task, such as the density of people at the bank entrance and the behavior patterns of people.

[0116] The scenario alarm strategy refers to the alarm rules and methods formulated based on scenario information and anomaly detection results, which are used to guide the triggering and response methods of the alarm system. For example, when a suspicious person appears at the bank entrance, an audible and visual alarm is triggered and the security personnel are notified.

[0117] The alarm method refers to the specific method and means taken when the alarm system is triggered, which is used to promptly notify relevant personnel and take countermeasures. For example, alarms are issued through sound and light alarms, text message notifications, phone calls, etc.

[0118] In one example, if the target prediction score is greater than or equal to the preset score threshold, the scene analysis module is triggered. The scene analysis module can use deep learning technology, such as semantic segmentation or target detection, to conduct in-depth analysis of video surveillance data and extract key scene information, such as the number of people, the number of vehicles, the appearance of specific objects (such as lost packages), crowd behavior patterns, etc. This information provides a key basis for the subsequent alarm strategy formulation. Then, the scene information obtained by the analysis is matched according to the preset scene alarm strategy. For example, if the policy stipulates that "when a sudden increase in the number of people is detected in the bank's surveillance video and accompanied by an abnormal situation of long-term stay, an audio-visual alarm is used and a text message is sent to the security personnel's mobile phone", then when these conditions are met, the corresponding alarm method will be automatically executed. In a specific case, a bank encountered a suspicious person wandering around and attempting to commit a crime. The target prediction score was identified as abnormal through the anomaly detection model. Then the scene analysis module accurately identified the abnormal increase in the number of people and the long-term stay behavior, and quickly started the audio-visual alarm, and sent a text message alarm to the security personnel, effectively preventing the occurrence of criminal behavior.

[0119] In the embodiment of the present application, when the target prediction score reaches a preset score threshold, the deep scene analysis of the video surveillance data is quickly initiated to accurately capture key scene information in the video, such as personnel flow, object status, and environmental changes. This step not only enhances the ability to understand the video content, but also lays a solid foundation for the precise matching of subsequent alarm strategies. Based on detailed scene information and preset scene alarm strategies, the most appropriate alarm method can be intelligently selected, whether it is an audio-visual alarm, SMS notification, or remote alarm, to ensure that abnormal information is conveyed to relevant personnel at the first time. This highly intelligent alarm mechanism not only improves the accuracy and timeliness of the alarm, but also effectively avoids unnecessary interference and false alarms, thereby greatly enhancing the overall performance and user experience of the video surveillance system.

[0120] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned video surveillance data and target prediction scores, the above-mentioned video surveillance data and target prediction scores can also be stored in a blockchain node.

[0121] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0122] The embodiments of the present application can build and optimize related models and networks, such as prediction models, based on artificial intelligence technology. Among them, artificial intelligence (AI) models are the crystallization of theory and practice that simulate the decision-making process of human intelligence through algorithms and data analysis to solve complex problems, predict future trends, or realize automation tasks. These models use a large amount of historical data and real-time information, and are trained and optimized through a specific algorithm framework to achieve efficient, accurate and reliable performance.

[0123] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0124] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0125] Further references Figure 3 , as a response to the above Figure 2 In order to realize the method shown in the figure, the present application provides an embodiment of a data processing device, and the device embodiment is Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0126] like Figure 3 As shown, the data processing device 400 of this embodiment includes: an acquisition module 401, a segmentation module 402, an instantiation module 403, a detection module 404 and an alarm module 405. Among them:

[0127] The acquisition module 401 is used to acquire video surveillance data within a preset time period;

[0128] A segmentation module 402 is used to segment the video surveillance data using a preset first sliding window to obtain a plurality of video segments, each of which includes a plurality of video frames;

[0129] An instantiation module 403 is used to instantiate each video clip to generate multiple instances, each instance containing multiple frames of continuous video images;

[0130] The detection module 404 is used to use a preset anomaly detection model to perform anomaly detection on multiple instances to obtain a target prediction score of the video surveillance data;

[0131] The alarm module 405 is used to trigger a preset alarm mechanism to issue an alarm if the target prediction score reaches a preset score threshold.

[0132] The embodiment of the present application can effectively extract key information from the video by acquiring video surveillance data within a preset time period and using sliding window technology for segmentation and instantiation processing. On this basis, a preset anomaly detection model is used to regard anomaly detection as a regression problem, wherein the anomaly detection model is trained by a sorting loss function, which can achieve high-precision recognition of abnormal segments in video surveillance data. This method not only overcomes the problems of subjective definition of abnormal behavior and unclear marking standards, but also can significantly improve the accuracy of video data detection when abnormal samples are scarce. Compared with the traditional video abnormal behavior detection method based on classification problems, this solution has stronger adaptability and practicality.

[0133] In one embodiment, the segmentation module 402 includes:

[0134] The detection submodule is used to detect the frame rate of the video surveillance data using a preset video frame rate detection algorithm to obtain video frame rate data of the video surveillance data;

[0135] The segmentation submodule is used to perform sliding window segmentation on the video monitoring data based on the video frame rate data, using a preset window size and window sliding step of the first sliding window to obtain multiple video clips.

[0136] The embodiment of the present application can accurately and efficiently obtain the frame rate information of the video surveillance data through the video frame rate detection algorithm, providing a reliable data basis for subsequent video processing. Based on the detected video frame rate data, the video surveillance data is segmented using the first sliding window strategy. This step makes full use of the accuracy of the frame rate data to ensure the continuity and integrity of each video segment in time. At the same time, by flexibly setting the window size and sliding step size, the length and overlap of the video segment can be adjusted according to actual needs to adapt to different analysis scenarios. This frame rate-based sliding window segmentation method not only improves the flexibility and efficiency of video data processing, but also helps to improve the accuracy and reliability of subsequent video content analysis.

[0137] In one embodiment, the instantiation module 403 includes:

[0138] A determination submodule, used to determine the window size and window sliding step length of the second sliding window corresponding to each video segment according to the time length of each video segment;

[0139] The sliding submodule is used to perform sliding processing on each video segment along the time dimension based on the window size and the window sliding step of the second sliding window to obtain multiple instances.

[0140] The embodiment of the present application can dynamically determine the window size and sliding step size of the second sliding window that matches each video clip according to the time length characteristics of each video clip. This strategy significantly enhances the flexibility and pertinence of video processing. By sliding the video clips along the time dimension, not only the continuity and integrity of each instance in time are ensured, but also the fine division of the video content is achieved. This fine division helps to more accurately capture the key events and details in the video, and improves the accuracy and efficiency of video content analysis. At the same time, since the size and step size of the sliding window are determined according to the time length of the video clip, the method can adaptively process video clips of different lengths, avoiding the information redundancy or omission problems caused by the fixed window size in the traditional method, thereby optimizing the overall effect of video processing and providing more reliable data support for subsequent applications such as video content analysis and event detection.

[0141] In one embodiment, the data processing device may further include:

[0142] A data acquisition module is used to acquire historical video data, and to segment the historical video data using a first sliding window to obtain a plurality of historical video segment samples;

[0143] A processing module, used for instantiating each historical video clip sample to generate multiple historical instances;

[0144] An extraction module, for extracting features from each historical instance using a pre-trained three-dimensional convolutional neural network to obtain a historical feature vector for each historical instance;

[0145] The training module is used to train the preset multi-instance regression learning model based on the historical feature vector and the preset ranking loss function using the gradient descent algorithm to obtain the preset anomaly detection model. The ranking loss function is used to measure the extent to which the score of the abnormal instance in each historical instance is higher than the score of the normal instance.

[0146] The embodiment of the present application can effectively segment the historical video data through the first sliding window, obtain a rich sample of historical video clips, and lay a solid foundation for subsequent feature extraction and model training. Each historical video clip sample is instantiated to generate historical instances that can be processed by the deep learning model (multi-instance regression learning model), thereby improving the flexibility and efficiency of data processing. The pre-trained three-dimensional convolutional neural network is used to extract features from historical instances, and a historical feature vector containing spatiotemporal information is obtained, which enhances the model's ability to understand the video content. Based on the historical feature vectors and the sorting loss function, the gradient descent algorithm is used to train the multi-instance regression learning model, and an anomaly detection model is successfully constructed. The model can accurately measure the degree to which the scores of abnormal instances in historical instances are higher than those of normal instances, realize the effective identification of abnormal video events, improve the accuracy and reliability of anomaly detection, and provide strong support for video surveillance and data security.

[0147] In one embodiment, the training module includes:

[0148] A construction submodule is used to construct a preset multi-instance regression learning model according to a preset ranking loss function;

[0149] The input submodule is used to input the historical feature vector into the multi-instance regression learning model to obtain the prediction score;

[0150] A calculation submodule, used to calculate the loss value of the ranking loss function based on the prediction score and the label information of the historical video data;

[0151] An update submodule, used to calculate the gradient of the parameters of the multi-instance regression learning model based on the loss value, update the parameters of the multi-instance regression learning model using the gradient, and return to execute the steps of obtaining historical video data to calculating the loss value of the sorting loss function until the loss value of the sorting loss function converges to the loss threshold;

[0152] As a submodule, a multi-instance regression learning model is used to converge the loss value to the loss threshold as a preset anomaly detection model.

[0153] The embodiment of the present application can make full use of the preset ranking loss function to guide the construction and optimization of the multi-instance regression learning model through a carefully designed process. First, the model can accurately predict the score based on the historical feature vector, which benefits from the model's deep mining ability for complex data relationships. Subsequently, by comparing the predicted score with the real label information of the historical video data, the loss value is accurately calculated. This process ensures the quantification and tracking of the model error. Furthermore, the gradient of the model parameters is inferred from the loss value, and the model parameters are iteratively updated accordingly. This refined parameter tuning strategy significantly improves the fitting accuracy and generalization ability of the model. Iterate until the loss value converges to the preset loss threshold, which not only ensures the stability of the training process, but also ensures that the final anomaly detection model has a high degree of accuracy and reliability.

[0154] In one embodiment, the detection module 404 includes:

[0155] An extraction submodule, for extracting features from multiple instances using a pre-trained three-dimensional convolutional neural network to obtain a feature vector for each instance;

[0156] The output submodule is used to input the feature vector into the anomaly detection model and output the target prediction score of the video surveillance data;

[0157] The determination submodule is used to determine that the video surveillance data is abnormal if the target prediction score reaches a preset score threshold, and to determine that the video surveillance data is not abnormal if the target prediction score does not reach the score threshold.

[0158] The embodiment of the present application can extract features from multiple instances through a pre-trained three-dimensional convolutional neural network, which can make full use of the spatiotemporal information of video data to generate accurate feature vectors rich in key information. These feature vectors, as inputs to the anomaly detection model, can enable the model to more accurately understand the content of the video surveillance data and output a prediction score closely related to abnormal behavior. By setting a reasonable score threshold, it is possible to accurately determine the abnormal behavior of video surveillance data, effectively distinguish normal behavior from abnormal behavior, and reduce the occurrence of false positives and false negatives.

[0159] In one embodiment, the alarm module 405 includes:

[0160] The parsing submodule is used to perform scene parsing on the video surveillance data to obtain scene information of the video surveillance data if the target prediction score reaches a preset score threshold;

[0161] The alarm submodule is used to determine the alarm method of the video surveillance data based on the preset scene alarm strategy and scene information, and to issue an alarm based on the alarm method.

[0162] In the embodiment of the present application, when the target prediction score reaches a preset score threshold, the deep scene analysis of the video surveillance data is quickly initiated to accurately capture key scene information in the video, such as personnel flow, object status, and environmental changes. This step not only enhances the ability to understand the video content, but also lays a solid foundation for the precise matching of subsequent alarm strategies. Based on detailed scene information and preset scene alarm strategies, the most appropriate alarm method can be intelligently selected, whether it is an audio-visual alarm, SMS notification, or remote alarm, to ensure that abnormal information is conveyed to relevant personnel at the first time. This highly intelligent alarm mechanism not only improves the accuracy and timeliness of the alarm, but also effectively avoids unnecessary interference and false alarms, thereby greatly enhancing the overall performance and user experience of the video surveillance system.

[0163] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0164] The computer device 6 includes a memory 61, a processor 62, and a network interface 63 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 6 having a memory 61, a processor 62, and a network interface 63, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASIC), programmable gate arrays (FPGA), digital signal processors (DSP), embedded devices, etc.

[0165] Computer devices can be computing devices such as desktop computers, notebooks, PDAs, and cloud servers. Computer devices can interact with users through keyboards, mice, remote controls, touch pads, or voice control devices.

[0166] The memory 61 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 61 can be an internal storage unit of the computer device 6, such as a hard disk or memory of the computer device 6. In other embodiments, the memory 61 can also be an external storage device of the computer device 6, such as a plug-in hard disk equipped on the computer device 6, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 61 can also include both the internal storage unit of the computer device 6 and its external storage device. In this embodiment, the memory 61 is generally used to store the operating system and various application software installed on the computer device 6, such as computer-readable instructions of data processing methods, etc. In addition, the memory 61 can also be used to temporarily store various types of data that have been output or are to be output.

[0167] The processor 62 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 62 is generally used to control the overall operation of the computer device 6. In this embodiment, the processor 62 is used to run computer-readable instructions stored in the memory 61 or process data, such as computer-readable instructions for running a data processing method.

[0168] The network interface 63 may include a wireless network interface or a wired network interface, and the network interface 63 is generally used to establish a communication connection between the computer device 6 and other electronic devices.

[0169] The embodiment of the present application can effectively extract key information from the video by acquiring video surveillance data within a preset time period and using sliding window technology for segmentation and instantiation processing. On this basis, a preset anomaly detection model is used to regard anomaly detection as a regression problem, wherein the anomaly detection model is trained by a sorting loss function, which can achieve high-precision recognition of abnormal segments in video surveillance data. This method not only overcomes the problems of subjective definition of abnormal behavior and unclear marking standards, but also can significantly improve the accuracy of video data detection when abnormal samples are scarce. Compared with the traditional video abnormal behavior detection method based on classification problems, this solution has stronger adaptability and practicality.

[0170] The present application also provides another embodiment, namely, providing a computer-readable storage medium, which stores computer-readable instructions. The computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the data processing method as described above.

[0171] The embodiment of the present application can effectively extract key information from the video by acquiring video surveillance data within a preset time period and using sliding window technology for segmentation and instantiation processing. On this basis, a preset anomaly detection model is used to regard anomaly detection as a regression problem, wherein the anomaly detection model is trained by a sorting loss function, which can achieve high-precision recognition of abnormal segments in video surveillance data. This method not only overcomes the problems of subjective definition of abnormal behavior and unclear marking standards, but also can significantly improve the accuracy of video data detection when abnormal samples are scarce. Compared with the traditional video abnormal behavior detection method based on classification problems, this solution has stronger adaptability and practicality.

[0172] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0173] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A data processing method, characterized in that: The steps include: Obtain video surveillance data within a preset time period; Using a preset first sliding window, the video surveillance data is segmented to obtain a plurality of video segments, each of which includes a plurality of video frames; Performing instantiation processing on each video clip to generate multiple instances, each instance containing multiple frames of continuous video images; Using a preset anomaly detection model to perform anomaly detection on the multiple instances to obtain a target prediction score for the video surveillance data; If the target prediction score reaches a preset score threshold, a preset alarm mechanism is triggered to issue an alarm.

2. The method according to claim 1, characterized in that The step of using a preset first sliding window to segment the video surveillance data to obtain a plurality of video clips specifically includes: Using a preset video frame rate detection algorithm to detect the frame rate of the video surveillance data, and obtain video frame rate data of the video surveillance data; Based on the video frame rate data, a preset window size and window sliding step of a first sliding window are used to perform sliding window segmentation on the video surveillance data to obtain a plurality of video clips.

3. The method according to claim 1, characterized in that: The step of instantiating each video clip to generate multiple instances specifically includes: Determining the window size and window sliding step length of the second sliding window corresponding to each video segment according to the time length of each video segment; Based on the window size and the window sliding step of the second sliding window, sliding processing is performed on each video segment along the time dimension to obtain multiple instances.

4. The method according to claim 1, characterized in that Before the step of using a preset anomaly detection model to perform anomaly detection on the multiple instances to obtain a target prediction score, the method further includes: Acquire historical video data, and use the first sliding window to segment the historical video data to obtain multiple historical video segment samples; Perform instantiation processing on each historical video clip sample to generate multiple historical instances; Using a pre-trained three-dimensional convolutional neural network, extracting features from each historical instance to obtain a historical feature vector for each historical instance; Based on the historical feature vectors and the preset ranking loss function, the preset multi-instance regression learning model is trained using a gradient descent algorithm to obtain a preset anomaly detection model. The ranking loss function is used to measure the extent to which the score of the abnormal instance in each historical instance is higher than the score of the normal instance.

5. The method according to claim 4, characterized in that The step of training the preset multi-instance regression learning model based on the historical feature vector and the preset ranking loss function using a gradient descent algorithm to obtain a trained anomaly detection model specifically includes: According to the preset ranking loss function, a preset multi-instance regression learning model is constructed; Inputting the historical feature vector into the multi-instance regression learning model to obtain a prediction score; Calculating a loss value of the ranking loss function based on the prediction score and the label information of the historical video data; Based on the loss value, calculate the gradient of the parameters of the multi-instance regression learning model, adopt the gradient, update the parameters of the multi-instance regression learning model, and return to execute the steps of obtaining historical video data to calculating the loss value of the sorting loss function until the loss value of the sorting loss function converges to the loss threshold; The multi-instance regression learning model whose loss value converges to a loss threshold is used as a preset anomaly detection model.

6. The method according to claim 4, characterized in that The step of using a preset anomaly detection model to perform anomaly detection on the multiple instances to obtain a target prediction score of the video surveillance data specifically includes: Using a pre-trained three-dimensional convolutional neural network, extracting features from the multiple instances to obtain a feature vector for each instance; Inputting the feature vector into the anomaly detection model and outputting a target prediction score of the video surveillance data; If the target prediction score reaches a preset score threshold, it is determined that the video surveillance data has an abnormality; if the target prediction score does not reach the score threshold, it is determined that the video surveillance data has no abnormality.

7. The method according to claim 1, characterized in that If the target prediction score reaches a preset score threshold, the step of triggering a preset alarm mechanism to issue an alarm specifically includes: If the target prediction score reaches a preset score threshold, scene analysis is performed on the video surveillance data to obtain scene information of the video surveillance data; Based on the preset scene alarm strategy and the scene information, an alarm method of the video surveillance data is determined, and an alarm is issued based on the alarm method.

8. A data processing device, characterized in that: include: An acquisition module is used to acquire video surveillance data within a preset time period; A segmentation module, configured to segment the video surveillance data using a preset first sliding window to obtain a plurality of video segments, each of which includes a plurality of video frames; An instantiation module, used for instantiating each video clip to generate multiple instances, each instance containing multiple frames of continuous video images; A detection module, configured to use a preset anomaly detection model to perform anomaly detection on the multiple instances to obtain a target prediction score of the video surveillance data; The alarm module is used to trigger a preset alarm mechanism to issue an alarm if the target prediction score reaches a preset score threshold.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the data processing method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the data processing method according to any one of claims 1 to 7 are implemented.

Images