Supplementary storage method and device for abnormal storage of video in a monitoring platform

Abstract

An embodiment of the present application provides a supplementary storage method and device for abnormal storage of video in a monitoring platform. The method includes: by reading the video information of a specified camera, comparing the video time in the camera with the preset video monitoring platform video time to obtain the vulnerable video time and continuous video time of the video in the monitoring platform, reading the video information of the video file corresponding to the continuous video time, calculating the bitstream frame rate deviation by calculating the frame rate of the video information, detecting the bitstream duration of the video information through a set bitstream duration detection model to obtain the bitstream duration deviation, judging the abnormal video time within the continuous video time through the bitstream frame rate deviation and the bitstream duration deviation, and then sending a video download request to the camera for abnormal video supplementary storage for the vulnerable video time and the abnormal video time within the continuous video time to obtain a complete monitored video file. The present application can improve the detection efficiency and accuracy of abnormal monitored video recordings.

Description

Technical Field

[0001] This application relates to the field of data processing, and specifically to a supplementary storage method and device for abnormal storage of videos in a monitoring platform. Background Art

[0002] In existing video surveillance systems, the storage of video recordings usually adopts a combination of on-board SD cards (memory cards) in front-end cameras and centralized storage at the back-end. The front-end cameras perform short-term storage through SD cards, with an expected storage time of approximately 3 days, and manage the storage space in a cyclic overwrite manner. The back-end performs centralized storage, with a relatively longer storage time, usually capable of saving recordings for about 3 months. However, this storage method faces some challenges and problems in practical applications.

[0003] Firstly, due to various reasons such as network problems and server failures, the recordings stored centrally at the back-end may not be saved completely. When an abnormal situation occurs and the recordings need to be retrieved, if the centrally stored recordings are incomplete, it will seriously affect the investigation and handling of the incident.

[0004] Secondly, even if the centralized storage at the back-end is normal, due to the short storage time of the SD cards in the front-end cameras and the cyclic overwrite method, once the recordings on the SD cards have been overwritten before an abnormality is detected and measures are taken, the centralized storage cannot effectively supplement the lost recordings.

[0005] In addition, there are obvious defects in the existing technology for handling lost recordings. On the one hand, supplementing recordings based on the recorded time may result in inaccurate recordings because there may be missing recordings in the middle that cannot be queried through the records. On the other hand, for lost recordings, currently, they are mainly queried manually on the camera web pages, which is not only inefficient but also very inconvenient for users in actual operations because the camera IP, account, and password information may be unknown, or the camera network is not within the office network, etc.

[0006] In summary, the existing video recording storage methods have problems such as inability to guarantee the integrity of recordings, difficulty in supplementing lost recordings, and inconvenience of manual query methods, and there is an urgent need for a more efficient and accurate video recording supplementary storage method to solve these problems. Summary of the Invention

[0007] In view of the problems in the existing technology, this application provides a supplementary storage method and device for abnormal storage of videos in a monitoring platform, which can improve the detection efficiency and accuracy of abnormal monitoring video recordings.

[0008] To solve at least one of the above problems, this application provides the following technical solutions:

[0009] In a first aspect, the present application provides a supplementary storage method for abnormal storage of video in a monitoring platform, including:

[0010] Read the video information of the camera according to a preset video recording detection rule, compare the camera video recording time obtained after the video information reading operation with the preset video monitoring platform video recording time, and determine the corresponding monitoring video recording schedule. Among them, the monitoring video recording schedule includes monitoring vulnerability video recording time and monitoring continuous video recording time;

[0011] Perform a video information reading operation on the video file corresponding to the monitoring continuous video recording time to obtain the corresponding continuous video data, perform a bitstream frame rate calculation operation on the continuous video data to determine the corresponding actual bitstream frame rate, compare the actual bitstream frame rate with the original bitstream frame rate of the camera to determine the corresponding bitstream frame rate deviation, and perform a bitstream duration detection operation on the continuous video data according to a set bitstream duration detection model to determine the corresponding bitstream duration deviation. Among them, the set bitstream duration detection model is obtained by calculating comprehensive features through a progressive fusion algorithm based on target features, contour features, and character features, and inputting the comprehensive features into a preset neural network model for model training;

[0012] Determine the corresponding abnormal video data and the monitoring abnormal video recording time corresponding to the abnormal video data according to the bitstream frame rate deviation and the bitstream duration deviation, and send a video recording download request to the camera according to the monitoring vulnerability video recording time and the monitoring abnormal video recording time. After the camera receives the video recording download request, send the video files corresponding to the monitoring vulnerability video recording time and the monitoring abnormal video recording time to the video monitoring platform for abnormal video supplementary storage operation to determine the corresponding complete monitoring video file.

[0013] Further, before performing the video information reading operation on the camera according to the preset video recording detection rule, it includes:

[0014] Perform a unique coding operation on all cameras to determine the corresponding camera coding;

[0015] Determine the corresponding video recording detection rule according to the camera coding, preset video recording detection interval, preset video recording detection time, and preset video file validity spot check frequency.

[0016] Further, the performing a bitstream frame rate calculation operation on the continuous video data to determine the corresponding actual bitstream frame rate includes:

[0017] Demultiplex the continuous video recording data according to a preset demultiplexing standard to determine corresponding continuous video recording video data, and frame the continuous video recording video data according to a preset framing standard to determine a corresponding continuous video recording video frame sequence;

[0018] Perform a traversal counting operation on the continuous video recording video frame sequence, and determine a corresponding actual bitstream frame rate according to the total number of video frames obtained after the traversal counting operation and the time of the continuous video recording video frame sequence.

[0019] Further, before performing the bitstream duration detection operation on the continuous video recording data according to the set bitstream duration detection model to determine a corresponding bitstream duration deviation, it includes:

[0020] Perform a feature extraction operation on a preset video frame data set according to a preset target detection model to determine corresponding target features, where the video frame data set is a data set containing video frame labels with timestamps, and the target features include bounding boxes, confidence levels, and category information;

[0021] Perform a contour extraction operation on the video frame data set according to a preset contour extraction algorithm to determine corresponding contour features, and the contour features include contour length, contour area, contour perimeter, and contour shape factor;

[0022] Perform a character recognition operation on the video frame data set according to a preset optical character recognition algorithm to determine corresponding character features, and the character features include character shape, character position, character confidence level, and character font;

[0023] Perform a feature fusion operation on the target features, the contour features, and the character features according to a preset progressive fusion algorithm, and input the comprehensive features obtained after the feature fusion operation into a preset neural network model for model training to determine a corresponding bitstream duration detection model.

[0024] Further, the performing a feature fusion operation on the target features, the contour features, and the character features according to a preset progressive fusion algorithm, and inputting the comprehensive features obtained after the feature fusion operation into a preset neural network model for model training to determine a corresponding bitstream duration detection model includes:

[0025] Perform an initial fusion operation on the target features and the contour features to determine corresponding first fusion features, perform an intermediate fusion operation on the first fusion features and the character features according to a dynamic bilateral cross-fusion algorithm to determine corresponding second fusion features, and perform a final fusion operation on the first fusion features and the second fusion features to determine corresponding comprehensive features;

[0026] Input the comprehensive features into a preset neural network model for model training to determine the corresponding bitstream duration detection model.

[0027] Further, the operation of detecting the bitstream duration of the continuous video data according to the preset bitstream duration detection model to determine the corresponding bitstream duration deviation includes:

[0028] Demultiplex the continuous video data according to the preset demultiplexing standard to determine the corresponding continuous video data of the video, and frame the continuous video data according to the preset framing standard to determine the corresponding continuous video frame sequence;

[0029] Perform character recognition on the continuous video frame sequence according to the preset bitstream duration detection model to determine the actual playing duration of the video frame;

[0030] Compare the actual playing duration of the video frame with the preset normal playing duration to determine the corresponding bitstream duration deviation.

[0031] Further, the operation of determining the corresponding abnormal video data and the monitoring abnormal video time corresponding to the abnormal video data according to the bitstream frame rate deviation and the bitstream duration deviation includes:

[0032] Judge whether the bitstream frame rate deviation and the bitstream duration deviation are within the normal range;

[0033] If not, perform an abnormal marking operation on the continuous video data within the abnormal range to determine the corresponding abnormal video data and the monitoring abnormal video time corresponding to the abnormal video data.

[0034] In a second aspect, the present application provides a supplementary storage device for abnormal storage of video in a monitoring platform, including:

[0035] A video recording schedule construction module, configured to read video recording information of a camera according to a preset video recording detection rule, compare the camera video recording time obtained after the video recording information reading operation with the preset video monitoring platform video recording time, and determine the corresponding monitoring video recording schedule, where the monitoring video recording schedule includes monitoring vulnerability video recording time and monitoring continuous video recording time;

[0036] A continuous monitoring anomaly judgment module is used to perform a video information reading operation on the video file corresponding to the continuous monitoring video recording time, obtain the corresponding continuous video recording data, perform a bitstream frame rate calculation operation on the continuous video recording data to determine the corresponding actual bitstream frame rate, compare the actual bitstream frame rate with the original bitstream frame rate of the camera to determine the corresponding bitstream frame rate deviation, and perform a bitstream duration detection operation on the continuous video recording data according to a set bitstream duration detection model to determine the corresponding bitstream duration deviation. Among them, the set bitstream duration detection model is obtained by calculating comprehensive features through a progressive fusion algorithm based on target features, contour features, and character features, and inputting the comprehensive features into a preset neural network model for model training;

[0037] An abnormal video supplementary storage module is used to determine the corresponding abnormal video data and the monitoring abnormal video time corresponding to the abnormal video data according to the bitstream frame rate deviation and the bitstream duration deviation, and send a video download request to the camera according to the monitoring vulnerability video time and the monitoring abnormal video time. After the camera receives the video download request, it sends the video file corresponding to the monitoring vulnerability video time and the monitoring abnormal video time to the video monitoring platform for abnormal video supplementary storage operation to determine the corresponding complete monitoring video file.

[0038] In a third aspect, the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the supplementary storage method for video anomaly storage of the monitoring platform are implemented.

[0039] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the supplementary storage method for video anomaly storage of the monitoring platform are implemented.

[0040] In a fifth aspect, the present application provides a computer program product, including a computer program / instructions. When the computer program / instructions are executed by a processor, the steps of the supplementary storage method for video anomaly storage of the monitoring platform are implemented.

[0041] As can be seen from the above technical solutions, the present application provides a supplementary storage method and device for video abnormal storage in a monitoring platform. By reading the video information of a specified camera, comparing the video time in this camera with the preset video monitoring platform video time, obtaining the vulnerable video time and continuous video time of the video in the monitoring platform, reading the video information of the video file corresponding to the continuous video time, calculating the bitstream frame rate deviation for this video information, detecting the bitstream duration of this video information through a set bitstream duration detection model to obtain the bitstream duration deviation, judging the abnormal video time within the continuous video time through the bitstream frame rate deviation and the bitstream duration deviation, and then sending a video download request to the camera for the vulnerable video time and the abnormal video time within the continuous video time for supplementary storage of abnormal videos, a complete monitoring video file can be obtained, thereby improving the detection efficiency and accuracy of abnormal monitoring video recordings. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0043] Figure 1 It is one of the flow diagrams of the supplementary storage method for video abnormal storage in the monitoring platform in the embodiments of the present application;

[0044] Figure 2 It is the second flow diagram of the supplementary storage method for video abnormal storage in the monitoring platform in the embodiments of the present application;

[0045] Figure 3 It is the third flow diagram of the supplementary storage method for video abnormal storage in the monitoring platform in the embodiments of the present application;

[0046] Figure 4 It is the fourth flow diagram of the supplementary storage method for video abnormal storage in the monitoring platform in the embodiments of the present application;

[0047] Figure 5 It is the fifth flow diagram of the supplementary storage method for video abnormal storage in the monitoring platform in the embodiments of the present application;

[0048] Figure 6 It is the sixth flow diagram of the supplementary storage method for video abnormal storage in the monitoring platform in the embodiments of the present application;

[0049] Figure 7 It is the seventh flow diagram of the supplementary storage method for video abnormal storage in the monitoring platform in the embodiments of the present application;

[0050] Figure 8 The structure diagram of the supplementary storage device for abnormal video storage in the monitoring platform in the embodiments of the present application;

[0051] Figure 9 The structure schematic diagram of the electronic device in the embodiments of the present application.

[0052] Reference numerals:

[0053] Electronic device 9600, central processing unit 9100, memory 9140, communication module 9110, input unit 9120, audio processor 9130, display 9160, power supply 9170, buffer memory 9141, application / function storage unit 9142, data storage unit 9143, driver program storage unit 9144, antenna 9111, speaker 9131, microphone 9132. Specific embodiments

[0054] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without making creative efforts shall fall within the protection scope of the present application.

[0055] In the technical solutions of the present application, the acquisition, storage, use, processing, etc. of data all comply with the relevant regulations of national laws and regulations.

[0056] Considering that the existing video recording storage methods have problems such as inability to guarantee the integrity of the recording, difficulty in supplementing the lost recording, and inconvenience in the manual query method. The present application provides a supplementary storage method and device for abnormal video storage in a monitoring platform. By reading the recording information of a specified camera, comparing the recording time in the camera with the recording time of the preset video monitoring platform, obtaining the vulnerable recording time and continuous recording time of the video in the monitoring platform, reading the recording information of the recording file corresponding to the continuous recording time, calculating the bitstream frame rate deviation for the recording information, detecting the bitstream duration of the recording information through a set bitstream duration detection model to obtain the bitstream duration deviation, judging the abnormal recording time within the continuous recording time through the bitstream frame rate deviation and the bitstream duration deviation, and then sending a recording download request to the camera for abnormal video supplementary storage for the vulnerable recording time and the abnormal recording time within the continuous recording time to obtain a complete monitoring recording file, thereby being able to improve the detection efficiency and accuracy of abnormal monitoring video recordings.

[0057] To improve the detection efficiency and accuracy of abnormal monitoring video recordings, this application provides an embodiment of a supplementary storage method for video abnormal storage in a monitoring platform. Refer to Figure 1 The supplementary storage method for video abnormal storage in the monitoring platform specifically includes the following content:

[0058] Step S101: Read the video recording information of the camera according to the preset video recording detection rules, compare the camera video recording time obtained after the video recording information reading operation with the preset video monitoring platform video recording time, and determine the corresponding monitoring video recording schedule. Among them, the monitoring video recording schedule includes the monitoring vulnerability video recording time and the monitoring continuous video recording time;

[0059] Optionally, the storage of monitoring videos in the video monitoring platform is based on the video recording time of the camera. However, due to reasons such as network fluctuations, the video data platform does not receive the video recording information of the camera at all, resulting in a vulnerability time, that is, the camera records video information, but the monitoring platform does not store it.

[0060] Optionally, in this embodiment, to initially detect the integrity of the video recording files stored in the video recording platform, a detection task is first formulated. According to the task regulations, the video recording time of the camera is read and compared with the video recording time of the video monitoring platform to find the video recording vulnerability time of the video monitoring platform.

[0061] Specifically, in terms of hardware deployment, deploy the video monitoring platform, then connect the camera to the video monitoring platform, and implement the video recording on-demand function. Finally, deploy the supplementary storage program so that it can read the video recording information of the video platform and the video recorder and the stored video files.

[0062] Specifically, in terms of the detection process, the detection task is completed by the supplementary storage program. First, the detection program in the supplementary storage program is used to formulate tasks, select the cameras to be detected, the video recording time (usually the day before detection), the detection start time (usually in the early morning), the time interval, and the sampling frequency of the validity of the video recording files. Among them, the camera can be a single camera or a group of cameras. Each camera in a group of cameras has a unique identification code for distinguishing each other;

[0063] Secondly, the detection program periodically reads the video recording information of the specified camera / camera group and obtains the return result, and merges the video recording times of each camera / camera group to obtain the complete video recording time of the camera;

[0064] Finally, compare the complete video recording time of the camera with the recording time stored in the video surveillance platform to find the vulnerable time for the storage of surveillance videos in the video surveillance platform, and generate a video recording schedule. The video recording schedule includes the vulnerable recording time for the storage of surveillance videos in the video surveillance platform and the continuous recording time for the storage of surveillance videos in the video surveillance platform. Record the paths of all existing video recording files and the normal duration (the recording duration set by the camera) during the continuous recording time, and store them in the database, laying a foundation for subsequent detection of the integrity of video recording files during the continuous recording time.

[0065] For example, the program reads the video recording information of the camera and compares it with the recording time of the video surveillance platform to generate a surveillance video recording schedule. It is found that the set recording duration of a certain camera is from 2 am to 5 am, but the recording time of this camera between 2 am and 3 am is missing on the platform. The missing time is recorded as the vulnerable recording time for surveillance, and the recording time when there are video recording files after 3 am for a certain camera is recorded as the continuous recording time for surveillance. The normal duration during the continuous recording time for surveillance is 2 hours from 3 am to 5 am.

[0066] Step S102: Perform an operation to read video recording information on the video recording file corresponding to the continuous recording time for surveillance to obtain the corresponding continuous recording data. Perform an operation to calculate the bitstream frame rate on the continuous recording data to determine the corresponding actual bitstream frame rate. Compare the actual bitstream frame rate with the original bitstream frame rate of the camera to determine the corresponding bitstream frame rate deviation. Perform an operation to detect the bitstream duration on the continuous recording data according to the set bitstream duration detection model to determine the corresponding bitstream duration deviation. The set bitstream duration detection model is obtained by calculating the comprehensive feature through the progressive fusion algorithm based on the target feature, contour feature, and character feature, and inputting the comprehensive feature into the preset neural network model for model training.

[0067] Optionally, in the centralized storage system, due to network fluctuations, server downtime, or other storage system problems, the video data may not be stored completely. In this case, although the normal duration extracted from the database is based on the recording time set by the camera, the actual video file may have video deviations due to missing some data. That is, the camera records video information, but the surveillance platform does not store it completely.

[0068] Optionally, in this embodiment, to further detect the integrity of video recording files during the continuous recording time in response to the above problems, bitstream frame rate detection and bitstream duration detection are applied to judge the integrity of video recording files.

[0069] Specifically, for bitstream frame rate detection, first, demultiplex the bitstream of the video file during continuous recording time according to the ISO / IEC 13818-1 standard, and split the composite bitstream into individual video streams and audio streams; then, parse the video data into several groups of frames according to the video coding specification (ITU-T H.264 or T-REC-H.265), and then compare the actual frame rate calculated by the ratio of the total number of video frames to the time with the frame rate of the original camera configuration to obtain the frame rate deviation.

[0070] It can be understood that the frame rate is an important factor affecting video smoothness. By detecting the deviation between the actual frame rate and the configured frame rate, it is possible to determine whether there are quality problems such as video stuttering and frame dropping, so as to evaluate the overall quality of the video. In a video surveillance system, a stable frame rate is crucial for both real-time monitoring and post-event playback. By detecting the frame rate deviation, the smoothness and integrity of the surveillance video can be ensured, and the reliability and effectiveness of the surveillance system can be improved.

[0071] Specifically, for bitstream duration detection, the duration detection is also built on the basis of demultiplexing and frame grouping. By setting a duration detection model, use the character overlay technology to parse the timestamp of each frame, and calculate the actual duration of the video through character overlay analysis. Compare the calculated duration with the expected recording duration (provided by the camera's recording settings and the database) to obtain the duration deviation.

[0072] More specifically, for the construction of the duration detection model, the duration detection model is divided into an image preprocessing module, a character detection module, a character recognition module, and a character overlay analysis module.

[0073] Image preprocessing module: Use the open-source frame extraction tool OpenCV to extract frames from the video to achieve frame-by-frame parsing. Then, use data augmentation techniques (denoising, contrast enhancement, brightness adjustment) to optimize the image quality of the video frames. By applying the median filter enhancement algorithm to each frame image, improve the visibility of characters and details in the video frames, so as to provide better input for the target detection algorithm. This process helps to analyze the video content in detail and provides a basis for subsequent character detection and duration deviation calculation.

[0074] Character Detection Module: This module includes a target detection algorithm and a contour extraction algorithm. First, the pre-trained target detection model YOLO is used to detect the character regions in video frames. The YOLO model can quickly and accurately locate the positions of characters in new images by learning a large amount of image data with labeled character regions during the training process. It outputs the bounding boxes of the character regions, including the center coordinates (x, y), width, and height of the bounding boxes. Then, an edge detection algorithm is used to determine the boundaries of the characters. The gradient magnitude and direction of the image are calculated by computing the differences in the horizontal and vertical directions of the image. The gradient magnitude represents the degree of brightness change at each pixel point of the image, while the gradient direction represents the direction of brightness change. Non-maximum suppression is performed on the gradient magnitude to retain only the gradient magnitudes of local maxima and remove other non-edge pixel points. A high threshold and a low threshold are set. Pixel points with gradient magnitudes higher than the high threshold are considered strong edges. Pixel points with gradient magnitudes between the high and low thresholds are considered edges if they are connected to strong edge pixel points, otherwise they are suppressed, thus obtaining clear character contours.

[0075] Character Recognition Module: Apply OCR technology to convert the detected character regions into text information. The extracted character region images are input into Tesseract, which can recognize the characters by analyzing features such as strokes and shapes in the images.

[0076] Character Overlay Analysis Module: Apply context technology and combine video content to analyze the intention and meaning of character overlays. The time characters in different frames change over time. By analyzing the changes of characters in the time series, the temporal relationship of the characters can be determined.

[0077] For example, the character overlay contains time information such as "2024 - 06 - 11 14:30:25". By analyzing these time characters, the actual start and end times of the video segment can be accurately determined. Comparing the analyzed time information with the duration recorded in the metadata of the video file can calibrate the actual duration of the video recording, thus discovering whether there is a situation where the video recording is truncated in advance or starts late.

[0078] For another example, studying the changes of time characters in different frames can detect the continuity of video time. In normal video recordings, the time characters should increase continuously. If there is a time jump, such as suddenly changing from "14:30:25" to "14:30:40", it indicates that 15 seconds of video recording may be missing in between. In scenarios where precise tracing of the chronological order of events is required, such as accident investigation and security monitoring, ensuring the integrity of the video is crucial.

[0079] More specifically, for the training of the duration detection model, prepare the data required for object detection, contour extraction, and optical character recognition and perform necessary preprocessing. Label the data to ensure that each sample used for training the model has corresponding object detection boxes, contour information, and character recognition results.

[0080] Input the dataset into the object detection model YOLOv5 to extract object features, which include bounding boxes, confidence levels, and class information. For each detected object, the model outputs a bounding box represented by four coordinate values, namely (x_min, y_min, x_max, y_max); the model also outputs a confidence level for each detected object, indicating the confidence of the model in detecting the object; the class information is used to identify whether the detected class is time-related data.

[0081] Input the dataset into the contour extraction algorithm to extract the contour features of the object, including contour length, contour area, contour perimeter, and contour shape factor. For each extracted contour, calculate the contour length, calculate the area of the region enclosed by each contour, calculate the contour perimeter, and calculate the contour shape factor to more comprehensively describe the content in the contour.

[0082] Input the dataset into the OCR model to extract character features, including character shape, character position, character confidence level, and character font. Extract the shape features and font features of each recognized character, extract the position of the character in the image, and output the confidence level for accurately locating and outputting the correct time text information for subsequent character overlay analysis.

[0083] Map the features of the above different modalities to the same dimension, and use the splicing method to preliminarily fuse the object features and contour features into a feature vector; then, perform dynamic bilateral cross-fusion on the preliminarily fused features and character features. For the two modalities, use the uncertainty estimator to evaluate the uncertainty of the model prediction. The modality with lower uncertainty is given a higher weight, and vice versa. Dynamically generate weights through uncertainty estimation, and use the generated dynamic weights to perform weighted fusion on the features of different modalities. By dynamically adjusting the fusion weights, it is possible to make full use of the information of high-quality modalities while reducing the interference of low-quality modalities, thus significantly improving the robustness and performance of the model; finally, gradually compress the feature map obtained by the encoder to the subsequent decoder through a progressive compression module PCM, make full use of the high-level features of the image, and perform final fusion on all features to form a comprehensive feature vector.

[0084] Input the fused comprehensive feature vector into a new neural network model, train the model using the labeled data, and use the optimal loss function to update the parameters of each module algorithm in reverse, so as to obtain a trained bitstream duration detection model that can predict the bitstream duration based on the fused features.

[0085] It can be understood that due to the certain correlation between the object detection and character recognition tasks, object detection can provide prior information about the spatial position for character recognition. During the training process, preferably, use a unified training set to optimize the entire model end-to-end, enabling the model to automatically learn the correlation between these two tasks. For example, the model can learn that when detecting a target region of a specific shape, it is more likely to recognize certain types of characters. This joint learning can enable the model to better utilize the complementary information between tasks and improve the overall performance. At the same time, the object detection model provides the position information of the character region, the contour extraction model provides the shape boundary information of the character, and the OCR model provides the text content information of the character. These information complement each other and can more comprehensively describe the character features. For example, in the case of partial occlusion or blur of the character, the position information and shape information can assist in the recognition of the text information, improving the accuracy of duration detection. Moreover, through multi-modal feature fusion, more discriminative feature representations can be learned. The fused features can better reflect the state of the character in the video bitstream, helping the duration detection model to more accurately judge duration anomalies. For example, the fused features can include information such as the integrity of the character region, the coherence of the character shape, and the accuracy of the character text, providing a richer basis for duration detection.

[0086] More specifically, for the application of the duration detection model, during the application process, input the processed real-time video frame data into the duration detection model. The model will identify the region containing the character, then further process the detected character region of the target to accurately determine the boundary of the character. Finally, send the character region after contour extraction into the OCR system to convert the characters in the image into text. Through the temporal analysis of the text and the character overlay analysis, detect the dynamic changes in the recording time, judge the increasing or decreasing trend of the video time, and thus further accurately infer the actual duration of the bitstream. By comparing the actual duration with the preset normal duration in the database, determine the bitstream duration deviation.

[0087] Step S103: Determine the corresponding abnormal video data and the monitoring abnormal video time corresponding to the abnormal video data according to the bitstream frame rate deviation and the bitstream duration deviation. Send a video download request to the camera according to the monitoring vulnerability video time and the monitoring abnormal video time. After the camera receives the video download request, send the video files corresponding to the monitoring vulnerability video time and the monitoring abnormal video time to the video monitoring platform for abnormal video supplementary storage operation, and determine the corresponding complete monitoring video file.

[0088] Optionally, in this embodiment, the video abnormality is jointly judged by the bitstream duration deviation and the bitstream frame rate deviation, and the time of the abnormal video file and the time of the vulnerability video file are used to send a video file download request to the camera, and the abnormal video and the vulnerability video are complemented into a complete video.

[0089] Specifically, if the floating range of the calculated average video frame rate is smaller than the frame rate configured by the camera, then it is judged that the actual frame rate is abnormal; if the calculated video recording duration is shorter than the database recording duration, then it is judged that the actual recording duration is abnormal; thus, the abnormal file is placed in the abnormal data recovery area.

[0090] Specifically, the above vulnerability detection and abnormality detection tasks are executed concurrently in a loop.

[0091] Specifically, for the abnormal data and the vulnerability data, respectively send a video data download request within this time to the corresponding camera according to their corresponding abnormal time and vulnerability time, and automatically use the data of the camera to supplement and store the abnormal data on the platform.

[0092] It can be understood that because the preset time of the loop detection task is preferably 1 day, therefore, from the discovery of abnormal video data to the supplementary storage of abnormal data, it will not exceed the 3-day storage time of the camera, and there will be no data coverage situation. At the same time, after the abnormal data is discovered, the video supplementary storage records are merged and generated into a video supplementary storage detection table in terms of the camera and the date, and the results are stored in the database. The results and logs are provided to the user interface for display through an interface or other means. In this way, it is convenient for users to query abnormal data in a timely manner, and provides a path and basis for the manual supplement of abnormal data.

[0093] This example shows how this embodiment detects vulnerabilities and abnormalities in the videos of the monitoring video platform through concurrent polling, and performs video supplementary storage based on the detection results to obtain a complete video file.

[0094] As can be seen from the above description, the supplementary storage method for video anomaly storage of the monitoring platform provided by the embodiments of the present application can compare the video recording time in a specified camera with the preset video monitoring platform recording time by reading the recording information of the specified camera, obtain the vulnerable video recording time and continuous recording time of the video in the monitoring platform, read the recording information of the recording file corresponding to the continuous recording time, calculate the bitstream frame rate deviation for the recording information, detect the bitstream duration of the recording information through a set bitstream duration detection model to obtain the bitstream duration deviation, determine the abnormal video recording time within the continuous recording time through the bitstream frame rate deviation and the bitstream duration deviation, and then send a recording download request to the camera for the vulnerable video recording time and the abnormal video recording time within the continuous recording time for supplementary storage of abnormal videos, obtaining a complete monitoring video file, thereby improving the detection efficiency and accuracy of abnormal monitoring video recordings.

[0095] In an embodiment of the supplementary storage method for video anomaly storage of the monitoring platform of the present application, referring to Figure 2 , it may further specifically include the following content:

[0096] Step S201: Perform a unique encoding operation on all cameras to determine the corresponding camera encoding;

[0097] Step S202: Determine the corresponding video recording detection rules according to the camera encoding, preset video recording detection interval, preset video recording detection time, and preset random inspection frequency of video recording file validity.

[0098] Optionally, in this embodiment, the camera is connected to the video monitoring platform and the video recording on-demand function is implemented.

[0099] Optionally, in this embodiment, the camera can be a single camera or a group of cameras, and each camera in a group of cameras has a unique identification encoding for distinguishing each other.

[0100] Optionally, in this embodiment, the detection program is used to formulate tasks, select the cameras to be detected, the recording time (usually the previous day), the detection start (usually in the early morning), the time interval, and the random inspection frequency of video recording file validity.

[0101] Through step S202, this embodiment constructs a polling detection task, realizes automated fixed-point detection, and lays a foundation for subsequent video recording anomaly recognition.

[0102] In an embodiment of the supplementary storage method for video anomaly storage of the monitoring platform of the present application, referring to Figure 3 , it may further specifically include the following content:

[0103] Step S301: Demultiplex the continuous video recording data according to a preset demultiplexing standard to determine the corresponding continuous video recording video data, and perform a framing operation on the continuous video recording video data according to a preset framing standard to determine the corresponding continuous video recording video frame sequence;

[0104] Step S302: Perform a traversal counting operation on the continuous video recording video frame sequence, and determine the corresponding actual bitstream frame rate according to the total number of video frames obtained after the traversal counting operation and the time of the continuous video recording video frame sequence.

[0105] Optionally, in this embodiment, first, demultiplex the bitstream of the video recording file in the continuous video recording time according to the ISO / IEC 13818-1 standard, and split the composite bitstream into individual video streams and audio streams; then, parse the video data into several frame group sequences according to the video coding specification (ITU-T H.264 or T-REC-H.265), then traverse the video frame sequence, and compare the actual frame rate calculated by the ratio of the total number of video frames obtained to the time with the frame rate of the original camera configuration to obtain the frame rate deviation.

[0106] It can be understood that the frame rate is an important factor affecting video smoothness. By detecting the deviation between the actual frame rate and the configured frame rate, it is possible to determine whether there are quality problems such as video stuttering and frame loss, thereby evaluating the overall quality of the video. In a video surveillance system, a stable frame rate is crucial for both real-time monitoring and post-event playback. By detecting the frame rate deviation, the smoothness and integrity of the surveillance video can be ensured, and the reliability and effectiveness of the surveillance system can be improved.

[0107] Through step S302, this embodiment realizes video frame rate calculation, laying a foundation for subsequent identification of abnormal video data.

[0108] In an embodiment of the supplementary storage method for video anomaly storage in the monitoring platform of the present application, refer to Figure 4 , and it may specifically include the following content:

[0109] Step S401: Perform a feature extraction operation on a preset video frame data set according to a preset target detection model to determine the corresponding target features, where the video frame data set is a data set containing video frame labels with timestamps, and the target features include bounding boxes, confidence levels, and class information;

[0110] Step S402: Perform a contour extraction operation on the video frame data set according to a preset contour extraction algorithm to determine the corresponding contour features, and the contour features include contour length, contour area, contour perimeter, and contour shape factor;

[0111] Step S403: Perform character recognition operations on the video frame dataset according to a preset optical character recognition algorithm to determine corresponding character features, where the character features include character shape, character position, character confidence, and character font;

[0112] Step S404: Perform feature fusion operations on the target feature, the contour feature, and the character feature according to a preset progressive fusion algorithm, and input the comprehensive feature obtained after the feature fusion operation into a preset neural network model for model training to determine a corresponding bitstream duration detection model.

[0113] Optionally, in this embodiment, this process is a model training process.

[0114] Specifically, prepare the data required for object detection, contour extraction, and optical character recognition and perform necessary preprocessing. Label the data to ensure that each sample used for training the model has corresponding object detection boxes, contour information, and character recognition results. It can be understood that using a unified training set for labeling can ensure the consistency of labeling, and sharing features can enable the model to optimize these features from multiple perspectives, which helps improve the generalization ability of the features and enables it to better adapt to different characters and scenarios.

[0115] Input the dataset into the object detection model YOLOv5 to extract object features, where the object features include bounding boxes, confidence, and class information. For each detected object, the model outputs a bounding box represented by four coordinate values, namely (x_min, y_min, x_max, y_max); the model also outputs a confidence for each detected object, indicating the confidence level of the model in detecting the object; the class information is used to identify whether the detected class is data with time.

[0116] Input the dataset into the contour extraction algorithm to extract the contour features of the object, including contour length, contour area, contour perimeter, and contour shape factor. For each extracted contour, calculate the contour length, calculate the area of the region enclosed by each contour, calculate the contour perimeter, and calculate the contour shape factor to more comprehensively describe the content in the contour.

[0117] Input the dataset into the OCR model to extract character features, including character shape, character position, character confidence, and character font. Extract the shape features and font features of each recognized character, extract the position of the character in the picture, and output the confidence, which is used to accurately locate and output the correct time text information for subsequent character overlay analysis.

[0118] Map the features of the above different modalities to the same dimension, use the progressive fusion algorithm to obtain the fused comprehensive features, and then input the comprehensive features into a new neural network model for model training to obtain the final bitstream duration detection model.

[0119] Through step S404, this embodiment successfully obtains the bitstream duration detection model, laying a foundation for subsequent detection of the true duration of the bitstream.

[0120] In an embodiment of the supplementary storage method for video anomaly storage in the monitoring platform of this application, refer to Figure 5 , it may also specifically include the following content:

[0121] Step S501: Perform an initial fusion operation on the target feature and the contour feature to determine the corresponding first fusion feature, perform an intermediate fusion operation on the first fusion feature and the character feature according to the dynamic bilateral cross-fusion algorithm to determine the corresponding second fusion feature, and perform a final fusion operation on the first fusion feature and the second fusion feature to determine the corresponding comprehensive feature;

[0122] Step S502: Input the comprehensive feature into a preset neural network model for model training to determine the corresponding bitstream duration detection model.

[0123] Optionally, in this embodiment, the target feature and the contour feature are initially fused in a splicing manner to form a feature vector; then, the initially fused feature and the character feature are dynamically bilaterally cross-fused. For the two modalities, an uncertainty estimator is used to evaluate the uncertainty of the model prediction. The modality with lower uncertainty is given a higher weight, and vice versa. The weights are dynamically generated through uncertainty estimation, and the generated dynamic weights are used to perform weighted fusion on the features of different modalities. By dynamically adjusting the fusion weights, the information of high-quality modalities can be fully utilized while reducing the interference of low-quality modalities, thus significantly improving the robustness and performance of the model; finally, through a progressive compression module PCM, the feature map obtained by the encoder is gradually compressed and transmitted to the subsequent decoder, making full use of the high-level features of the image, and finally fusing all the features to form a comprehensive feature vector.

[0124] Input the fused comprehensive feature vector into a new neural network model, train the model with the labeled data, and use the optimal loss function to update the parameters of each module algorithm in reverse, so as to obtain the trained bitstream duration detection model, enabling it to predict the bitstream duration according to the fused features.

[0125] Through step S502, this embodiment successfully obtains the bitstream duration detection model through multi-modal fusion, laying a foundation for subsequent detection of the true duration of the bitstream.

[0126] In an embodiment of the supplementary storage method for video abnormal storage in the monitoring platform of the present application, refer to Figure 6 , the following specific content may also be included:

[0127] Step S601: Demultiplex the continuous video recording data according to a preset demultiplexing standard to determine the corresponding continuous video recording video data, and perform a framing operation on the continuous video recording video data according to a preset framing standard to determine the corresponding continuous video recording video frame sequence;

[0128] Step S602: Perform a character recognition operation on the continuous video recording video frame sequence according to a preset bitstream duration detection model to determine the actual playing duration of the corresponding video frame;

[0129] Step S603: Compare the actual playing duration of the video frame with a preset normal playing duration to determine the corresponding bitstream duration deviation.

[0130] Optionally, in this embodiment, the same operations of demultiplexing and framing the video recording are performed as in step S301.

[0131] Optionally, in this embodiment, the subsequent application of the bitstream duration detection model is to input the processed real-time video frame data into the duration detection model. The model will identify the area containing characters, then further process the detected target character area to accurately determine the boundaries of the characters, and finally send the character area after contour extraction into the OCR system to convert the characters in the image into text. Through the timing analysis and character overlay analysis of the text, the dynamic change of the recording time is detected, the increasing or decreasing trend of the video time is judged, so as to further accurately infer the actual duration of the bitstream. By comparing the actual duration with the preset normal duration in the database, the bitstream duration deviation is determined.

[0132] Through step S602, in this embodiment, the actual duration of the bitstream is successfully obtained through the bitstream duration detection model, and the duration deviation is obtained through comparison, laying a foundation for subsequent judgment of video file abnormality.

[0133] In an embodiment of the supplementary storage method for video abnormal storage in the monitoring platform of the present application, refer to Figure 7 , the following specific content may also be included:

[0134] Step S701: Judge whether the bitstream frame rate deviation and the bitstream duration deviation are within the normal range;

[0135] Step S702: If not, perform an abnormality marking operation on the continuous video recording data within the abnormal range to determine the corresponding abnormal video recording data and the monitoring abnormal video recording time corresponding to the abnormal video recording data.

[0136] Optionally, in this embodiment, video anomalies are judged jointly by the deviation of the bitstream duration and the deviation of the bitstream frame rate, and the time of the abnormal video file and the time of the vulnerability video file are used to send a video file download request to the camera to complete the abnormal video and the vulnerability video into a complete video.

[0137] Specifically, if the floating range of the calculated average video frame rate is smaller than the frame rate configured by the camera, then it is judged that the actual frame rate is abnormal; if the calculated video recording duration is less than the database recording duration, then it is judged that the actual recording duration is abnormal; thus, the abnormal files are placed in the abnormal data recovery area.

[0138] Through step S702, this embodiment successfully obtains the abnormal video file, and obtains the abnormal time corresponding to the road direction file through the information of the abnormal video file.

[0139] To improve the detection efficiency and accuracy of abnormal monitoring video recordings, this application provides an embodiment of a supplementary storage device for monitoring platform video abnormal storage, which implements all or part of the content of the supplementary storage method for monitoring platform video abnormal storage. Refer to Figure 8 The supplementary storage device for monitoring platform video abnormal storage specifically includes the following contents:

[0140] The video recording schedule construction module 10 is used to read the video recording information of the camera according to the preset video recording detection rules, compare the camera video recording time obtained after the video recording information reading operation with the preset video monitoring platform video recording time, and determine the corresponding monitoring video recording schedule, where the monitoring video recording schedule includes monitoring vulnerability video recording time and monitoring continuous video recording time;

[0141] The continuous monitoring anomaly judgment module 20 is used to read the video recording information of the video recording file corresponding to the monitoring continuous video recording time to obtain the corresponding continuous video recording data, perform a bitstream frame rate calculation operation on the continuous video recording data to determine the corresponding actual bitstream frame rate, compare the actual bitstream frame rate with the original bitstream frame rate of the camera to determine the corresponding bitstream frame rate deviation, and perform a bitstream duration detection operation on the continuous video recording data according to the set bitstream duration detection model to determine the corresponding bitstream duration deviation, where the set bitstream duration detection model is obtained by calculating the comprehensive feature through the progressive fusion algorithm based on the target feature, contour feature, and character feature, and inputting the comprehensive feature into the preset neural network model for model training.

[0142] The abnormal video supplementary storage module 30 is used to determine corresponding abnormal video data and the monitoring abnormal video time corresponding to the abnormal video data according to the bitstream frame rate deviation and the bitstream duration deviation, and send a video download request to the camera according to the monitoring vulnerability video time and the monitoring abnormal video time, so that after the camera receives the video download request, it sends the video files corresponding to the monitoring vulnerability video time and the monitoring abnormal video time to the video monitoring platform for abnormal video supplementary storage operation, and determines the corresponding complete monitoring video file.

[0143] As can be seen from the above description, the supplementary storage device for video anomaly storage of the monitoring platform provided by the embodiment of the present application can compare the video time in the camera with the preset video monitoring platform video time by reading the video information of the specified camera, obtain the vulnerability video time and continuous video time of the video in the monitoring platform, read the video information of the video file corresponding to the continuous video time, calculate the frame rate of the video information to obtain the bitstream frame rate deviation, detect the bitstream duration of the video information through the set bitstream duration detection model to obtain the bitstream duration deviation, judge the abnormal video time within the continuous video time through the bitstream frame rate deviation and the bitstream duration deviation, and then send a video download request to the camera for abnormal video supplementary storage for the vulnerability video time and the abnormal video time within the continuous video time, so as to obtain a complete monitoring video file, thereby improving the detection efficiency and accuracy of abnormal monitoring video recordings.

[0144] From a hardware perspective, in order to improve the detection efficiency and accuracy of abnormal monitoring video recordings, the present application provides an embodiment of an electronic device for implementing all or part of the content in the supplementary storage method for video anomaly storage of the monitoring platform. The electronic device specifically includes the following:

[0145] A processor, a memory, a communication interface, and a bus; wherein, the processor, the memory, and the communication interface complete communication with each other through the bus; the communication interface is used to realize information transmission between the supplementary storage method for video anomaly storage of the monitoring platform and related devices such as the core business system, the user terminal, and the relevant database, etc. This logic controller can be a desktop computer, a tablet computer, a mobile terminal, etc., and this embodiment is not limited thereto. In this embodiment, this logic controller can be implemented with reference to the embodiments of the supplementary storage method for video anomaly storage of the monitoring platform and the embodiments of the supplementary storage method for video anomaly storage of the monitoring platform, and its content is incorporated herein, and the repeated parts will not be described again.

[0146] It can be understood that the user terminal may include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a personal digital assistant (PDA), a vehicle-mounted device, a smart wearable device, etc. Among them, the smart wearable device may include smart glasses, a smart watch, a smart bracelet, etc.

[0147] In practical applications, part of the supplementary storage method for abnormal video storage on the monitoring platform may be executed on the electronic device side as described above, or all operations may be completed in the client device. Specifically, it can be selected according to the processing capacity of the client device and the limitations of the user usage scenario, etc. This application does not make any limitations in this regard. If all operations are completed in the client device, the client device may further include a processor.

[0148] The above-mentioned client device may have a communication module (i.e., a communication unit), and can communicate with a remote server to realize data transmission with the server. The server may include a server on the task scheduling center side, and in other implementation scenarios, it may also include a server of an intermediate platform, such as a server of a third-party server platform with a communication link to the task scheduling center server. The server may include a single computer device, or may include a server cluster composed of multiple servers, or a server structure of a distributed device.

[0149] Figure 9 It is a schematic block diagram of the system composition of the electronic device 9600 according to an embodiment of the present application. As Figure 9 shown, the electronic device 9600 may include a central processing unit 9100 and a memory 9140; the memory 9140 is coupled to the central processing unit 9100. It should be noted that this Figure 9 is exemplary; other types of structures can also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0150] In one embodiment, the function of the supplementary storage method for abnormal video storage on the monitoring platform may be integrated into the central processing unit 9100. Among them, the central processing unit 9100 may be configured to perform the following controls:

[0151] Step S101: Perform a video information reading operation on the camera according to a preset video recording detection rule, compare the camera video recording time obtained after the video information reading operation with the preset video monitoring platform recording time, and determine the corresponding monitoring video recording schedule, where the monitoring video recording schedule includes monitoring vulnerability recording time and monitoring continuous recording time;

[0152] Step S102: Perform a video information reading operation on the video file corresponding to the continuous video recording time of the monitoring, obtain the corresponding continuous video recording data, perform a bitstream frame rate calculation operation on the continuous video recording data, determine the corresponding actual bitstream frame rate, compare the actual bitstream frame rate with the original bitstream frame rate of the camera, determine the corresponding bitstream frame rate deviation, and perform a bitstream duration detection operation on the continuous video recording data according to the set bitstream duration detection model to determine the corresponding bitstream duration deviation. Among them, the set bitstream duration detection model is obtained by calculating the comprehensive features through the progressive fusion algorithm based on the target features, contour features, and character features, and inputting the comprehensive features into the preset neural network model for model training;

[0153] Step S103: Determine the corresponding abnormal video recording data and the monitoring abnormal video recording time corresponding to the abnormal video recording data according to the bitstream frame rate deviation and the bitstream duration deviation, and send a video recording download request to the camera according to the monitoring vulnerability video recording time and the monitoring abnormal video recording time. After the camera receives the video recording download request, send the video file corresponding to the monitoring vulnerability video recording time and the monitoring abnormal video recording time to the video monitoring platform for abnormal video supplementary storage operation to determine the corresponding complete monitoring video file.

[0154] As can be seen from the above description, the electronic device provided in the embodiment of the present application reads the video information of the specified camera, compares the video recording time in the camera with the video recording time of the preset video monitoring platform, obtains the vulnerability video recording time and the continuous video recording time of the video in the monitoring platform, reads the video information of the video file corresponding to the continuous video recording time, calculates the bitstream frame rate of the video information to obtain the bitstream frame rate deviation, performs a bitstream duration detection on the video information through the set bitstream duration detection model to obtain the bitstream duration deviation, determines the abnormal video recording time within the continuous video recording time through the bitstream frame rate deviation and the bitstream duration deviation, and then sends a video recording download request to the camera for abnormal video supplementary storage for the vulnerability video recording time and the abnormal video recording time within the continuous video recording time to obtain a complete monitoring video file, thereby improving the detection efficiency and accuracy of abnormal monitoring video recordings.

[0155] In another implementation manner, the supplementary storage method for video anomaly storage on the monitoring platform can be separately configured from the central processing unit 9100. For example, the supplementary storage method for video anomaly storage on the monitoring platform can be configured as a chip connected to the central processing unit 9100, and the function of the supplementary storage method for video anomaly storage on the monitoring platform is realized through the control of the central processing unit.

[0156] Such as Figure 9As shown, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It should be noted that the electronic device 9600 does not necessarily have to include Figure 9 all the components shown in Figure 9 ; in addition, the electronic device 9600 may further include

[0157] As Figure 9 shown, the central processing unit 9100, sometimes also referred to as a controller or operation control, may include a microprocessor or other processor devices and / or logic devices. The central processing unit 9100 receives inputs and controls the operations of the various components of the electronic device 9600.

[0158] Among them, the memory 9140, for example, may be one or more of a buffer, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory, or other suitable devices. It can store the above information related to failures, and can also store programs for executing relevant information. And the central processing unit 9100 can execute the programs stored in the memory 9140 to implement information storage or processing, etc.

[0159] The input unit 9120 provides inputs to the central processing unit 9100. The input unit 9120 is, for example, a key or a touch input device. The power supply 9170 is used to supply power to the electronic device 9600. The display 9160 is used to display display objects such as images and texts. The display may be, for example, an LCD display, but is not limited thereto.

[0160] The memory 9140 may be a solid-state memory. For example, it may be a read-only memory (ROM), a random access memory (RAM), a SIM card, etc. It may also be a memory that stores information even when powered off, can be selectively erased and has more data. Examples of such a memory are sometimes referred to as EPROMs, etc. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage section 9142, which is used to store application programs and function programs or the processes for operating the electronic device 9600 through the central processing unit 9100.

[0161] The memory 9140 may further include a data storage unit 9143 for storing data such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers of the electronic device for communication functions and / or for performing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0162] The communication module 9110 is a transmitter / receiver that transmits and receives signals via the antenna 9111. The communication module 9110 is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as in the case of a conventional mobile communication terminal.

[0163] Based on different communication technologies, multiple communication modules 9110 may be provided in the same electronic device, such as a cellular network module, a Bluetooth module, and / or a wireless local area network module, etc. The communication module 9110 is also coupled to the speaker 9131 and the microphone 9132 via the audio processor 9130 to provide an audio output via the speaker 9131 and receive an audio input from the microphone 9132, thereby implementing normal telecommunication functions. The audio processor 9130 may include any suitable buffers, decoders, amplifiers, etc. Additionally, the audio processor 9130 is also coupled to the central processor 9100, so that recording can be performed on the local machine through the microphone 9132, and the sound stored on the local machine can be played through the speaker 9131.

[0164] Embodiments of the present application also provide a computer-readable storage medium capable of implementing all steps of the supplementary storage method for video anomaly storage of a monitoring platform where the execution subject in the above embodiments is a server or a client. A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, all steps of the supplementary storage method for video anomaly storage of a monitoring platform where the execution subject in the above embodiments is a server or a client are implemented. For example, when the processor executes the computer program, the following steps are implemented:

[0165] Step S101: Perform a video information reading operation on the camera according to a preset video recording detection rule, compare the camera video recording time obtained after the video information reading operation with the preset video monitoring platform video recording time, and determine the corresponding monitoring video recording schedule, where the monitoring video recording schedule includes monitoring vulnerability video recording time and monitoring continuous video recording time;

[0166] Step S102: Perform an operation of reading video information on the video file corresponding to the continuous video recording time of the monitoring, obtain the corresponding continuous video recording data, perform an operation of calculating the bitstream frame rate on the continuous video recording data, determine the corresponding actual bitstream frame rate, compare the actual bitstream frame rate with the original bitstream frame rate of the camera, determine the corresponding bitstream frame rate deviation, and perform an operation of detecting the bitstream duration on the continuous video recording data according to the set bitstream duration detection model to determine the corresponding bitstream duration deviation, where the set bitstream duration detection model is obtained by calculating the comprehensive feature through the progressive fusion algorithm based on the target feature, contour feature, and character feature, and inputting the comprehensive feature into the preset neural network model for model training;

[0167] Step S103: Determine the corresponding abnormal video recording data and the monitoring abnormal video recording time corresponding to the abnormal video recording data according to the bitstream frame rate deviation and the bitstream duration deviation, and send a video recording download request to the camera according to the monitoring vulnerability video recording time and the monitoring abnormal video recording time, so that after the camera receives the video recording download request, it sends the video file corresponding to the monitoring vulnerability video recording time and the monitoring abnormal video recording time to the video monitoring platform for abnormal video supplementary storage operation to determine the corresponding complete monitoring video file.

[0168] As can be seen from the above description, the computer-readable storage medium provided by the embodiment of the present application reads the video information of the specified camera, compares the video recording time in the camera with the video recording time of the preset video monitoring platform, obtains the vulnerability video recording time and continuous video recording time of the video in the monitoring platform, reads the video information of the video file corresponding to the continuous video recording time, calculates the bitstream frame rate of the video information to obtain the bitstream frame rate deviation, performs a bitstream duration detection on the video information through the set bitstream duration detection model to obtain the bitstream duration deviation, determines the abnormal video recording time within the continuous video recording time through the bitstream frame rate deviation and the bitstream duration deviation, and then sends a video recording download request to the camera for abnormal video supplementary storage for the vulnerability video recording time and the abnormal video recording time within the continuous video recording time to obtain a complete monitoring video file, thereby improving the detection efficiency and accuracy of abnormal monitoring video recording.

[0169] The embodiment of the present application also provides a computer program product capable of implementing all steps of the supplementary storage method for video anomaly storage of the monitoring platform whose execution subject is a server or a client in the above embodiment. When the computer program / instructions are executed by a processor, the steps of the supplementary storage method for video anomaly storage of the monitoring platform are implemented. For example, the computer program / instructions implement the following steps:

[0170] Step S101: Perform a video information reading operation on the camera according to a preset video recording detection rule, compare the camera video recording time obtained after the video information reading operation with the preset video monitoring platform video recording time, and determine the corresponding monitoring video recording schedule. Among them, the monitoring video recording schedule includes monitoring vulnerability video recording time and monitoring continuous video recording time;

[0171] Step S102: Perform a video information reading operation on the video file corresponding to the monitoring continuous video recording time to obtain the corresponding continuous video data, perform a bitstream frame rate calculation operation on the continuous video data to determine the corresponding actual bitstream frame rate, compare the actual bitstream frame rate with the original bitstream frame rate of the camera to determine the corresponding bitstream frame rate deviation, and perform a bitstream duration detection operation on the continuous video data according to a set bitstream duration detection model to determine the corresponding bitstream duration deviation. Among them, the set bitstream duration detection model is obtained by calculating comprehensive features through a progressive fusion algorithm based on target features, contour features, and character features, and inputting the comprehensive features into a preset neural network model for model training;

[0172] Step S103: Determine the corresponding abnormal video data and the monitoring abnormal video recording time corresponding to the abnormal video data according to the bitstream frame rate deviation and the bitstream duration deviation, and send a video recording download request to the camera according to the monitoring vulnerability video recording time and the monitoring abnormal video recording time. After the camera receives the video recording download request, send the video file corresponding to the monitoring vulnerability video recording time and the monitoring abnormal video recording time to the video monitoring platform for abnormal video supplementary storage operation to determine the corresponding complete monitoring video file.

[0173] As can be seen from the above description, the computer program product provided by the embodiments of the present application reads the video information of a specified camera, compares the video recording time in the camera with the video recording time of the preset video monitoring platform, obtains the vulnerability video recording time and continuous video recording time of the video in the monitoring platform, reads the video information of the video file corresponding to the continuous video recording time, calculates the bitstream frame rate of the video information to obtain the bitstream frame rate deviation, performs a bitstream duration detection on the video information through a set bitstream duration detection model to obtain the bitstream duration deviation, determines the abnormal video recording time within the continuous video recording time through the bitstream frame rate deviation and the bitstream duration deviation, and then sends a video recording download request for abnormal video supplementary storage to the camera for the vulnerability video recording time and the abnormal video recording time within the continuous video recording time, so as to obtain a complete monitoring video file, thereby improving the detection efficiency and accuracy of abnormal monitoring video recording.

[0174] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, an apparatus, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) that contain computer-usable program code.

[0175] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (apparatuses), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0176] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that implements the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0177] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0178] Specific embodiments are applied in the present invention to elaborate on the principles and implementation manners of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A method for supplementing storage of abnormal video storage on a monitoring platform, characterized in that: The method comprises: Performing a video information reading operation on the camera according to a preset video detection rule, comparing the camera recording time obtained after the video information reading operation with the preset video monitoring platform recording time, and determining a corresponding monitoring video schedule, wherein the monitoring video schedule includes monitoring vulnerability recording time and monitoring continuous recording time; Performing a video information reading operation on the video file corresponding to the monitoring continuous video recording time to obtain corresponding continuous video recording data, performing a code stream frame rate calculation operation on the continuous video recording data to determine the corresponding actual code stream frame rate, performing a comparison operation on the actual code stream frame rate with the original code stream frame rate of the camera to determine the corresponding code stream frame rate deviation, performing a code stream duration detection operation on the continuous video recording data according to a set code stream duration detection model to determine the corresponding code stream duration deviation, wherein the set code stream duration detection model is a comprehensive feature calculated by a progressive fusion algorithm based on target features, contour features and character features, and the comprehensive feature is input into a preset God General network model for model training to obtain the result; Determine the corresponding abnormal video data and the abnormal monitoring video time corresponding to the abnormal video data according to the bitstream frame rate deviation and the bitstream duration deviation, and send a video download request to the camera according to the monitoring loophole video time and the abnormal monitoring video time, so that after the camera receives the video download request, it sends the video file corresponding to the monitoring loophole video time and the abnormal monitoring video time to the video monitoring platform for abnormal video supplementary storage operation, and determines the corresponding complete monitoring video file.

2. The method for supplementing storage of abnormal video storage of a monitoring platform according to claim 1, characterized in that: Before the video information reading operation is performed on the camera according to the preset video detection rule, the method includes: Perform unique coding operations on all cameras to determine the corresponding camera codes; The corresponding video detection rule is determined according to the camera code, the preset video detection interval, the preset video detection time and the preset video file validity spot check frequency.

3. The method for supplementing storage of abnormal video storage of a monitoring platform according to claim 1, characterized in that: The performing a code stream frame rate calculation operation on the continuous video data to determine a corresponding actual code stream frame rate includes: Demultiplexing the continuous video data according to a preset demultiplexing standard to determine the corresponding continuous video data, and framing the continuous video data according to a preset framing standard to determine the corresponding continuous video frame sequence; A traversal counting operation is performed on the continuous recording video frame sequence, and a corresponding actual code stream frame rate is determined according to the total number of video frames obtained after the traversal counting operation and the time of the continuous recording video frame sequence.

4. The method for supplementing storage of abnormal video storage of a monitoring platform according to claim 1, characterized in that: Before performing a code stream duration detection operation on the continuous video data according to the set code stream duration detection model to determine the corresponding code stream duration deviation, the method includes: Performing a feature extraction operation on a preset video frame data set according to a preset target detection model to determine corresponding target features, wherein the video frame data set is a data set including a timestamp video frame label, and the target features include a bounding box, a confidence level, and a category information; Performing a contour extraction operation on the video frame data set according to a preset contour extraction algorithm to determine corresponding contour features, wherein the contour features include contour length, contour area, contour perimeter, and contour shape factor; Performing a character recognition operation on the video frame data set according to a preset optical character recognition algorithm to determine corresponding character features, wherein the character features include character shape, character position, character confidence, and character font; A feature fusion operation is performed on the target feature, the contour feature and the character feature according to a preset progressive fusion algorithm, and the comprehensive feature obtained after the feature fusion operation is input into a preset neural network model for model training to determine the corresponding code stream duration detection model.

5. The method for supplementing storage of abnormal video storage of a monitoring platform according to claim 4 is characterized in that: The step of performing a feature fusion operation on the target feature, the contour feature, and the character feature according to a preset progressive fusion algorithm, inputting the comprehensive feature obtained after the feature fusion operation into a preset neural network model for model training, and determining a corresponding bitstream duration detection model includes: Performing an initial fusion operation on the target feature and the contour feature to determine a corresponding first fusion feature, performing an intermediate fusion operation on the first fusion feature and the character feature according to a dynamic bilateral cross fusion algorithm to determine a corresponding second fusion feature, and performing a final fusion operation on the first fusion feature and the second fusion feature to determine a corresponding comprehensive feature; The comprehensive features are input into a preset neural network model for model training to determine a corresponding bitstream duration detection model.

6. The method for supplementing storage of abnormal video storage of a monitoring platform according to claim 1, characterized in that: The performing a code stream duration detection operation on the continuous video data according to the set code stream duration detection model to determine the corresponding code stream duration deviation includes: Demultiplexing the continuous video data according to a preset demultiplexing standard to determine the corresponding continuous video data, and framing the continuous video data according to a preset framing standard to determine the corresponding continuous video frame sequence; Performing character recognition operation on the continuous recorded video frame sequence according to the set bit stream duration detection model to determine the actual playback duration of the corresponding video frame; The actual playback duration of the video frame is compared with the preset normal playback duration to determine the corresponding bitstream duration deviation.

7. The method for supplementing storage of abnormal video storage of a monitoring platform according to claim 1, characterized in that: The determining, according to the bitstream frame rate deviation and the bitstream duration deviation, corresponding abnormal video data and the monitoring abnormal video recording time corresponding to the abnormal video data comprises: Determine whether the bitstream frame rate deviation and the bitstream duration deviation are within a normal range; If not, the continuous video data within the abnormal range is subjected to an abnormal marking operation to determine the corresponding abnormal video data and the monitoring abnormal video time corresponding to the abnormal video data.

8. A supplementary storage device for abnormal video storage of a monitoring platform, characterized in that: The device comprises: A recording schedule construction module is used to read the recording information of the camera according to the preset recording detection rules, compare the camera recording time obtained after the recording information reading operation with the preset video monitoring platform recording time, and determine the corresponding monitoring recording schedule, wherein the monitoring recording schedule includes the monitoring vulnerability recording time and the monitoring continuous recording time; The continuous monitoring abnormality judgment module is used to read the video information of the video file corresponding to the continuous video recording time of the monitoring, obtain the corresponding continuous video data, perform a code stream frame rate calculation operation on the continuous video data, determine the corresponding actual code stream frame rate, compare the actual code stream frame rate with the original code stream frame rate of the camera, determine the corresponding code stream frame rate deviation, perform a code stream duration detection operation on the continuous video data according to a set code stream duration detection model, and determine the corresponding code stream duration deviation, wherein the set code stream duration detection model is obtained by calculating the comprehensive features through a progressive fusion algorithm based on target features, contour features and character features, and the comprehensive features are input into a preset God General network model for model training; The abnormal video supplement storage module is used to determine the corresponding abnormal video data and the monitoring abnormal video time corresponding to the abnormal video data according to the bit stream frame rate deviation and the bit stream duration deviation, and send a video download request to the camera according to the monitoring loophole video time and the monitoring abnormal video time, so that after the camera receives the video download request, it sends the video file corresponding to the monitoring loophole video time and the monitoring abnormal video time to the video monitoring platform for abnormal video supplement storage operation, and determines the corresponding complete monitoring video file.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method for supplementing storage of abnormal video storage of the monitoring platform described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for supplementing storage of abnormal video storage of a monitoring platform as described in any one of claims 1 to 7 are implemented.

Images