Supplementary storage method and device for abnormal video storage of monitoring platform

Abstract

The embodiment of the invention provides a supplementary storage method and device for monitoring platform video abnormal storage, and the method comprises the steps: reading the video recording information of a specified camera, comparing the video recording time of the camera with the video recording time of a preset video monitoring platform, obtaining the vulnerability video recording time and the continuous video recording time of a video in the monitoring platform, and storing the vulnerability video recording time and the continuous video recording time of the video in the monitoring platform; reading video recording information of a video recording file corresponding to the continuous video recording time, performing frame rate calculation on the video recording information to obtain a code stream frame rate deviation, performing code stream time length detection on the video recording information by setting a code stream time length detection model to obtain a code stream time length deviation, and outputting the code stream time length deviation. The abnormal video recording time in the continuous video recording time is judged according to the code stream frame rate deviation and the code stream duration deviation, and then a video downloading request is sent to the camera for the vulnerability video recording time and the abnormal video recording time in the continuous video recording time to carry out abnormal video supplementary storage, so that a complete monitoring video file is obtained. According to the invention, the detection efficiency and accuracy of the abnormal monitoring video recording can be improved.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a method and device for supplementing storage of abnormal video storage on a monitoring platform. Background Art

[0002] In existing video surveillance systems, video recordings are usually stored in a combination of SD cards (memory cards) built into front-end cameras and centralized storage at the back end. Front-end cameras use SD cards for short-term storage, with an estimated storage time of about three days, and use a circular overwrite method to manage storage space. The back end uses centralized storage, which has a relatively long storage time and can usually store about three months of recordings. However, this storage method faces some challenges and problems in actual applications.

[0003] First, due to network problems, server failures and other reasons, the videos stored centrally at the back end may not be fully saved. When an abnormal situation occurs and the videos need to be reviewed, if the centrally stored videos are incomplete, it will seriously affect the investigation and handling of the incident.

[0004] Secondly, even if the back-end centralized storage is normal, the storage time of the SD card of the front-end camera is short and it uses a circular overwrite method. Once the abnormality is discovered and measures are taken, the video on the SD card has been overwritten. In this case, the centralized storage cannot effectively supplement the lost video.

[0005] In addition, the existing technology has obvious defects in dealing with lost videos. On the one hand, supplementing the video by recording the time may lead to inaccurate video, because there may be a situation where the intermediate video is missing, but it cannot be queried through the record. On the other hand, for the lost video, the current main method is to manually query the camera web page, which is not only inefficient, but also very inconvenient for users in actual operation because the camera IP, account, and password information may be unknown, or the camera network is not in the office network.

[0006] In summary, the existing video recording storage method has problems such as the inability to guarantee the integrity of the video recording, the difficulty in supplementing the lost video recording, and the inconvenience of manual query. A more efficient and accurate video recording supplementary storage method is urgently needed to solve these problems. Summary of the invention

[0007] In view of the problems in the prior art, the present application provides a method and device for supplementing storage of abnormal video storage on a monitoring platform, which can improve the detection efficiency and accuracy of abnormal monitoring video recordings.

[0008] In order to solve at least one of the above problems, the present application provides the following technical solutions: In a first aspect, the present application provides a method for supplementing storage of abnormal video storage on a monitoring platform, comprising: Performing a video information reading operation on the camera according to a preset video detection rule, comparing the camera recording time obtained after the video information reading operation with the preset video monitoring platform recording time, and determining a corresponding monitoring video schedule, wherein the monitoring video schedule includes monitoring vulnerability recording time and monitoring continuous recording time; Performing a video information reading operation on the video file corresponding to the monitoring continuous video recording time to obtain corresponding continuous video recording data, performing a code stream frame rate calculation operation on the continuous video recording data to determine the corresponding actual code stream frame rate, performing a comparison operation on the actual code stream frame rate with the original code stream frame rate of the camera to determine the corresponding code stream frame rate deviation, performing a code stream duration detection operation on the continuous video recording data according to a set code stream duration detection model to determine the corresponding code stream duration deviation, wherein the set code stream duration detection model is a comprehensive feature calculated by a progressive fusion algorithm based on target features, contour features and character features, and the comprehensive feature is input into a preset God General network model for model training to obtain the result; Determine the corresponding abnormal video data and the abnormal monitoring video time corresponding to the abnormal video data according to the bitstream frame rate deviation and the bitstream duration deviation, and send a video download request to the camera according to the monitoring loophole video time and the abnormal monitoring video time, so that after the camera receives the video download request, it sends the video file corresponding to the monitoring loophole video time and the abnormal monitoring video time to the video monitoring platform for abnormal video supplementary storage operation, and determines the corresponding complete monitoring video file.

[0009] Furthermore, before the video information reading operation is performed on the camera according to the preset video detection rule, the method includes: Perform unique coding operations on all cameras to determine the corresponding camera codes; The corresponding video detection rule is determined according to the camera code, the preset video detection interval, the preset video detection time and the preset video file validity spot check frequency.

[0010] Further, the performing a code stream frame rate calculation operation on the continuous video data to determine the corresponding actual code stream frame rate includes: Demultiplexing the continuous video data according to a preset demultiplexing standard to determine the corresponding continuous video data, and framing the continuous video data according to a preset framing standard to determine the corresponding continuous video frame sequence; A traversal counting operation is performed on the continuous recording video frame sequence, and a corresponding actual code stream frame rate is determined according to the total number of video frames obtained after the traversal counting operation and the time of the continuous recording video frame sequence.

[0011] Furthermore, before performing a code stream duration detection operation on the continuous video data according to the set code stream duration detection model to determine the corresponding code stream duration deviation, the method includes: Performing a feature extraction operation on a preset video frame data set according to a preset target detection model to determine corresponding target features, wherein the video frame data set is a data set including time-stamped video frame labels, and the target features include a bounding box, a confidence level, and category information; Performing a contour extraction operation on the video frame data set according to a preset contour extraction algorithm to determine corresponding contour features, wherein the contour features include contour length, contour area, contour perimeter, and contour shape factor; Performing a character recognition operation on the video frame data set according to a preset optical character recognition algorithm to determine corresponding character features, wherein the character features include character shape, character position, character confidence, and character font; A feature fusion operation is performed on the target feature, the contour feature and the character feature according to a preset progressive fusion algorithm, and the comprehensive feature obtained after the feature fusion operation is input into a preset neural network model for model training to determine the corresponding code stream duration detection model.

[0012] Furthermore, the step of performing a feature fusion operation on the target feature, the contour feature, and the character feature according to a preset progressive fusion algorithm, inputting the comprehensive feature obtained after the feature fusion operation into a preset neural network model for model training, and determining a corresponding bitstream duration detection model includes: Performing an initial fusion operation on the target feature and the contour feature to determine a corresponding first fusion feature, performing an intermediate fusion operation on the first fusion feature and the character feature according to a dynamic bilateral cross fusion algorithm to determine a corresponding second fusion feature, and performing a final fusion operation on the first fusion feature and the second fusion feature to determine a corresponding comprehensive feature; The comprehensive features are input into a preset neural network model for model training to determine a corresponding bitstream duration detection model.

[0013] Further, the performing a code stream duration detection operation on the continuous video data according to a preset code stream duration detection model to determine a corresponding code stream duration deviation includes: Demultiplexing the continuous video data according to a preset demultiplexing standard to determine the corresponding continuous video data, and framing the continuous video data according to a preset framing standard to determine the corresponding continuous video frame sequence; Performing character recognition operation on the continuous recorded video frame sequence according to a preset bitstream duration detection model to determine the actual playback duration of the corresponding video frame; The actual playback duration of the video frame is compared with the preset normal playback duration to determine the corresponding bitstream duration deviation.

[0014] Further, the determining corresponding abnormal video data and the monitoring abnormal video time corresponding to the abnormal video data according to the bitstream frame rate deviation and the bitstream duration deviation includes: Determine whether the bitstream frame rate deviation and the bitstream duration deviation are within a normal range; If not, the continuous video data within the abnormal range is subjected to an abnormal marking operation to determine the corresponding abnormal video data and the monitoring abnormal video time corresponding to the abnormal video data.

[0015] In a second aspect, the present application provides a supplementary storage device for abnormal storage of video on a monitoring platform, comprising: A recording schedule construction module is used to read the recording information of the camera according to the preset recording detection rules, compare the camera recording time obtained after the recording information reading operation with the preset video monitoring platform recording time, and determine the corresponding monitoring recording schedule, wherein the monitoring recording schedule includes the monitoring vulnerability recording time and the monitoring continuous recording time; The continuous monitoring abnormality judgment module is used to read the video information of the video file corresponding to the continuous video recording time of the monitoring, obtain the corresponding continuous video data, perform a code stream frame rate calculation operation on the continuous video data, determine the corresponding actual code stream frame rate, compare the actual code stream frame rate with the original code stream frame rate of the camera, determine the corresponding code stream frame rate deviation, perform a code stream duration detection operation on the continuous video data according to a set code stream duration detection model, and determine the corresponding code stream duration deviation, wherein the set code stream duration detection model is obtained by calculating the comprehensive features through a progressive fusion algorithm based on target features, contour features and character features, and the comprehensive features are input into a preset God General network model for model training; The abnormal video supplement storage module is used to determine the corresponding abnormal video data and the monitoring abnormal video time corresponding to the abnormal video data according to the bit stream frame rate deviation and the bit stream duration deviation, and send a video download request to the camera according to the monitoring loophole video time and the monitoring abnormal video time, so that after the camera receives the video download request, it sends the video file corresponding to the monitoring loophole video time and the monitoring abnormal video time to the video monitoring platform for abnormal video supplement storage operation, and determines the corresponding complete monitoring video file.

[0016] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the method for supplementary storage of abnormal video storage of the monitoring platform are implemented.

[0017] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for supplementing storage of abnormal video storage of a monitoring platform.

[0018] In a fifth aspect, the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the supplementary storage method for abnormal storage of video on a monitoring platform.

[0019] It can be seen from the above technical scheme that the present application provides a method and device for supplementary storage of abnormal video storage of a monitoring platform, which reads the recording information of a specified camera, compares the recording time in the camera with the recording time of a preset video monitoring platform, obtains the loophole recording time and continuous recording time of the video in the monitoring platform, reads the recording information of the recording file corresponding to the continuous recording time, performs frame rate calculation on the recording information to obtain the bitstream frame rate deviation, performs bitstream duration detection on the recording information by setting a bitstream duration detection model, obtains the bitstream duration deviation, judges the abnormal recording time within the continuous recording time by the bitstream frame rate deviation and the bitstream duration deviation, and then sends a recording download request to the camera for the loophole recording time and the abnormal recording time within the continuous recording time to supplement the storage of the abnormal video, and obtains a complete monitoring video file, thereby improving the detection efficiency and accuracy of abnormal monitoring video recording. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 One of the flow diagrams of the supplementary storage method for abnormal storage of video on the monitoring platform in the embodiment of the present application; Figure 2 The second flowchart of the method for supplementing storage of abnormal video storage of the monitoring platform in the embodiment of the present application is shown; Figure 3 The third flowchart of the method for supplementing storage of abnormal video storage of the monitoring platform in the embodiment of the present application; Figure 4 The fourth flowchart of the method for supplementing storage of abnormal video storage of the monitoring platform in the embodiment of the present application; Figure 5 The fifth flowchart of the method for supplementing storage of abnormal video storage of the monitoring platform in the embodiment of the present application; Figure 6 6 is a flow chart of a method for supplementing storage of abnormal video storage of a monitoring platform in an embodiment of the present application; Figure 7 FIG7 is a flow chart of a supplementary storage method for abnormal storage of video on a monitoring platform in an embodiment of the present application; Figure 8 A structural diagram of a supplementary storage device for abnormal storage of video on a monitoring platform in an embodiment of the present application; Fig. 9 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application.

[0022] Reference numerals: Electronic device 9600, central processing unit 9100, memory 9140, communication module 9110, input unit 9120, audio processor 9130, display 9160, power supply 9170, buffer memory 9141, application / function storage unit 9142, data storage unit 9143, driver program storage unit 9144, antenna 9111, speaker 9131, microphone 9132. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] The acquisition, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0025] Considering the existing video recording storage methods, there are problems such as the inability to guarantee the integrity of the recording, the difficulty in supplementing lost recordings, and the inconvenience of manual query methods. The present application provides a method and device for supplementing abnormal storage of video on a monitoring platform, which reads the recording information of a specified camera, compares the recording time in the camera with the preset video monitoring platform recording time, obtains the leaky recording time and continuous recording time of the video on the monitoring platform, reads the recording information of the recording file corresponding to the continuous recording time, performs frame rate calculation on the recording information to obtain the code stream frame rate deviation, performs code stream duration detection on the recording information by setting a code stream duration detection model, obtains the code stream duration deviation, judges the abnormal recording time within the continuous recording time by the code stream frame rate deviation and the code stream duration deviation, and then sends a recording download request to the camera for the abnormal recording time within the leaky recording time and the continuous recording time to supplement the abnormal video storage, and obtains a complete monitoring video file, thereby improving the detection efficiency and accuracy of abnormal monitoring video recording.

[0026] In order to improve the detection efficiency and accuracy of abnormal monitoring video recordings, the present application provides an embodiment of a supplementary storage method for abnormal storage of video on a monitoring platform, see Figure 1 The supplementary storage method for abnormal video storage of the monitoring platform specifically includes the following contents: Step S101: performing a video information reading operation on a camera according to a preset video detection rule, comparing the camera recording time obtained after the video information reading operation with the preset video monitoring platform recording time, and determining a corresponding monitoring recording schedule, wherein the monitoring recording schedule includes a monitoring vulnerability recording time and a monitoring continuous recording time; Optionally, the surveillance video storage in the video surveillance platform is based on the recording time of the camera, but due to network fluctuations and other reasons, the video data platform does not receive the camera's recording information at all, resulting in a loophole time, that is, the camera recorded the video information, but the monitoring platform did not store it.

[0027] Optionally, in this embodiment, in order to address the above-mentioned problem and preliminarily detect the integrity of the video files stored in the video recording platform, a detection task is first formulated, the recording time of the camera is read according to the task requirements, and compared with the recording time of the video surveillance platform to find out the recording vulnerability time of the video surveillance platform.

[0028] Specifically, in terms of hardware deployment, deploy a video surveillance platform, then connect the camera to the video surveillance platform, and implement the video recording on demand function. Finally, deploy a supplementary storage program so that it can read the video information of the video platform and recorder as well as the stored video files.

[0029] Specifically, in terms of the detection process, the detection task is completed by the supplementary storage program. First, the detection program in the supplementary storage program is used to formulate the task, select the camera to be detected, the recording time (usually the day before the detection), the detection start (usually in the early morning), the time interval, and the frequency of spot checks on the validity of the video files. Among them, the camera can be a single camera or a group of cameras. Each camera in a group of cameras has a unique identification code to distinguish each other; Secondly, the detection program reads the recording information of the specified camera / camera group at regular intervals and obtains the returned result. The recording time of each camera / camera group is merged to obtain the recording time of the complete camera. Finally, the complete camera recording time is compared with the recording time stored in the video surveillance platform to find the loophole time of the surveillance video storage in the video surveillance platform and generate a recording schedule, where the recording schedule includes the loophole recording time of the surveillance video storage in the video surveillance platform and the continuous recording time of the surveillance video storage in the video surveillance platform. All existing video file paths and normal duration (recording duration set by the camera) in the continuous recording time are recorded and stored in the database, laying a foundation for subsequent detection of the integrity of the video files in the continuous recording time.

[0030] For example, the program reads the video information of the camera and compares it with the video recording time of the video monitoring platform to generate a monitoring recording schedule. It is found that the recording time set for a certain camera is from 2 am to 5 am, but the platform has missing recording time between 2 am and 3 am for a certain camera. The missing time is recorded as the monitoring vulnerability recording time, and the recording time of a certain camera with a video file after 3 am is recorded as the monitoring continuous recording time. The normal duration of the monitoring continuous recording time is 2 hours from 3 am to 5 am.

[0031] Step S102: performing a video information reading operation on the video file corresponding to the monitoring continuous video recording time to obtain corresponding continuous video recording data, performing a code stream frame rate calculation operation on the continuous video recording data to determine the corresponding actual code stream frame rate, performing a comparison operation on the actual code stream frame rate with the original code stream frame rate of the camera to determine the corresponding code stream frame rate deviation, performing a code stream duration detection operation on the continuous video recording data according to a set code stream duration detection model to determine the corresponding code stream duration deviation, wherein the set code stream duration detection model is a comprehensive feature calculated by a progressive fusion algorithm based on target features, contour features and character features, and the comprehensive feature is input into a preset God General network model for model training to obtain the comprehensive feature; Optionally, in a centralized storage system, the video data may not be stored completely due to network fluctuations, server downtime, or other storage system problems. In this case, although the normal duration extracted from the database is based on the recording time set by the camera, the actual video file may be missing some data, resulting in video deviation. That is, the camera recorded the video information, but the monitoring platform did not store it completely.

[0032] Optionally, in this embodiment, in order to further detect the integrity of the video file during the continuous video recording time to address the above problem, bitstream frame rate detection and bitstream duration detection are applied to determine the integrity of the video file.

[0033] Specifically, for bitstream frame rate detection, first, the bitstream of the video file in the continuous recording time is demultiplexed according to the ISO / IEC 13818-1 standard, and the composite bitstream is split into independent video streams and audio streams; then, according to the video coding specification (ITU-T H.264 or T-REC-H.265), the video data is parsed into several frame groups, and then the actual frame rate calculated by the ratio of the total number of video frames to time is compared with the frame rate originally configured by the camera to obtain the frame rate deviation.

[0034] It is understandable that frame rate is an important factor affecting video fluency. By detecting the deviation between the actual frame rate and the configured frame rate, it is possible to determine whether the video has quality problems such as freezes and frame drops, thereby evaluating the overall quality of the video. In a video surveillance system, a stable frame rate is essential for real-time monitoring and post-playback. By detecting frame rate deviation, the fluency and integrity of the surveillance video can be ensured, improving the reliability and effectiveness of the surveillance system.

[0035] Specifically, the code stream duration detection is also built on the basis of demultiplexing and framing. By setting the duration detection model and using the character overlay technology to parse the timestamp of each frame, the actual duration of the video is calculated through character overlay analysis. The calculated duration is compared with the expected recording duration (provided by the camera's recording settings and database) to obtain the duration deviation.

[0036] More specifically, the construction of the duration detection model is divided into an image preprocessing module, a character detection module, a character recognition module, and a character overlay analysis module.

[0037] Image preprocessing module: Use the open source frame extraction tool OpenCV to extract frames from the video to achieve frame-by-frame analysis. Then, use data enhancement techniques (denoising, contrast enhancement, brightness adjustment) to optimize the image quality of the video frames. By applying the median filter enhancement algorithm on each frame, the visibility of characters and details in the video frames is improved, thereby providing better input for the target detection algorithm. This process helps to analyze the video content in detail and provides a basis for subsequent character detection and duration deviation calculation.

[0038] Character detection module: This module includes target detection algorithm and contour extraction algorithm. First, the pre-trained target detection model YOLO is used to detect the character area in the video frame. The YOLO model can quickly and accurately locate the position of the character in the new image by learning a large amount of image data with annotated character areas during the training process. It will output the bounding box of the character area, including the center coordinates (x, y), width and height of the bounding box. Then, the edge detection algorithm is used to determine the boundary of the character. The gradient amplitude and direction of the image are calculated by calculating the difference in the horizontal and vertical directions of the image. The gradient amplitude indicates the degree of brightness change of the image at each pixel, while the gradient direction indicates the direction of brightness change; the gradient amplitude is non-maximum suppressed to retain only the gradient amplitude of the local maximum value, and other non-edge pixels are removed. A high threshold and a low threshold are set. Pixels with gradient amplitudes higher than the high threshold are considered to be strong edges. Pixels with gradient amplitudes between the high and low thresholds are also considered to be edges if they are connected to strong edge pixels, otherwise they are suppressed, so that clear character contours can be obtained.

[0039] Character recognition module: Use OCR technology to convert the detected character area into text information. The extracted character area image is input into Tesseract, which will recognize the characters by analyzing the strokes, shapes and other features in the image.

[0040] Character overlay analysis module: Apply context technology and combine video content to analyze the intention and meaning of character overlay. The temporal characters in different frames will change over time. By analyzing the changes in the characters in the time series, the temporal relationship of the characters can be determined.

[0041] For example, the character overlay contains time information, such as "2024-06-11 14:30:25". By analyzing these time characters, the actual start and end time of the video clip can be accurately determined. By comparing the analyzed time information with the duration recorded in the metadata of the video file, the actual duration of the recording can be calibrated to find out whether the recording is cut off early or delayed.

[0042] For another example, studying the changes in time characters in different frames can detect the continuity of video time. In normal video recordings, time characters should be continuously increasing. If there is a time jump, such as a sudden change from "14:30:25" to "14:30:40", it means that 15 seconds of video may be lost in the middle. In scenarios where the chronological order of events needs to be accurately traced, such as accident investigations and security monitoring, it is crucial to ensure the integrity of the video.

[0043] More specifically, the training of the duration detection model requires preparing the data required for object detection, contour extraction, and optical character recognition and performing necessary preprocessing. The data is labeled to ensure that each sample used to train the model has a corresponding object detection box, contour information, and character recognition result.

[0044] The dataset is input into the target detection model YOLOv5 to extract target features, which include bounding boxes, confidence levels, and category information. For each detected object, the model outputs a bounding box represented by four coordinate values, namely (x_min, y_min, x_max, y_max); the model also outputs a confidence level for each detected object, indicating the model's confidence in the object detection; category information is used to identify whether the detected category is data with time.

[0045] The dataset is input into the contour extraction algorithm to extract the contour features of the target, including contour length, contour area, contour perimeter and contour shape factor. For each extracted contour, the contour length, the area of ​​the area enclosed by each contour, the contour perimeter and the contour shape factor are calculated to more comprehensively describe the content in the contour.

[0046] The dataset is input into the OCR model to extract character features, including character shape, character position, character confidence, and character font. The shape features and font features of each recognized character are extracted, the position of the character in the image is extracted, and the confidence is output to accurately locate and output the correct time text information for subsequent character overlay analysis.

[0047] The features of the above different modalities are mapped to the same dimension, and the target features and contour features are preliminarily fused using the splicing method to form a feature vector; then, the preliminarily fused features are dynamically bilaterally cross-fused with the character features. For the two modalities, the uncertainty estimator is used to evaluate the uncertainty of the model prediction. The modality with lower uncertainty is given a higher weight, and vice versa. The weights are dynamically generated through uncertainty estimation, and the generated dynamic weights are used to perform weighted fusion of the features of different modalities. By dynamically adjusting the fusion weights, the information of high-quality modalities can be fully utilized, while the interference of low-quality modalities can be reduced, thereby significantly improving the robustness and performance of the model; finally, the feature map obtained by the encoder is gradually compressed and transmitted to the subsequent decoder through a progressive compression module PCM, making full use of the high-level features of the image, and finally fusing all features to form a comprehensive feature vector.

[0048] The fused comprehensive feature vector is input into a new neural network model, the model is trained using labeled data, and the parameters of each module algorithm are reversely updated using the optimal loss function to obtain a trained bitstream duration detection model that can predict the bitstream duration based on the fused features.

[0049] It is understandable that, due to the certain correlation between target detection and character recognition tasks, target detection can provide prior information of spatial position for character recognition. During the training process, preferably, the entire model is optimized end-to-end using a unified training set, so that the model automatically learns the correlation between the two tasks. For example, the model can learn that when a target area of ​​a certain shape is detected, it is more inclined to recognize certain types of characters. This joint learning enables the model to better utilize the complementary information between tasks and improve overall performance. At the same time, the target detection model provides the position information of the character area, the contour extraction model provides the shape boundary information of the character, and the OCR model provides the text content information of the character. This information complements each other and can more comprehensively describe the character features. For example, when the character is partially blocked or blurred, the position information and shape information can assist the recognition of text information and improve the accuracy of duration detection. Moreover, through multimodal feature fusion, more discriminative feature representations can be learned. The fused features can better reflect the state of the character in the video bitstream, which helps the duration detection model to more accurately judge the duration anomaly. For example, the fused features can include multiple information such as the integrity of the character area, the coherence of the character shape, and the accuracy of the character text, providing a richer basis for duration detection.

[0050] More specifically, the application of the duration detection model is to input the processed real-time video frame data into the duration detection model. The model will identify the area containing characters, and then further process the target detected character area to accurately determine the boundaries of the characters. Finally, the character area after contour extraction is sent to the OCR system to convert the characters in the image into text. Through the time series analysis and character superposition analysis of the text, the dynamic changes of the recording time are detected, and the increase and decrease trend of the video time is judged, so as to further accurately infer the actual duration of the code stream, and determine the deviation of the code stream duration by comparing the actual duration with the normal market preset in the database.

[0051] Step S103: Determine the corresponding abnormal video data and the monitoring abnormal video time corresponding to the abnormal video data according to the bitstream frame rate deviation and the bitstream duration deviation, and send a video download request to the camera according to the monitoring vulnerability video time and the monitoring abnormal video time, so that after the camera receives the video download request, it sends the video file corresponding to the monitoring vulnerability video time and the monitoring abnormal video time to the video monitoring platform for abnormal video supplementary storage operation, and determines the corresponding complete monitoring video file.

[0052] Optionally, in this embodiment, the video anomaly is determined by jointly judging the bitstream duration deviation and the bitstream frame rate deviation, and the time of the abnormal video file and the time of the flawed video file are used to send a video file download request to the camera to complete the abnormal video and the flawed video into a complete video.

[0053] Specifically, if the calculated average video frame rate is less than the frame rate configured by the camera within a certain floating range, then the actual frame rate is judged to be abnormal; if the calculated video recording time is less than the database recording time, then the actual recording time is judged to be abnormal; thus, the abnormal file is placed in the abnormal data recovery area.

[0054] Specifically, the above vulnerability detection and anomaly detection tasks are executed in a concurrent loop.

[0055] Specifically, for abnormal data and vulnerability data, a request for downloading video data within the time is sent to the corresponding camera according to the corresponding abnormal time and vulnerability time, and the data of the camera is automatically used to supplement the abnormal data on the storage platform.

[0056] It is understandable that, because the preset time of the cyclic detection task is preferably 1 day, the period from the discovery of abnormal video data to the supplementary storage of abnormal data will not exceed the 3-day storage time of the camera, and data overwriting will not occur. At the same time, after the abnormal data is discovered, the video supplementary storage records are merged to generate a video supplementary storage detection table based on the camera and date, and the results are stored in the database. The results and logs are provided to the user interface for display through an interface or other means. In this way, it is convenient for users to query abnormal data in a timely manner, and a path and basis are provided for manual supplementation of abnormal data.

[0057] This example demonstrates how this embodiment performs vulnerability and anomaly detection on the surveillance video platform video through concurrent polling, and performs video supplementary storage based on the detection results to obtain a complete video file.

[0058] From the above description, it can be seen that the supplementary storage method for abnormal storage of video on the monitoring platform provided in the embodiment of the present application can read the recording information of the specified camera, compare the recording time in the camera with the recording time of the preset video monitoring platform, obtain the loophole recording time and continuous recording time of the video in the monitoring platform, read the recording information of the recording file corresponding to the continuous recording time, perform frame rate calculation on the recording information to obtain the bitstream frame rate deviation, perform bitstream duration detection on the recording information by setting a bitstream duration detection model to obtain the bitstream duration deviation, judge the abnormal recording time within the continuous recording time by the bitstream frame rate deviation and the bitstream duration deviation, and then send a recording download request to the camera for the loophole recording time and the abnormal recording time within the continuous recording time to perform supplementary storage of the abnormal video, and obtain a complete monitoring video file, thereby improving the detection efficiency and accuracy of abnormal monitoring video recording.

[0059] In one embodiment of the supplementary storage method for abnormal video storage of the monitoring platform of the present application, see Figure 2 , and can also include the following: Step S201: Perform unique coding operations on all cameras to determine corresponding camera codes; Step S202: Determine a corresponding video detection rule according to the camera code, a preset video detection interval, a preset video detection time, and a preset video file validity spot check frequency.

[0060] Optionally, in this embodiment, the camera is connected to a video monitoring platform and implements a video recording on demand function.

[0061] Optionally, in this embodiment, the camera may be one camera or a group of cameras, and each camera in the group of cameras has a unique identification code for distinguishing each other.

[0062] Optionally, in this embodiment, the detection program is used to formulate tasks, select the camera to be detected, the recording time (usually the day before the detection), the detection start (usually in the early morning), the time interval, and the frequency of spot checks on the validity of the video files.

[0063] Through step S202, this embodiment constructs a polling detection task, realizes automatic fixed-point detection, and lays a foundation for subsequent video anomaly recognition.

[0064] In one embodiment of the supplementary storage method for abnormal video storage of the monitoring platform of the present application, see Figure 3 , and can also include the following: Step S301: performing a demultiplexing operation on the continuous video data according to a preset demultiplexing standard to determine corresponding continuous video data, and performing a framing operation on the continuous video data according to a preset framing standard to determine a corresponding continuous video frame sequence; Step S302: performing a traversal counting operation on the continuous video frame sequence, and determining a corresponding actual bitstream frame rate according to the total number of video frames obtained after the traversal counting operation and the time of the continuous video frame sequence.

[0065] Optionally, in this embodiment, first, the code stream of the video file in the continuous video recording time is demultiplexed according to the ISO / IEC 13818-1 standard, and the composite code stream is split into independent video streams and audio streams; then, according to the video coding specification (ITU-T H.264 or T-REC-H.265), the video data is parsed into a number of frame group sequences, and then the video frame sequence is traversed, and the actual frame rate calculated by the ratio of the total number of video frames to the time is compared with the frame rate originally configured by the camera to obtain the frame rate deviation.

[0066] It is understandable that frame rate is an important factor affecting video fluency. By detecting the deviation between the actual frame rate and the configured frame rate, it is possible to determine whether the video has quality problems such as freezes and frame drops, thereby evaluating the overall quality of the video. In a video surveillance system, a stable frame rate is essential for real-time monitoring and post-playback. By detecting frame rate deviation, the fluency and integrity of the surveillance video can be ensured, improving the reliability and effectiveness of the surveillance system.

[0067] Through step S302, this embodiment realizes the video frame rate calculation, laying a foundation for the subsequent identification of abnormal video data.

[0068] In one embodiment of the supplementary storage method for abnormal video storage of the monitoring platform of the present application, see Figure 4 , and can also include the following: Step S401: performing a feature extraction operation on a preset video frame data set according to a preset target detection model to determine corresponding target features, wherein the video frame data set is a data set containing time-stamped video frame labels, and the target features include a bounding box, a confidence level, and category information; Step S402: performing a contour extraction operation on the video frame data set according to a preset contour extraction algorithm to determine corresponding contour features, wherein the contour features include contour length, contour area, contour perimeter, and contour shape factor; Step S403: performing a character recognition operation on the video frame data set according to a preset optical character recognition algorithm to determine corresponding character features, wherein the character features include character shape, character position, character confidence, and character font; Step S404: performing a feature fusion operation on the target feature, the contour feature and the character feature according to a preset progressive fusion algorithm, inputting the comprehensive feature obtained after the feature fusion operation into a preset neural network model for model training, and determining a corresponding bitstream duration detection model.

[0069] Optionally, in this embodiment, this process is a model training process.

[0070] Specifically, prepare the data required for target detection, contour extraction, and optical character recognition and perform necessary preprocessing. Label the data to ensure that each sample used to train the model has a corresponding target detection box, contour information, and character recognition result. It is understandable that using a unified training set for labeling can ensure the consistency of the labeling, and sharing features can enable the model to optimize these features from multiple angles, which helps to improve the generalization ability of the features and enable them to better adapt to different characters and scenes.

[0071] The dataset is input into the target detection model YOLOv5 to extract target features, which include bounding boxes, confidence levels, and category information. For each detected object, the model outputs a bounding box represented by four coordinate values, namely (x_min, y_min, x_max, y_max); the model also outputs a confidence level for each detected object, indicating the model's confidence in the object detection; category information is used to identify whether the detected category is data with time.

[0072] The dataset is input into the contour extraction algorithm to extract the contour features of the target, including contour length, contour area, contour perimeter and contour shape factor. For each extracted contour, the contour length, the area of ​​the area enclosed by each contour, the contour perimeter and the contour shape factor are calculated to more comprehensively describe the content in the contour.

[0073] The dataset is input into the OCR model to extract character features, including character shape, character position, character confidence, and character font. The shape features and font features of each recognized character are extracted, the position of the character in the image is extracted, and the confidence is output to accurately locate and output the correct time text information for subsequent character overlay analysis.

[0074] The features of the above different modes are mapped to the same dimension, and the fused comprehensive features are obtained using a progressive fusion algorithm. The comprehensive features are then input into a new neural network model for model training to obtain the final bitstream duration detection model.

[0075] Through step S404, this embodiment successfully obtains a code stream duration detection model, laying a foundation for subsequent detection of the actual duration of the code stream.

[0076] In one embodiment of the supplementary storage method for abnormal video storage of the monitoring platform of the present application, see Figure 5 , and can also include the following: Step S501: performing an initial fusion operation on the target feature and the contour feature to determine a corresponding first fusion feature, performing an intermediate fusion operation on the first fusion feature and the character feature according to a dynamic bilateral cross fusion algorithm to determine a corresponding second fusion feature, performing a final fusion operation on the first fusion feature and the second fusion feature to determine a corresponding comprehensive feature; Step S502: input the comprehensive features into a preset neural network model for model training to determine a corresponding bitstream duration detection model.

[0077] Optionally, in this embodiment, the target features and contour features are preliminarily fused by splicing to form a feature vector; then, the preliminarily fused features are dynamically bilaterally cross-fused with the character features, and for the two modalities, an uncertainty estimator is used to evaluate the uncertainty of the model prediction. The modality with lower uncertainty is given a higher weight, and vice versa. The weights are dynamically generated through uncertainty estimation, and the generated dynamic weights are used to perform weighted fusion of features of different modalities. By dynamically adjusting the fusion weights, the information of high-quality modalities can be fully utilized, while the interference of low-quality modalities can be reduced, thereby significantly improving the robustness and performance of the model; finally, the feature map obtained by the encoder is gradually compressed and transmitted to the subsequent decoder through a progressive compression module PCM, making full use of the high-level features of the image, and finally fusing all features to form a comprehensive feature vector.

[0078] The fused comprehensive feature vector is input into a new neural network model, the model is trained using labeled data, and the parameters of each module algorithm are reversely updated using the optimal loss function to obtain a trained bitstream duration detection model that can predict the bitstream duration based on the fused features.

[0079] Through step S502, this embodiment successfully obtains a code stream duration detection model through multi-modal fusion, laying a foundation for subsequent detection of the actual duration of the code stream.

[0080] In one embodiment of the supplementary storage method for abnormal video storage of the monitoring platform of the present application, see Figure 6 , and can also include the following: Step S601: performing a demultiplexing operation on the continuous video data according to a preset demultiplexing standard to determine corresponding continuous video data, and performing a framing operation on the continuous video data according to a preset framing standard to determine a corresponding continuous video frame sequence; Step S602: performing a character recognition operation on the continuous recorded video frame sequence according to a preset bitstream duration detection model to determine the actual playback duration of the corresponding video frame; Step S603: Compare the actual playback duration of the video frame with the preset normal playback duration to determine the corresponding bitstream duration deviation.

[0081] Optionally, in this embodiment, the recorded video is demultiplexed and framed in the same manner as step S301.

[0082] Optionally, in this embodiment, the subsequent application is the code stream duration detection model. The application process inputs the processed real-time video frame data into the duration detection model. The model will identify the area containing characters, and then further process the target detected character area to accurately determine the boundaries of the characters. Finally, the character area after contour extraction is sent to the OCR system to convert the characters in the image into text. Through the time series analysis and character superposition analysis of the text, the dynamic changes of the recording time are detected, and the increase and decrease trend of the video time is judged, so as to further accurately infer the actual duration of the code stream, and determine the code stream duration deviation by comparing the actual duration with the normal market preset in the database.

[0083] Through step S602, this embodiment successfully obtains the actual duration of the bitstream through the bitstream duration detection model, and obtains the duration deviation through comparison, laying a foundation for subsequent judgment of abnormal video files.

[0084] In one embodiment of the supplementary storage method for abnormal video storage of the monitoring platform of the present application, see Figure 7 , and can also include the following: Step S701: determining whether the bitstream frame rate deviation and the bitstream duration deviation are within a normal range; Step S702: If not, the continuous video data within the abnormal range is subjected to an abnormal marking operation to determine the corresponding abnormal video data and the monitoring abnormal video time corresponding to the abnormal video data.

[0085] Optionally, in this embodiment, the video anomaly is determined by jointly judging the bitstream duration deviation and the bitstream frame rate deviation, and the time of the abnormal video file and the time of the flawed video file are used to send a video file download request to the camera to complete the abnormal video and the flawed video into a complete video.

[0086] Specifically, if the calculated average video frame rate is less than the frame rate configured by the camera within a certain floating range, then the actual frame rate is judged to be abnormal; if the calculated video recording time is less than the database recording time, then the actual recording time is judged to be abnormal; thus, the abnormal file is placed in the abnormal data recovery area.

[0087] Through step S702, this embodiment successfully obtains the abnormal video file, and obtains the abnormal time corresponding to the route file through the information of the abnormal video file.

[0088] In order to improve the detection efficiency and accuracy of abnormal monitoring video recordings, the present application provides an embodiment of a monitoring platform video abnormal storage supplementary storage device for implementing all or part of the content of the monitoring platform video abnormal storage supplementary storage method, see Figure 8 The supplementary storage device for abnormal video storage of the monitoring platform specifically includes the following contents: The recording schedule construction module 10 is used to read the recording information of the camera according to the preset recording detection rules, compare the camera recording time obtained after the recording information reading operation with the preset video monitoring platform recording time, and determine the corresponding monitoring recording schedule, wherein the monitoring recording schedule includes the monitoring vulnerability recording time and the monitoring continuous recording time; The continuous monitoring abnormality judgment module 20 is used to read the video information of the video file corresponding to the continuous video recording time of the monitoring, obtain the corresponding continuous video data, perform a code stream frame rate calculation operation on the continuous video data, determine the corresponding actual code stream frame rate, compare the actual code stream frame rate with the original code stream frame rate of the camera, determine the corresponding code stream frame rate deviation, perform a code stream duration detection operation on the continuous video data according to a set code stream duration detection model, and determine the corresponding code stream duration deviation, wherein the set code stream duration detection model is obtained by calculating the comprehensive features through a progressive fusion algorithm based on the target features, contour features and character features, and inputting the comprehensive features into a preset God General network model for model training; The abnormal video supplement storage module 30 is used to determine the corresponding abnormal video data and the monitoring abnormal video time corresponding to the abnormal video data according to the bit stream frame rate deviation and the bit stream duration deviation, and send a video download request to the camera according to the monitoring loophole video time and the monitoring abnormal video time, so that after the camera receives the video download request, it sends the video file corresponding to the monitoring loophole video time and the monitoring abnormal video time to the video monitoring platform for abnormal video supplement storage operation, and determines the corresponding complete monitoring video file.

[0089] From the above description, it can be seen that the supplementary storage device for abnormal storage of video on the monitoring platform provided in the embodiment of the present application can read the recording information of the specified camera, compare the recording time in the camera with the recording time of the preset video monitoring platform, obtain the loophole recording time and continuous recording time of the video in the monitoring platform, read the recording information of the recording file corresponding to the continuous recording time, perform frame rate calculation on the recording information to obtain the bitstream frame rate deviation, perform bitstream duration detection on the recording information by setting a bitstream duration detection model to obtain the bitstream duration deviation, judge the abnormal recording time within the continuous recording time by the bitstream frame rate deviation and the bitstream duration deviation, and then send a recording download request to the camera for the loophole recording time and the abnormal recording time within the continuous recording time to perform supplementary storage of the abnormal video, and obtain a complete monitoring video file, thereby improving the detection efficiency and accuracy of abnormal monitoring video recording.

[0090] From the hardware level, in order to improve the detection efficiency and accuracy of abnormal monitoring video recordings, the present application provides an embodiment of an electronic device for implementing all or part of the content of the supplementary storage method for abnormal storage of video on the monitoring platform, and the electronic device specifically includes the following content: Processor, memory, communication interface and bus; wherein the processor, memory and communication interface communicate with each other through the bus; the communication interface is used to realize the information transmission between the supplementary storage method of abnormal video storage of the monitoring platform and the core business system, user terminal and related database and other related devices; the logic controller can be a desktop computer, a tablet computer and a mobile terminal, etc., and the present embodiment is not limited thereto. In the present embodiment, the logic controller can be implemented with reference to the embodiment of the supplementary storage method of abnormal video storage of the monitoring platform and the embodiment of the supplementary storage method of abnormal video storage of the monitoring platform in the embodiment, and the contents thereof are incorporated herein, and the repeated parts are not repeated.

[0091] It is understandable that the user terminal may include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a personal digital assistant (PDA), a vehicle-mounted device, a smart wearable device, etc. Among them, the smart wearable device may include smart glasses, a smart watch, a smart bracelet, etc.

[0092] In practical applications, part of the supplementary storage method for abnormal storage of video on the monitoring platform can be executed on the electronic device side as described above, or all operations can be completed in the client device. The specific selection can be based on the processing capability of the client device and the limitations of the user's usage scenario. This application does not limit this. If all operations are completed in the client device, the client device may also include a processor.

[0093] The client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side, and other implementation scenarios may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, or a server cluster consisting of multiple servers, or a server structure of a distributed device.

[0094] Fig. 9 FIG. 9 is a schematic block diagram of the system structure of the electronic device 9600 according to an embodiment of the present application. Fig. 9 As shown, the electronic device 9600 may include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. It is worth noting that Fig. 9 is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0095] In one embodiment, the supplementary storage method function of the monitoring platform video abnormality storage can be integrated into the central processor 9100. The central processor 9100 can be configured to perform the following control: Step S101: performing a video information reading operation on a camera according to a preset video detection rule, comparing the camera recording time obtained after the video information reading operation with the preset video monitoring platform recording time, and determining a corresponding monitoring recording schedule, wherein the monitoring recording schedule includes a monitoring vulnerability recording time and a monitoring continuous recording time; Step S102: performing a video information reading operation on the video file corresponding to the monitoring continuous video recording time to obtain corresponding continuous video recording data, performing a code stream frame rate calculation operation on the continuous video recording data to determine the corresponding actual code stream frame rate, performing a comparison operation on the actual code stream frame rate with the original code stream frame rate of the camera to determine the corresponding code stream frame rate deviation, performing a code stream duration detection operation on the continuous video recording data according to a set code stream duration detection model to determine the corresponding code stream duration deviation, wherein the set code stream duration detection model is a comprehensive feature calculated by a progressive fusion algorithm based on target features, contour features and character features, and the comprehensive feature is input into a preset God General network model for model training to obtain the comprehensive feature; Step S103: Determine the corresponding abnormal video data and the monitoring abnormal video time corresponding to the abnormal video data according to the bitstream frame rate deviation and the bitstream duration deviation, and send a video download request to the camera according to the monitoring vulnerability video time and the monitoring abnormal video time, so that after the camera receives the video download request, it sends the video file corresponding to the monitoring vulnerability video time and the monitoring abnormal video time to the video monitoring platform for abnormal video supplementary storage operation, and determines the corresponding complete monitoring video file.

[0096] From the above description, it can be seen that the electronic device provided by the embodiment of the present application reads the recording information of the specified camera, compares the recording time in the camera with the recording time of the preset video monitoring platform, obtains the loophole recording time and continuous recording time of the video in the monitoring platform, reads the recording information of the recording file corresponding to the continuous recording time, performs frame rate calculation on the recording information to obtain the bitstream frame rate deviation, performs bitstream duration detection on the recording information by setting a bitstream duration detection model to obtain the bitstream duration deviation, judges the abnormal recording time within the continuous recording time by the bitstream frame rate deviation and the bitstream duration deviation, and then sends a recording download request to the camera for the abnormal recording time within the loophole recording time and the continuous recording time to supplement the storage of the abnormal video, and obtains a complete monitoring video file, thereby improving the detection efficiency and accuracy of abnormal monitoring video recording.

[0097] In another embodiment, the supplementary storage method for storing abnormal video of the monitoring platform can be configured separately from the central processing unit 9100. For example, the supplementary storage method for storing abnormal video of the monitoring platform can be configured as a chip connected to the central processing unit 9100, and the function of the supplementary storage method for storing abnormal video of the monitoring platform can be realized through the control of the central processing unit.

[0098] like Fig. 9As shown, the electronic device 9600 may also include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily have to include Fig. 9 In addition, the electronic device 9600 may also include Fig. 9 For components not shown, reference may be made to the prior art.

[0099] like Fig. 9 As shown, the central processing unit 9100 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor device and / or logic device. The central processing unit 9100 receives input and controls the operation of various components of the electronic device 9600.

[0100] The memory 9140 may be, for example, one or more of a cache, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory or other suitable devices. The above-mentioned information related to the failure may be stored, and a program for executing the relevant information may also be stored. The CPU 9100 may execute the program stored in the memory 9140 to implement information storage or processing, etc.

[0101] The input unit 9120 provides input to the central processing unit 9100. The input unit 9120 is, for example, a key or a touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used to display display objects such as images and texts. The display may be, for example, an LCD display, but is not limited thereto.

[0102] The memory 9140 may be a solid-state memory, such as a read-only memory (ROM), a random access memory (RAM), a SIM card, etc. It may also be a memory that saves information even when the power is off, can be selectively erased, and is provided with more data, examples of which are sometimes referred to as EPROMs, etc. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142, which is used to store application programs and function programs or processes for executing the operation of the electronic device 9600 through the central processor 9100.

[0103] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for communication functions of the electronic device and / or for executing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0104] The communication module 9110 is a transmitter / receiver that sends and receives signals via the antenna 9111. The communication module 9110 is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal.

[0105] Based on different communication technologies, multiple communication modules 9110 may be provided in the same electronic device, such as a cellular network module, a Bluetooth module and / or a wireless local area network module, etc. The communication module 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide an audio output via the speaker 9131 and receive an audio input from the microphone 9132, thereby realizing a common telecommunication function. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. In addition, the audio processor 9130 is also coupled to the central processor 9100, so that recording can be performed on the machine through the microphone 9132, and the sound stored on the machine can be played through the speaker 9131.

[0106] The embodiments of the present application also provide a computer-readable storage medium capable of implementing all the steps of the method for supplementing storage of abnormal video storage of a monitoring platform whose execution subject is a server or a client in the above embodiments. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, all the steps of the method for supplementing storage of abnormal video storage of a monitoring platform whose execution subject is a server or a client in the above embodiments are implemented. For example, when the processor executes the computer program, the following steps are implemented: Step S101: performing a video information reading operation on a camera according to a preset video detection rule, comparing the camera recording time obtained after the video information reading operation with the preset video monitoring platform recording time, and determining a corresponding monitoring recording schedule, wherein the monitoring recording schedule includes a monitoring vulnerability recording time and a monitoring continuous recording time; Step S102: performing a video information reading operation on the video file corresponding to the monitoring continuous video recording time to obtain corresponding continuous video recording data, performing a code stream frame rate calculation operation on the continuous video recording data to determine the corresponding actual code stream frame rate, performing a comparison operation on the actual code stream frame rate with the original code stream frame rate of the camera to determine the corresponding code stream frame rate deviation, performing a code stream duration detection operation on the continuous video recording data according to a set code stream duration detection model to determine the corresponding code stream duration deviation, wherein the set code stream duration detection model is a comprehensive feature calculated by a progressive fusion algorithm based on target features, contour features and character features, and the comprehensive feature is input into a preset God General network model for model training to obtain the comprehensive feature; Step S103: Determine the corresponding abnormal video data and the monitoring abnormal video time corresponding to the abnormal video data according to the bitstream frame rate deviation and the bitstream duration deviation, and send a video download request to the camera according to the monitoring vulnerability video time and the monitoring abnormal video time, so that after the camera receives the video download request, it sends the video file corresponding to the monitoring vulnerability video time and the monitoring abnormal video time to the video monitoring platform for abnormal video supplementary storage operation, and determines the corresponding complete monitoring video file.

[0107] From the above description, it can be seen that the computer-readable storage medium provided in the embodiment of the present application reads the recording information of the specified camera, compares the recording time in the camera with the recording time of the preset video monitoring platform, obtains the loophole recording time and continuous recording time of the video in the monitoring platform, reads the recording information of the recording file corresponding to the continuous recording time, performs frame rate calculation on the recording information to obtain the bitstream frame rate deviation, performs bitstream duration detection on the recording information by setting a bitstream duration detection model to obtain the bitstream duration deviation, judges the abnormal recording time within the continuous recording time by the bitstream frame rate deviation and the bitstream duration deviation, and then sends a recording download request to the camera for the abnormal recording time within the loophole recording time and the continuous recording time to supplement the storage of the abnormal video, and obtains a complete monitoring video file, thereby improving the detection efficiency and accuracy of abnormal monitoring video recording.

[0108] The embodiments of the present application also provide a computer program product capable of implementing all the steps of the method for supplementing storage of abnormal video storage of a monitoring platform in the above-mentioned embodiment, the execution subject of which is a server or a client. When the computer program / instruction is executed by a processor, the steps of the method for supplementing storage of abnormal video storage of a monitoring platform are implemented. For example, the computer program / instruction implements the following steps: Step S101: performing a video information reading operation on a camera according to a preset video detection rule, comparing the camera recording time obtained after the video information reading operation with the preset video monitoring platform recording time, and determining a corresponding monitoring recording schedule, wherein the monitoring recording schedule includes a monitoring vulnerability recording time and a monitoring continuous recording time; Step S102: performing a video information reading operation on the video file corresponding to the monitoring continuous video recording time to obtain corresponding continuous video recording data, performing a code stream frame rate calculation operation on the continuous video recording data to determine the corresponding actual code stream frame rate, performing a comparison operation on the actual code stream frame rate with the original code stream frame rate of the camera to determine the corresponding code stream frame rate deviation, performing a code stream duration detection operation on the continuous video recording data according to a set code stream duration detection model to determine the corresponding code stream duration deviation, wherein the set code stream duration detection model is a comprehensive feature calculated by a progressive fusion algorithm based on target features, contour features and character features, and the comprehensive feature is input into a preset God General network model for model training to obtain the comprehensive feature; Step S103: Determine the corresponding abnormal video data and the monitoring abnormal video time corresponding to the abnormal video data according to the bitstream frame rate deviation and the bitstream duration deviation, and send a video download request to the camera according to the monitoring vulnerability video time and the monitoring abnormal video time, so that after the camera receives the video download request, it sends the video file corresponding to the monitoring vulnerability video time and the monitoring abnormal video time to the video monitoring platform for abnormal video supplementary storage operation, and determines the corresponding complete monitoring video file.

[0109] From the above description, it can be seen that the computer program product provided by the embodiment of the present application reads the recording information of the specified camera, compares the recording time in the camera with the recording time of the preset video monitoring platform, obtains the loophole recording time and continuous recording time of the video in the monitoring platform, reads the recording information of the recording file corresponding to the continuous recording time, performs frame rate calculation on the recording information to obtain the bitstream frame rate deviation, performs bitstream duration detection on the recording information by setting a bitstream duration detection model to obtain the bitstream duration deviation, judges the abnormal recording time within the continuous recording time by the bitstream frame rate deviation and the bitstream duration deviation, and then sends a recording download request to the camera for the abnormal recording time within the loophole recording time and the continuous recording time to supplement the storage of the abnormal video, and obtains a complete monitoring video file, thereby improving the detection efficiency and accuracy of abnormal monitoring video recording.

[0110] It should be understood by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0111] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (apparatus), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0112] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0113] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0114] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for supplementing storage of abnormal video storage on a monitoring platform, characterized in that: The method comprises: Performing a video information reading operation on the camera according to a preset video detection rule, comparing the camera recording time obtained after the video information reading operation with the preset video monitoring platform recording time, and determining a corresponding monitoring video schedule, wherein the monitoring video schedule includes monitoring vulnerability recording time and monitoring continuous recording time; Performing a video information reading operation on the video file corresponding to the monitoring continuous video recording time to obtain corresponding continuous video recording data, performing a code stream frame rate calculation operation on the continuous video recording data to determine the corresponding actual code stream frame rate, performing a comparison operation on the actual code stream frame rate with the original code stream frame rate of the camera to determine the corresponding code stream frame rate deviation, performing a code stream duration detection operation on the continuous video recording data according to a set code stream duration detection model to determine the corresponding code stream duration deviation, wherein the set code stream duration detection model is a comprehensive feature calculated by a progressive fusion algorithm based on target features, contour features and character features, and the comprehensive feature is input into a preset God General network model for model training to obtain the result; Determine the corresponding abnormal video data and the abnormal monitoring video time corresponding to the abnormal video data according to the bitstream frame rate deviation and the bitstream duration deviation, and send a video download request to the camera according to the monitoring loophole video time and the abnormal monitoring video time, so that after the camera receives the video download request, it sends the video file corresponding to the monitoring loophole video time and the abnormal monitoring video time to the video monitoring platform for abnormal video supplementary storage operation, and determines the corresponding complete monitoring video file.

2. The method for supplementing storage of abnormal video storage of a monitoring platform according to claim 1, characterized in that: Before the video information reading operation is performed on the camera according to the preset video detection rule, the method includes: Perform unique coding operations on all cameras to determine the corresponding camera codes; The corresponding video detection rule is determined according to the camera code, the preset video detection interval, the preset video detection time and the preset video file validity spot check frequency.

3. The method for supplementing storage of abnormal video storage of a monitoring platform according to claim 1, characterized in that: The performing a code stream frame rate calculation operation on the continuous video data to determine a corresponding actual code stream frame rate includes: Demultiplexing the continuous video data according to a preset demultiplexing standard to determine the corresponding continuous video data, and framing the continuous video data according to a preset framing standard to determine the corresponding continuous video frame sequence; A traversal counting operation is performed on the continuous recording video frame sequence, and a corresponding actual code stream frame rate is determined according to the total number of video frames obtained after the traversal counting operation and the time of the continuous recording video frame sequence.

4. The method for supplementing storage of abnormal video storage of a monitoring platform according to claim 1, characterized in that: Before performing a code stream duration detection operation on the continuous video data according to the set code stream duration detection model to determine the corresponding code stream duration deviation, the method includes: Performing a feature extraction operation on a preset video frame data set according to a preset target detection model to determine corresponding target features, wherein the video frame data set is a data set including a timestamp video frame label, and the target features include a bounding box, a confidence level, and a category information; Performing a contour extraction operation on the video frame data set according to a preset contour extraction algorithm to determine corresponding contour features, wherein the contour features include contour length, contour area, contour perimeter, and contour shape factor; Performing a character recognition operation on the video frame data set according to a preset optical character recognition algorithm to determine corresponding character features, wherein the character features include character shape, character position, character confidence, and character font; A feature fusion operation is performed on the target feature, the contour feature and the character feature according to a preset progressive fusion algorithm, and the comprehensive feature obtained after the feature fusion operation is input into a preset neural network model for model training to determine the corresponding code stream duration detection model.

5. The method for supplementing storage of abnormal video storage of a monitoring platform according to claim 4 is characterized in that: The step of performing a feature fusion operation on the target feature, the contour feature, and the character feature according to a preset progressive fusion algorithm, inputting the comprehensive feature obtained after the feature fusion operation into a preset neural network model for model training, and determining a corresponding bitstream duration detection model includes: Performing an initial fusion operation on the target feature and the contour feature to determine a corresponding first fusion feature, performing an intermediate fusion operation on the first fusion feature and the character feature according to a dynamic bilateral cross fusion algorithm to determine a corresponding second fusion feature, and performing a final fusion operation on the first fusion feature and the second fusion feature to determine a corresponding comprehensive feature; The comprehensive features are input into a preset neural network model for model training to determine a corresponding bitstream duration detection model.

6. The method for supplementing storage of abnormal video storage of a monitoring platform according to claim 1, characterized in that: The performing a code stream duration detection operation on the continuous video data according to the set code stream duration detection model to determine the corresponding code stream duration deviation includes: Demultiplexing the continuous video data according to a preset demultiplexing standard to determine the corresponding continuous video data, and framing the continuous video data according to a preset framing standard to determine the corresponding continuous video frame sequence; Performing character recognition operation on the continuous recorded video frame sequence according to the set bit stream duration detection model to determine the actual playback duration of the corresponding video frame; The actual playback duration of the video frame is compared with the preset normal playback duration to determine the corresponding bitstream duration deviation.

7. The method for supplementing storage of abnormal video storage of a monitoring platform according to claim 1, characterized in that: The determining, according to the bitstream frame rate deviation and the bitstream duration deviation, corresponding abnormal video data and the monitoring abnormal video recording time corresponding to the abnormal video data comprises: Determine whether the bitstream frame rate deviation and the bitstream duration deviation are within a normal range; If not, the continuous video data within the abnormal range is subjected to an abnormal marking operation to determine the corresponding abnormal video data and the monitoring abnormal video time corresponding to the abnormal video data.

8. A supplementary storage device for abnormal video storage of a monitoring platform, characterized in that: The device comprises: A recording schedule construction module is used to read the recording information of the camera according to the preset recording detection rules, compare the camera recording time obtained after the recording information reading operation with the preset video monitoring platform recording time, and determine the corresponding monitoring recording schedule, wherein the monitoring recording schedule includes the monitoring vulnerability recording time and the monitoring continuous recording time; The continuous monitoring abnormality judgment module is used to read the video information of the video file corresponding to the continuous video recording time of the monitoring, obtain the corresponding continuous video data, perform a code stream frame rate calculation operation on the continuous video data, determine the corresponding actual code stream frame rate, compare the actual code stream frame rate with the original code stream frame rate of the camera, determine the corresponding code stream frame rate deviation, perform a code stream duration detection operation on the continuous video data according to a set code stream duration detection model, and determine the corresponding code stream duration deviation, wherein the set code stream duration detection model is obtained by calculating the comprehensive features through a progressive fusion algorithm based on target features, contour features and character features, and the comprehensive features are input into a preset God General network model for model training; The abnormal video supplement storage module is used to determine the corresponding abnormal video data and the monitoring abnormal video time corresponding to the abnormal video data according to the bit stream frame rate deviation and the bit stream duration deviation, and send a video download request to the camera according to the monitoring loophole video time and the monitoring abnormal video time, so that after the camera receives the video download request, it sends the video file corresponding to the monitoring loophole video time and the monitoring abnormal video time to the video monitoring platform for abnormal video supplement storage operation, and determines the corresponding complete monitoring video file.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method for supplementing storage of abnormal video storage of the monitoring platform described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for supplementing storage of abnormal video storage of a monitoring platform as described in any one of claims 1 to 7 are implemented.

Images