A large-scale video frame data extraction system and method

Abstract

The application discloses a large-scale video frame data extraction system and method, comprising an extractor, a queue, a collector and an uploader; the extractor is used for extracting video frames in a video stream, and delivering the obtained video frames to the queue after decoding and filtering; the collector is used for taking all the video frames from the queue at a fixed frequency, and performing batch packaging on the video frames to obtain a plurality of data packets; and the uploader is used for uploading the data packets of the collector to a storage system for storage; the extraction system and method greatly improve the utilization rate of system resources and network bandwidth, reduce redundant data, thereby greatly reducing the number of frame extraction servers to be deployed, and reducing the construction cost of the learning situation analysis system.

Description

Technical Field

[0001] This invention relates to the field of data extraction and storage technology, and in particular to a large-scale video frame data extraction system and method. Background Technology

[0002] The student learning analysis system aims to conduct large-scale, long-term analysis and data mining of video surveillance data from thousands of cameras on university campuses. This reveals potential patterns and helps university administrators gain a deeper understanding of the school's educational and teaching situation. For example, the student profiling function helps the school understand students' psychological state; the abnormal behavior analysis function can promptly detect various security risks and issue alerts; and the security analysis function can provide detailed real-time analysis data, including: personnel distribution, walking trajectories, and vehicle tracking.

[0003] However, extracting images in real time from thousands of cameras distributed across multiple campuses and storing them in a big data platform is a challenge. Video data, due to its large volume, is often difficult to analyze on a large scale. Analyzing extracted video frames is a viable solution compared to directly analyzing the video. Extracting video frames from a large volume of video is a computationally intensive and bandwidth-intensive task. The current common practice is to capture video frames from each live video stream, but only decode and store key frames, discarding non-key frames—this is known as frame skipping. This significantly reduces CPU and network bandwidth consumption, but when faced with a large-scale video frame extraction task involving thousands of video streams, simply using frame skipping is insufficient, especially when transmitting a large number of video frames across network segments, where network bandwidth becomes a bottleneck. Summary of the Invention

[0004] Based on the technical problems existing in the background technology, this invention proposes a large-scale video frame data extraction system and method, which greatly improves the utilization rate of system resources and network bandwidth, reduces redundant data, thereby significantly reducing the number of frame extraction servers that need to be deployed and reducing the construction cost of the learning analysis system.

[0005] This invention proposes a large-scale video frame data extraction system and method, including an extractor, a queue, a collector, and an uploader;

[0006] The extractor is used to extract video frames from a video stream and then decode and filter the obtained video frames before sending them to a queue.

[0007] The collector is used to retrieve all video frames from the queue at a fixed frequency and package the video frames into multiple data packets in batches.

[0008] The uploader is used to upload the collector's data packets to the storage system for storage.

[0009] Furthermore, the extraction system includes multiple extractors, a queue, a collector, and multiple uploaders;

[0010] Each extractor is used to extract video frames from one video stream, and then decodes and filters the obtained video frames before sending them to the queue;

[0011] Each uploader is used to upload a data packet from the collector to the storage system for storage.

[0012] Furthermore, the video frames stored in the queue are named <camera id>.jpg, and the length of the queue is set to the number of video streams for which video frames need to be extracted. During the enqueueing process, if the queue is full or a video frame file with the same name already exists in the queue, the video frame will be automatically discarded.

[0013] Furthermore, the extractor is specifically used for the following operations:

[0014] (a1) Capture video frames from one video stream and determine whether they are keyframes. If they are keyframes, proceed to (a2); otherwise, proceed to (a8).

[0015] (a2) Decode the key frame, and calculate the difference between the current time corresponding to the current decoded key frame and the enqueue time corresponding to the most recent decoded key frame to obtain the difference Δt. If Δt is greater than or equal to the set time threshold, proceed to (a3); if Δt is less than the set time threshold, proceed to (a5).

[0016] (a3) The extractor needs to extract video frames from one video stream, obtain the queue lock, and determine whether the queue is not full and has no duplicate names. If so, proceed to (a4); otherwise, release the queue lock and proceed to (a8).

[0017] (a4) Release the queue lock and enqueue the image, record the most recent enqueue time and store the most recent enqueue image;

[0018] (a5) Determine if the camera is indoors. If not, proceed to (a6); if so, proceed to (a7).

[0019] (a6) Calculate the histogram similarity s1, and determine whether s1 is less than the pixel threshold 1. If it is, proceed to (a3); otherwise, proceed to (a8).

[0020] (a7) Calculate the pixel similarity s2, and determine whether s2 is less than the pixel threshold 2. If it is, proceed to (a3); otherwise, proceed to (a8).

[0021] (a8) Discard the captured video frames.

[0022] Furthermore, in (a6) calculating the histogram similarity s1, the specific steps are as follows:

[0023] The length and width of the current image and the most recently enqueued image lf are reduced by one-tenth respectively, and linear interpolation is used to eliminate errors caused by slight camera shake; the length of the reduced image is denoted as L, and the width is denoted as W.

[0024] Convert both images to grayscale, where the value of each pixel ranges from [0, 255].

[0025] Count the number of pixels, 'a', where the absolute value of the difference between pixel values ​​at the same position in two images is greater than 10, pixel by pixel.

[0026] Calculate the pixel similarity s1 = a / (LW).

[0027] Furthermore, in (a7) calculating pixel similarity s2, the specific steps are as follows:

[0028] Both images are cut into a uniform 9-grid to obtain multiple slice images. Each slice image has the same size, with the length denoted as L and the width as W.

[0029] Convert the corresponding slices of the two images at the same position into grayscale images, where the value of each pixel ranges from [0, 255].

[0030] For each of the two grayscale images, count the number of pixels corresponding to the 256 pixel values ​​in the range [0, 255] to obtain two histograms;

[0031] Calculate the number of pixel values ​​b in the range [0,255] where the absolute value of the difference in the number of pixels corresponding to each pixel value is greater than L*W / 256;

[0032] Calculate the histogram similarity s of the sliced ​​images ij = b / 256, thus obtaining the pixel similarity s2 between the two images.

[0033] Furthermore, the collector uses a file byte stream concatenation method to merge multiple video frames into a data packet to be uploaded. The data packet file to be uploaded consists of a file header and a file body. The first four bytes of the file header are an Int value, which specifies the number of video frames in the data packet to be uploaded. Then there are n×3 Int values, which are grouped into groups of three, specifying the camera ID, the start byte position and the end byte position of the video frame data for each video frame file.

[0034] Furthermore, the uploader uploads the data packets by calling the web interface provided by the storage system.

[0035] A method for large-scale video frame data extraction includes:

[0036] The collector retrieves all video frames from the queue at a fixed frequency, packages the video frames into multiple data packets, and uploads these multiple data packets to the storage system for storage. The video frames in the queue are obtained by the extractor extracting video frames from one video stream, decoding and filtering the obtained video frames, and then sending them to the queue.

[0037] Furthermore, based on multiple extractors, a queue, a collector, and multiple uploaders, the extraction method specifically includes:

[0038] A collector retrieves all video frames from a queue at a fixed frequency, packages the video frames into multiple data packets, and uploads these data packets to a storage system via multiple uploaders for storage. The video frames in the queue are obtained by multiple extractors extracting video frames from multiple video streams, decoding and filtering the obtained video frames, and then sending them to the queue.

[0039] The advantages of the large-scale video frame data extraction system and method provided by this invention are as follows: The large-scale video frame data extraction system and method provided in this invention uses a pipelined processing structure to extract frames in parallel from a large number of video streams, and uses a refined frame extraction process to remove video frames with low value to the background analysis algorithm, which greatly improves the utilization rate of system resources and network bandwidth, reduces redundant data, and thus significantly reduces the number of frame extraction servers that need to be deployed. The number of push servers can be reduced to about one-fifth of the original number, reducing the construction cost of the learning analysis system. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the structure of the present invention;

[0041] Figure 2 Here is a flowchart of the extractor's workflow;

[0042] Figure 3 This is a schematic diagram of the data packet file structure. Detailed Implementation

[0043] The technical solution of the present invention will now be described in detail through specific embodiments. Many specific details are set forth in the following description to provide a thorough understanding of the invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0044] like Figures 1 to 3As shown, the present invention proposes a large-scale video frame data extraction system, including an extractor, a queue, a collector, and an uploader; the extractor is used to extract video frames from a video stream, and after decoding and filtering the obtained video frames, it sends them to the queue; the collector is used to take all video frames from the queue at a fixed frequency, and package the video frames in batches to obtain multiple data packets; the uploader is used to upload the data packets from the collector to the storage system for storage.

[0045] Compared with existing storage methods, this embodiment uses a pipelined processing structure to perform parallel frame extraction on a large number of video streams, and uses a refined frame extraction process to remove video frames with low value to the background analysis algorithm (only key frames are processed), which greatly improves the utilization of system resources and network bandwidth, reduces redundant data, and thus significantly reduces the number of frame extraction servers that need to be deployed. The number of push servers can be reduced to about one-fifth of the original number, reducing the construction cost of the learning analysis system.

[0046] In this embodiment, for large-scale video frames, the extractor, queue, collector, and uploader are each clearly defined components. Therefore, this embodiment can be considered a pipeline structure composed of multiple clearly defined components working together, achieving efficient utilization of computing resources and network bandwidth. Based on the frame skipping method, an adaptive filtering mechanism is used to remove a large number of low-value video frames, significantly reducing the number of video frames that need to be uploaded to the big data platform. This enables real-time extraction of large-scale video frame data, providing a high-quality data source for the learning analysis system. After using this system, under the same hardware configuration, the resource utilization of each server used for frame extraction is significantly improved. Compared to the ordinary frame skipping method, the number of video streams that can be extracted by each server increases by approximately five times.

[0047] The adaptive filtering mechanism is specifically as follows: Figure 2 As shown, the frame upload frequency can be dynamically adjusted based on whether there is movement in the video. The algorithm determines whether to skip frame similarity checks based on the time of the most recent frame upload from each camera, preventing a camera from being deemed offline due to prolonged periods without uploading video frames.

[0048] The following is a detailed explanation:

[0049] 1) System Structure

[0050] For large-scale video frame extraction and uploading tasks, employing a pipelined structure composed of clearly defined components can significantly improve system resource utilization. Based on this idea, the system pipelined structure adopted in this embodiment is as follows: Figure 1 As shown:

[0051] The entire large-scale video frame extraction system consists of multiple extractors, a queue, a collector, and multiple uploaders. Each extractor is responsible for extracting video frames from a single live video stream. The extractor filters out video frames that are of low value to the analysis algorithm, and valuable video frames are written to the queue, i.e., the obtained keyframes are written to the queue. The collector retrieves all video frames from the queue at a fixed frequency (e.g., once per second), then divides them into batches (e.g., 20 frames per batch), packages each batch of video frames into a data packet, and then starts an uploader that calls the web interface provided by the big data platform to upload this batch of video frames to the storage system.

[0052] The video frames stored in the queue are named <camera id>.jpg. The queue length is set to the number of video streams to be extracted. During the enqueueing process, if the queue is full or a video frame with the same name already exists, it is automatically discarded. This ensures that the collector uploads video frames at a fixed frequency.

[0053] Each packaged data packet corresponds to an uploader, which is responsible for uploading one data packet to the big data platform of the storage system. Each uploader is a thread object. Uploaders must complete the data upload operation within a specified time; if a timeout occurs, the upload will stop, and an exception will be logged. Regardless of success or failure, each uploader will always complete its execution within a finite time and release the memory resources it occupies.

[0054] 2) Extractor Workflow

[0055] Generally speaking, frame extraction mainly involves three steps: frame capture, decoding, and filtering. However, to minimize the consumption of computing resources, network bandwidth, and storage resources, a refined processing flow is needed to remove video frames that are of low value to the analysis algorithm, such as... Figure 2 As shown.

[0056] (a1) Capture video frames from one video stream and determine whether they are keyframes. If they are keyframes, proceed to (a2); otherwise, proceed to (a8).

[0057] Only keyframes are decoded, while non-keyframes are discarded. Two buffer variables are set in the extractor: the most recent enqueue time `lt` and the most recent enqueue image `lf`. Keyframe decoding uses ffmpeg's `avcodec_send_packet()` and `avcodec_receive_frame()` functions to decode the captured data frames, obtaining YUV420p video frames. The decoding process can employ existing frame-skipping methods for keyframe decoding.

[0058] By calling ffmpeg's av_read_frame() function, the obtained data frame is an AVPacket data structure containing a field indicating whether it is a keyframe. The process of obtaining keyframes can adopt the keyframe determination method in the existing frame skipping method.

[0059] (a2) Decode the key frame, and calculate the difference between the current time corresponding to the current decoded key frame and the enqueue time corresponding to the most recent decoded key frame to obtain the difference Δt. If Δt is greater than or equal to the set time threshold, proceed to (a3); if Δt is less than the set time threshold, proceed to (a5).

[0060] When Δt is greater than or equal to a set time threshold, it indicates that the camera has not uploaded video frames for a long time. To prevent the learning analysis algorithm from classifying the camera as offline, a frame needs to be uploaded immediately to attempt to enqueue it. Otherwise, when Δt is less than the set time threshold, the similarity between the current frame and lf needs to be compared. If the similarity is low, it is assumed that there is a person or object moving in the frame, and thus it is considered a valuable video frame for the analysis algorithm, and an attempt is made to upload that frame.

[0061] (a3) The extractor needs to extract video frames from one video stream, obtain the queue lock, and determine whether the queue is not full and has no duplicate names. If so, proceed to (a4); otherwise, release the queue lock and proceed to (a8).

[0062] (a4) Release the queue lock and enqueue the image, record the most recent enqueue time and store the most recent enqueue image;

[0063] (a5) Determine if the camera is indoors. If not, proceed to (a6); if so, proceed to (a7).

[0064] (a6) Calculate the histogram similarity s1, and determine whether s1 is less than the pixel threshold 1. If it is, proceed to (a3); otherwise, proceed to (a8).

[0065] (a7) Calculate the pixel similarity s2, and determine whether s2 is less than the pixel threshold 2. If it is, proceed to (a3); otherwise, proceed to (a8).

[0066] (a8) Discard the captured video frames.

[0067] Pixel threshold 1 and pixel threshold 2 are determined based on the actual scene to achieve the optimal filtering effect. In this embodiment, pixel threshold 1 is 0.995 and pixel threshold 2 is 0.975.

[0068] Since multiple extractors write data to the queue or check if a certain element exists in the queue, accessing the queue requires a locking and unlocking process to prevent mutual interference. Furthermore, the enqueueing process is not always successful; if the queue is full or if a video frame from that camera already exists in the queue but has not yet been sent, the enqueue request is abandoned. After each successful enqueueing, the extractor considers the extracted data frames to have been uploaded, updates the most recent enqueue time (lt), and caches the most recently enqueued image (lf).

[0069] Different methods are used to calculate the similarity between the current video frame and the most recently enqueued image lf, depending on the camera's installation location. For indoor cameras, due to less interference, a more accurate pixel comparison method can be used to calculate the pixel similarity s1, with a value ranging from [0-1]. The larger the value, the higher the similarity. The calculation process for comparing the pixel similarity of two images is as follows:

[0070] ● Reduce the length and width of the current image and the most recently enqueued image (lf) by one-tenth each, using linear interpolation to eliminate errors caused by slight camera shake; the length of the reduced image...

[0071] Degree is denoted as L, and width as W;

[0072] ● Convert both images to grayscale, where the value of each pixel ranges from [0, 255].

[0073] ● Count the number of pixels, 'a', where the absolute value of the difference between pixel values ​​at the same position in two images is greater than 10, pixel by pixel;

[0074] ● Calculate pixel similarity s1 = a / (LW);

[0075] For outdoor cameras, the movement of objects in the frame is more noticeable due to the influence of trees and wind. In this case, histogram similarity is more effective in determining whether people or objects are moving. Its value ranges from [0-1], with higher values ​​indicating higher similarity. The calculation process for the pixel similarity s² between two images is as follows:

[0076] Both images (the current video frame and the most recently enqueued image lf) are sliced ​​into uniform 9-grids. For each pair of slices, histogram similarity is calculated. Finally, these 9 similarity scores are multiplied together to obtain the histogram similarity s2. The histogram similarity calculation process for each pair of slices is as follows:

[0077] ◆Slice both images into a uniform 9-grid to obtain multiple slice images. Each slice from the two images...

[0078] The images are all the same size, with length denoted as L and width as W;

[0079] ◆ Convert corresponding slices of two images to grayscale, and take the value of each pixel.

[0080] The value range is [0, 255];

[0081] ◆ Count the number of pixels corresponding to the 256 pixel values ​​in the range [0, 255] for each of the two grayscale images.

[0082] Count the numbers to obtain two histograms;

[0083] ◆Calculate the number of pixels corresponding to each pixel value within the range [0, 255] for two histograms.

[0084] The number of pixel values ​​b whose absolute value of the difference is greater than L*W / 256;

[0085] ◆ Calculate the histogram similarity s of the sliced ​​images ij = b / 256, thus obtaining the pixel similarity s2 between the two images.

[0086] The image similarity calculation methods used in this embodiment are relatively simple. This is because we expect each server to extract frames from as many video streams as possible, so we must use similarity comparison methods with less computational cost to save computing resources.

[0087] 3) Collector Packing

[0088] To conserve network bandwidth and reduce the number of network requests, video frames are packaged in batches in the collector, and then an uploader is started for each data packet to upload it to the big data platform for storage.

[0089] Larger batches will lead to greater data transmission latency, while smaller batches will increase the number of network accesses; fine-tuning is necessary based on actual needs. In this embodiment, a batch size of 20 was used after actual testing. Each video frame image has a data size of approximately 100KB, resulting in a data packet of approximately 2MB. This allows for rapid uploading to a big data platform with minimal transmission latency. This batch size setting was determined through extensive experimentation, not arbitrarily decided upon.

[0090] To minimize the computational overhead of the packaging process, this embodiment uses a simple file byte stream concatenation method to merge multiple video frames into a single upload data packet, without employing compression technology, as compression consumes computational resources. The file format of the upload data packet is as follows: Figure 3As shown, the data packet to be uploaded consists of a header and a body. The first four bytes of the header are an Int value specifying the number of video frames in the data packet. This is followed by n×3 Int values, grouped in sets of three, specifying the camera ID corresponding to each video frame file, the start byte position of that group of video frame data, and the end byte position. This frame data packaging method minimizes the overhead on computing resources.

[0091] It should be noted that not all keyframes are stored in the database; low-value keyframes are also discarded. The frame skipping method only retains keyframes; this embodiment, based on the frame skipping method, determines whether there is a dynamic scene in the video. If the current scene is static, most keyframes will be discarded, maintaining only a minimum push frequency; if the current scene is dynamic, such as someone walking or a car moving, then all keyframes will be pushed to the server as much as possible during this period, as shown in (a1) to (a8).

[0092] As an example:

[0093] In a well-known and efficient campus student monitoring system in Hefei, there were originally 1,103 campus cameras that needed to be monitored. However, with the advancement of campus construction, the actual number of cameras that need to be monitored has now reached 2,433.

[0094] The large-scale video frame data extraction system applied to the learning situation analysis system requires that for each camera, one video frame be collected and saved from the Rtsp video stream every second, and real-time and long-term analysis be performed on them to complete functions such as personnel distribution calculation, pedestrian trajectory analysis, abnormal behavior monitoring and early warning.

[0095] The original method used frame skipping, which involved extracting and uploading frames individually for each camera. This was inefficient, as each server with 16 CPU cores and a gigabit network card could only handle frame extraction from about 100 cameras. The system construction cost was also too high.

[0096] In response to this situation, considering the excessive redundancy in video data, with most uploaded video frames being identical, it not only consumes a significant amount of network bandwidth and storage space but also greatly impacts the efficiency of backend algorithm analysis and mining. The traditional, commonly used frame skipping method still has considerable room for improvement.

[0097] During project implementation, a large-scale video frame extraction system was developed using the large-scale video frame extraction method described in this embodiment. This alleviated the pressure on video frame extraction, transmission, storage, and analysis caused by the excessive amount of data. The number of servers used for video frame extraction was reduced from the originally planned 25 to 5, saving system construction costs, and the amount of stored data was reduced to about 1 / 6 of the original. The calculation speed of learning analysis algorithms such as people distribution calculation and pedestrian trajectory analysis was also significantly improved.

[0098] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A large scale video frame data extraction system, characterized by, The extraction system comprises an extractor, a queue, a collector and an uploader; The extractor is configured to extract video frames from a video stream and deliver the decoded filtered video frames to the queue; The collector is configured to take all the video frames from the queue at a fixed frequency, and pack the video frames in batches to obtain a plurality of data packets; The uploader is configured to upload the data packets from the collector to a storage system for storage; The extractor is specifically configured to: (a1) capture video frames from a video stream, and determine whether the video frames are key frames, if yes, go to (a2), if not, go to (a8); (a2) decode the key frames, and calculate a difference value Δt between a current time corresponding to the decoded key frame and an enqueue time corresponding to a latest decoded key frame, if Δt is greater than or equal to a set time threshold, go to (a3), if Δt is less than the set time threshold, go to (a5); (a3) the extractor extracts video frames from a video stream, obtains a queue lock, and determines whether the queue is not full and has no duplicate name, if yes, go to (a4), if not, releases the queue lock and goes to (a8); (a4) releases the queue lock and enqueues the queue, records a latest enqueue time and stores a latest enqueued image; (a5) determines whether the camera is located indoors, if not, goes to (a6), if yes, goes to (a7); (a6) calculates a histogram similarity s1, and determines whether s1 is less than a pixel threshold 1, if yes, goes to (a3), if not, goes to (a8); (a7) calculates a pixel similarity s2, and determines whether s2 is less than a pixel threshold 2, if yes, goes to (a3), if not, goes to (a8); (a8) discards the captured video frames.

2. The large scale video frame data decimation system of claim 1, wherein, The extraction system comprises a plurality of extractors, a queue, a collector and a plurality of uploaders; Each extractor is configured to extract video frames from a video stream and deliver the decoded filtered video frames to the queue; Each uploader is configured to upload a data packet from the collector to a storage system for storage.

3. The large scale video frame data decimation system of claim 1, wherein, The video frames stored in the queue are named in the manner of <camera id>.jpg, and the length of the queue is set to the number of video streams of the extracted video frames. During the enqueueing of the video frames, if the queue is full or there is already a video frame file with the same name in the queue, the video frame is discarded automatically.

4. The large scale video frame data decimation system of claim 1, wherein, In (a6), the histogram similarity s1 is calculated as follows: The length and width of the current image and the latest enqueued image lf are reduced by one-tenth respectively, and a linear interpolation method is used to eliminate errors caused by slight camera shaking; the reduced image length is denoted as L, and the width is denoted as W; The two images are converted into grayscale images, and the value range of each pixel value is [0, 255]; The absolute value difference of the pixel values of the same position of the two images is counted pixel by pixel, and the number of pixels a whose absolute value difference is greater than 10 is counted; The pixel similarity s1 is calculated as a / (LW).

5. The large scale video frame data decimation system of claim 1, wherein, In (a7), the pixel similarity s2 is calculated as follows: The two images are cut into uniform 9x9 grids to obtain a plurality of slice images, and the size of each slice image of the two images is the same, with a length denoted as L and a width denoted as W; Convert the slice images of the same position of the two images into gray images, and the value range of each pixel value is [0, 255]; Count the number of pixel points corresponding to 256 pixel values in the range of [0, 255] for the two gray images respectively to obtain two histograms; The two histograms are calculated in the range of [0, 255], and the absolute value of the difference between the number of pixel points corresponding to each pixel value is greater than L The pixel value number b of W / 256 Computing a histogram similarity s of the slice images ij = b / 256, and thus the pixel similarity s2 of the two images.

6. The large scale video frame data decimation system of claim 1, wherein, The collector adopts a file byte stream splicing manner to combine multiple video frames into one data packet to be uploaded; the data packet file to be uploaded is composed of a file header and a file body, the first four bytes of the file header is an Int value, which specifies the number of video frames in the data packet to be uploaded, and then there are n*3 Int values, each group of three specifies the camera id corresponding to each video frame file, the starting byte position and the ending byte position of the video frame data of the group.

7. The large scale video frame data decimation system of claim 1, wherein, The uploader uploads the data packet by calling the Web interface provided by the storage system.

8. A method for large scale video frame data extraction, the method comprising: The method comprises the following steps: The collector takes all the video frames from the queue at a fixed frequency, packs the video frames in batches to obtain multiple data packets, and uploads the multiple data packets to the storage system through the uploader for storage, wherein the video frames in the queue are video frames in a video stream extracted by the extractor, and the obtained video frames are decoded and filtered to be delivered to the queue; The extractor is specifically used for the following operations: (a1) capturing video frames in a video stream, determining whether the video frames are key frames, if yes, entering (a2), if not, entering (a8); (a2) decoding the key frames, calculating the difference Δt between the current time corresponding to the current decoded key frame and the queue-in time corresponding to the last decoded key frame, if Δt is greater than or equal to a set time threshold, entering (a3), if Δt is less than the set time threshold, entering (a5); (a3) the extractor needs to extract video frames in a video stream, obtains a queue lock, and determines whether the queue is not full and has no duplicate names, if yes, entering (a4), if not, releasing the queue lock and entering (a8); (a4) releasing the queue lock and entering the queue, recording the latest queue-in time and storing the latest queue-in image; (a5) determining whether the camera is located indoors, if not, entering (a6), if yes, entering (a7); (a6) calculating the histogram similarity s1, determining whether s1 is less than a pixel threshold 1, if yes, entering (a3), if not, entering (a8); (a7) calculating the pixel similarity s2, determining whether s2 is less than a pixel threshold 2, if yes, entering (a3), if not, entering (a8); (a8) discarding the captured video frames.

9. The method of claim 8, wherein, Based on multiple extractors, one queue, one collector and multiple uploaders, the extraction method specifically comprises: One collector takes all the video frames from one queue at a fixed frequency, packs the video frames in batches to obtain multiple data packets, and uploads the multiple data packets to the storage system through multiple uploaders for storage, wherein the video frames in the queue are video frames in multiple video streams extracted by multiple extractors, and the obtained video frames are decoded and filtered to be delivered to the queue.

Images