Frame extraction method and system for streaming media monitoring video

Abstract

The invention relates to a frame extraction method and system for a streaming media monitoring video, and the method comprises the steps: a streaming media server receives a frame extraction request sent by a third-party platform, obtains and analyzes a monitoring video stream according to the frame extraction request, carries out the frame extraction of the monitoring video stream according to the frame extraction parameter of the frame extraction request, and carries out the frame extraction of the monitoring video stream; and storing the obtained target binary video frame data to a data cache queue so as to consume and obtain the target binary video frame data from the data cache queue, and performing compression decoding on the target binary video frame data according to an output format of a third-party platform, and returning the compressed and decoded target binary video frame data to the third-party platform. Therefore, the frame extraction logic is embedded in the streaming media server, one-time analysis and multi-time multiplexing are achieved, resource loss is reduced, the resource utilization rate is increased, the response time is shortened by using the data cache queue, and the stability and expandability of the system are maintained.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and system for extracting frames from streaming media surveillance videos. Background Technology

[0002] In current security monitoring systems, streaming media servers are responsible for receiving, processing, and distributing video streams for real-time viewing, storage, or analysis by the monitoring platform. In certain scenarios, such as intelligent analysis or image archiving, the monitoring platform needs to extract image frames from the video stream at a certain frequency. Existing frame extraction technologies typically rely on a separate frame extraction service, which obtains video frames by parsing real-time transmission protocols and program streams. While existing video frame extraction schemes can meet the frame extraction needs of monitoring platforms to some extent, they have the following shortcomings:

[0003] (i) Both streaming media servers and frame extraction services need to parse the video stream separately, resulting in the same video stream being parsed multiple times, which not only increases the computational overhead but also reduces the overall processing efficiency of the system.

[0004] (ii) In high-concurrency scenarios, existing frame extraction solutions are prone to becoming a performance bottleneck of the system because they need to continuously parse and process a large number of video streams. In addition, the frame extraction service not only needs to process a large number of video streams, but also needs to perform frequent transcoding operations, which will lead to a longer response time under high load, affecting the scalability and stability of the system. Summary of the Invention

[0005] The technical problem to be solved by this invention is: This invention provides a method and system for extracting frames from streaming media monitoring videos, which enables the video stream parsed by the streaming media server to be directly reused by the frame extraction service, avoiding the same video stream being parsed multiple times, reducing resource consumption, improving resource utilization, and enhancing the performance and concurrent processing capabilities of the frame extraction service, thereby reducing response time and maintaining the stability and scalability of the system.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0007] In a first aspect, the present invention provides a frame extraction method for streaming media monitoring video, comprising:

[0008] The streaming media server receives a frame extraction request sent by a third-party platform, acquires and parses the monitoring video stream according to the frame extraction request, and extracts frames from the monitoring video stream according to the frame extraction parameters of the frame extraction request to obtain target binary video frame data. The target binary video frame data is then stored in a data cache queue.

[0009] The target binary video frame data is consumed from the data cache queue, compressed and decoded according to the output format of the third-party platform to obtain the compressed and decoded target binary video frame data, and then returned to the third-party platform.

[0010] The beneficial effects of this invention are as follows: By embedding frame extraction logic within the streaming media server, the streaming media server can directly extract frames from the monitoring video stream according to the frame extraction parameters of the frame extraction request. This avoids the traditional practice of repeatedly parsing the same video stream by the streaming media server and the frame extraction service, reducing resource consumption and improving resource utilization. Furthermore, the obtained target binary video frame data is stored in a data cache queue, which can be directly consumed from the data cache queue during subsequent compression and decoding, eliminating the need to continuously pull the entire monitoring video stream. This significantly reduces bandwidth consumption, especially in high-concurrency scenarios, which can significantly reduce network load, improve the performance and concurrent processing capabilities of the frame extraction service, reduce response time, and maintain system stability and scalability.

[0011] Optionally, the frame extraction parameters include frame extraction interval, frame extraction count, and frame type. Simultaneously extracting frames from the monitoring video stream according to the frame extraction parameters of the frame extraction request to obtain target binary video frame data includes:

[0012] Simultaneously, the monitoring video stream is frame-sampling based on the frame-sampling interval, the number of frame-sampling times, and the frame type, and the dynamic feature values ​​of the monitoring video stream are extracted in real time.

[0013] The frame-sampling interval and the number of frame-samplings are dynamically adjusted based on the dynamic feature values ​​to obtain the dynamically adjusted frame-sampling interval and the dynamically adjusted number of frame-samplings.

[0014] Based on the dynamically adjusted frame extraction interval and the dynamically adjusted number of frame extractions, target binary video frame data that conforms to the frame type are filtered from the monitoring video stream.

[0015] Optionally, the dynamic feature value includes the proportion of the motion region, and the step of dynamically adjusting the frame skipping interval and the number of frame skipping based on the dynamic feature value to obtain the dynamically adjusted frame skipping interval and the dynamically adjusted number of frame skipping includes:

[0016] Determine whether the proportion of the motion area exceeds a first threshold. If yes, mark the monitoring video stream as a high dynamic scene; otherwise, mark the monitoring video stream as a static scene.

[0017] The system acquires historical frame-sampling effect data from the third-party platform, generates adjustment parameters based on the historical frame-sampling effect data and the reinforcement learning model, and automatically shortens the frame-sampling interval and the number of frame-samplings in the high-dynamic scene according to the adjustment parameters to obtain the automatically shortened frame-sampling interval and the automatically shortened number of frame-samplings. The system also automatically extends the frame-sampling interval and the number of frame-samplings in the static scene according to the adjustment parameters to obtain the automatically extended frame-sampling interval and the automatically extended number of frame-samplings.

[0018] As described above, when extracting frames from a monitoring video stream according to the extraction parameters requested by the extraction request, the monitoring video stream is marked into different extraction scenarios based on the dynamic feature values ​​extracted in real time. When dynamically adjusting the extraction parameters, not only are the different marked scenarios considered, but historical extraction effect data is also taken into account. Based on the historical extraction effect data and the adjustment parameters generated by the reinforcement learning model, the extraction interval and number of extractions are automatically shortened for monitoring video streams with high dynamic scenarios, and the extraction interval and number of extractions are automatically extended for monitoring video streams with static scenarios. This achieves precise extraction by separating dynamic and static scenarios, reduces the proportion of invalid frames, and improves processing efficiency.

[0019] Optionally, the target binary video frame data includes a timestamp and a device ID, and storing the target binary video frame data in a data cache queue includes:

[0020] A time-series index caching mechanism is adopted to store the target binary video frame data into a data cache queue according to the timestamp and the device ID. Target binary video frame data that conforms to the first time period is selected as hot data based on the timestamp, while target binary video frame data that does not conform to the first time period is selected as cold data. The hot data is stored in a memory queue, and the cold data is stored on the disk.

[0021] As described above, the target binary video frame data is stored in the data cache queue according to the timestamp and device ID. That is, it is stored through a two-dimensional index, supports millimeter-level frame positioning, and separates the cold and hot data of the target binary video data according to the timestamp, taking into account both response speed and storage efficiency, and realizing fast frame compression in high-concurrency scenarios.

[0022] Optionally, the step of compressing and decoding the target binary video frame data according to the output format of the third-party platform includes:

[0023] A transcoding task is created according to the output format of the third-party platform, and the transcoding task is passed to the transcoding thread pool to call the GPU acceleration unit to compress and decode the target binary video frame data.

[0024] As described above, the GPU acceleration unit is used to compress and decode the target binary video frame data, improving the compression and decryption speed. It also supports dynamic adaptation of the output format, automatically switching the output format according to the needs of third-party platforms, adapting to multiple scenarios, and optimizing the user experience.

[0025] In a second aspect, the present invention provides a frame extraction system for streaming media monitoring video, comprising:

[0026] A streaming media server is used to receive frame extraction requests sent by a third-party platform, acquire and parse the monitoring video stream according to the frame extraction request, extract frames from the monitoring video stream according to the frame extraction parameters of the frame extraction request to obtain target binary video frame data, and store the target binary video frame data in a data cache queue.

[0027] A transcoding server is used to consume the target binary video frame data from the data cache queue, compress and decode the target binary video frame data according to the output format of the third-party platform to obtain compressed and decoded target binary video frame data, and return the compressed and decoded target binary video frame data to the third-party platform.

[0028] The beneficial effects of this invention are as follows: By embedding frame extraction logic within the streaming media server, the streaming media server can directly extract frames from the monitoring video stream according to the frame extraction parameters of the frame extraction request. This avoids the traditional practice of repeatedly parsing the same video stream by the streaming media server and the frame extraction service, reducing resource consumption and improving resource utilization. Furthermore, the obtained target binary video frame data is stored in a data cache queue, which can be directly consumed from the data cache queue during subsequent compression and decoding, eliminating the need to continuously pull the entire monitoring video stream. This significantly reduces bandwidth consumption, especially in high-concurrency scenarios, which can significantly reduce network load, improve the performance and concurrent processing capabilities of the frame extraction service, reduce response time, and maintain system stability and scalability.

[0029] Optionally, the frame extraction parameters include frame extraction interval, number of frame extractions, and frame type, and the streaming media server includes:

[0030] An intelligent frame extraction server is used to simultaneously extract frames from the monitoring video stream based on the frame extraction interval, the number of frame extractions, and the frame type, and extract dynamic feature values ​​of the monitoring video stream in real time; dynamically adjust the frame extraction interval and the number of frame extractions based on the dynamic feature values ​​to obtain dynamically adjusted frame extraction interval and dynamically adjusted number of frame extractions; and filter target binary video frame data that conforms to the frame type from the monitoring video stream based on the dynamically adjusted frame extraction interval and dynamically adjusted number of frame extractions.

[0031] The dynamic feature value includes the proportion of the motion region, and the intelligent frame extraction server is specifically:

[0032] Determine whether the proportion of the motion area exceeds a first threshold. If yes, mark the monitoring video stream as a high dynamic scene; otherwise, mark the monitoring video stream as a static scene.

[0033] The system acquires historical frame-sampling effect data from the third-party platform, generates adjustment parameters based on the historical frame-sampling effect data and the reinforcement learning model, and automatically shortens the frame-sampling interval and the number of frame-samplings in the high-dynamic scene according to the adjustment parameters to obtain the automatically shortened frame-sampling interval and the automatically shortened number of frame-samplings. The system also automatically extends the frame-sampling interval and the number of frame-samplings in the static scene according to the adjustment parameters to obtain the automatically extended frame-sampling interval and the automatically extended number of frame-samplings.

[0034] As described above, when extracting frames from a monitoring video stream according to the extraction parameters requested by the extraction request, the monitoring video stream is marked into different extraction scenarios based on the dynamic feature values ​​extracted in real time. When dynamically adjusting the extraction parameters, not only are the different marked scenarios considered, but historical extraction effect data is also taken into account. Based on the historical extraction effect data and the adjustment parameters generated by the reinforcement learning model, the extraction interval and number of extractions are automatically shortened for monitoring video streams with high dynamic scenarios, and the extraction interval and number of extractions are automatically extended for monitoring video streams with static scenarios. This achieves precise extraction by separating dynamic and static scenarios, reduces the proportion of invalid frames, and improves processing efficiency.

[0035] Optionally, the streaming media server includes:

[0036] The intelligent caching server is used to store the target binary video frame data into a data cache queue according to the timestamp and the device ID using a time-series index caching mechanism. It also filters out target binary video frame data that conforms to a first time period as hot data based on the timestamp, and selects target binary video frame data that does not conform to the first time period as cold data. The hot data is stored in a memory queue, and the cold data is stored in a disk.

[0037] As described above, the target binary video frame data is stored in the data cache queue according to the timestamp and device ID. That is, it is stored through a two-dimensional index, supports millimeter-level frame positioning, and separates the cold and hot data of the target binary video data according to the timestamp, taking into account both response speed and storage efficiency, and realizing fast frame compression in high-concurrency scenarios.

[0038] Optionally, the transcoding server specifically comprises:

[0039] A transcoding task is created based on the output format of the third-party platform. The transcoding task is then passed to the transcoding thread pool to call the GPU acceleration unit to compress and decode the target binary video frame data into the target image.

[0040] A transcoding task is created according to the output format of the third-party platform, and the transcoding task is passed to the transcoding thread pool to call the GPU acceleration unit to compress and decode the target binary video frame data. Attached Figure Description

[0041] Figure 1 A flowchart illustrating a method for extracting frames from streaming media surveillance video provided in this embodiment;

[0042] Figure 2 This is a schematic diagram of the overall process of a frame extraction method for streaming media monitoring video provided in this embodiment;

[0043] Figure 3 This is a schematic diagram of the structure of a frame extraction system for streaming media monitoring video provided in this embodiment.

[0044] [Explanation of Labels in the Attached Image]

[0045] 1. A frame extraction system for streaming media surveillance video;

[0046] 2. Streaming media server; 21. Intelligent frame extraction server; 22. Intelligent caching server;

[0047] 3. Transcoding server. Detailed Implementation

[0048] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present invention can be understood more clearly and thoroughly, and that the scope of the present invention can be fully conveyed to those skilled in the art.

[0049] Example 1

[0050] Please refer to Figures 1 to 2 This invention provides a frame extraction method for streaming media monitoring video, comprising the following steps:

[0051] S1. The streaming media server receives a frame extraction request sent by a third-party platform, obtains and parses the monitoring video stream according to the frame extraction request, and extracts frames from the monitoring video stream according to the frame extraction parameters of the frame extraction request to obtain target binary video frame data, and stores the target binary video frame data in the data cache queue.

[0052] In this embodiment, as Figure 2As shown, the streaming media server receives frame extraction requests from a third-party platform. These requests include the extraction interval, number of extractions, and frame type. The extraction request can be configured by the third-party platform. Upon receiving the request, the streaming media server creates a frame push task to request streaming from the front-end IPC device, thereby acquiring the monitoring video stream. The server then parses the acquired video stream and extracts frames according to the extraction parameters in the request. For example, if the extraction interval is 10 seconds, the number of extractions is 1, and the frame type is keyframe, then the monitoring video stream is extracted according to the standard of extracting one keyframe every 10 seconds. The specific extraction process is as follows:

[0053] a. Determine whether the current frame in the monitoring video stream is a keyframe. If not, continue with step a; if yes, proceed with step b.

[0054] b. Determine if the frame extraction interval has been reached. If not, wait until the frame extraction interval is reached. If yes, execute c.

[0055] c. Determine if the frame extraction count has been reached. If not, return to execute a. If yes, the frame extraction is complete, and the target binary video frame data is obtained.

[0056] The obtained target binary video frame data will be assembled according to a pre-negotiated format, and the assembled target binary video frame data will be stored in the data cache queue.

[0057] At this time, the frame extraction parameters in step S1 include the frame extraction interval, the number of frame extractions, and the frame type. Simultaneously extracting frames from the monitoring video stream according to the frame extraction parameters of the frame extraction request to obtain the target binary video frame data includes:

[0058] S11. Simultaneously, the monitoring video stream is frame-sampling based on the frame-sampling interval, the number of frame-sampling times, and the frame type, and the dynamic feature values ​​of the monitoring video stream are extracted in real time.

[0059] S12. The frame-skipping interval and the number of frame-skippings are dynamically adjusted according to the dynamic feature value to obtain the dynamically adjusted frame-skipping interval and the dynamically adjusted number of frame-skippings.

[0060] At this time, the dynamic feature value includes the proportion of the motion region, and step S11 includes:

[0061] S111. Determine whether the proportion of the motion area exceeds a first threshold. If yes, mark the monitoring video stream as a high dynamic scene; otherwise, mark the monitoring video stream as a static scene.

[0062] S112. Obtain historical frame-sampling effect data from the third-party platform, generate adjustment parameters based on the historical frame-sampling effect data and the reinforcement learning model, automatically shorten the frame-sampling interval and the number of frame-samplings in the high-dynamic scene according to the adjustment parameters, and obtain the automatically shortened frame-sampling interval and the automatically shortened number of frame-samplings. Then, automatically extend the frame-sampling interval and the number of frame-samplings in the static scene according to the adjustment parameters, and obtain the automatically extended frame-sampling interval and the automatically extended number of frame-samplings.

[0063] S13. Select target binary video frame data that conforms to the frame type from the monitoring video stream according to the dynamically adjusted frame extraction interval and the dynamically adjusted number of frame extractions.

[0064] In this embodiment, as Figure 2 As shown, when extracting frames from the monitoring video stream according to the extraction parameters of the extraction request, dynamic feature values ​​of the monitoring video stream are extracted in real time. The dynamic feature values ​​include: the proportion of the moving area and the target contour change rate. The extraction interval and the number of extractions are dynamically adjusted according to the dynamic feature values. When the proportion of the moving area exceeds the first threshold, which is 30%, the monitoring video stream is marked as a high dynamic scene. Conversely, the monitoring video stream is marked as a static scene. That is, the monitoring video stream is divided into extraction scenes. At the same time, a second threshold is set. When the proportion of the moving area exceeds the second threshold, the monitoring video stream is marked as a sudden abnormal scene. For sudden abnormal scenes, instant extraction will be triggered. By default, the extraction strategy is to extract 3 key frames continuously within 1 second.

[0065] Historical frame extraction effect data from a third-party platform is obtained. Adjustment parameters are generated based on the historical frame extraction effect data and the reinforcement learning model. The frame extraction interval and number of frames are automatically shortened according to the adjustment parameters to obtain the automatically shortened frame extraction interval and the automatically shortened number of frames. The frame extraction interval and number of frames are automatically extended according to the adjustment parameters to obtain the automatically extended frame extraction interval and the automatically extended number of frames. Based on the dynamically adjusted frame extraction interval and the dynamically adjusted number of frames, target binary video frame data that meets the frame type are filtered from the monitoring video stream. The first threshold and the second threshold can be adjusted according to the actual situation, and the first threshold and the second threshold are different.

[0066] At this point, the target binary video frame data in step S1 includes a timestamp and a device ID, and storing the target binary video frame data in a data cache queue includes:

[0067] S13. Using a time-series index caching mechanism, the target binary video frame data is stored in a data cache queue according to the timestamp and the device ID. Target binary video frame data that conforms to the first time period is selected as hot data based on the timestamp, while target binary video frame data that does not conform to the first time period is selected as cold data. The hot data is stored in a memory queue, and the cold data is stored in a disk.

[0068] In this embodiment, as Figure 2 As shown, a time-series index caching mechanism is adopted to store target binary video frame data in a data cache queue according to timestamp and device ID. That is, a two-dimensional index storage is used, and target binary video frame data that meets the first time period is selected as hot data based on the timestamp, while target binary video frame data that does not meet the first time period is selected as cold data. That is, when the first time period is [0, 60 minutes], target binary video frame data in [0, 60 minutes] is selected as hot data, and target binary video frame data that exceeds 60 minutes is selected as cold data. Hot data is stored in a memory queue, and cold data is stored on disk, realizing the separate storage of hot and cold data. The first time period can be adjusted according to the actual situation.

[0069] S2. Consume the target binary video frame data from the data cache queue, compress and decode the target binary video frame data according to the output format of the third-party platform to obtain the compressed and decoded target binary video frame data, and return the compressed and decoded target binary video frame data to the third-party platform.

[0070] At this point, the compression and decoding of the target binary video frame data according to the output format of the third-party platform in step S2 includes:

[0071] S21. Create a transcoding task according to the output format of the third-party platform, and pass the transcoding task into the transcoding thread pool to call the GPU acceleration unit to compress and decode the target binary video frame data.

[0072] In this embodiment, as Figure 2As shown, the target binary video frame data is consumed from the data cache queue. A transcoding task is created according to the output format of the third-party platform. For example, if the output format is YUV and the target binary video frame data is JPEG, the transcoding task is created as follows: transcode from JPEG to YUV. The transcoding task is passed to the transcoding thread pool to call the GPU acceleration unit to compress and decode the target binary video frame data, and the compressed and decoded target binary video frame data is obtained. The compressed and decoded target binary video frame data is returned to the third-party platform. At the same time, the frame extraction effect data of the third-party platform, such as call rate and missed detection flags, is collected to optimize and update the adjustment parameters.

[0073] Example 2

[0074] Please refer to Figure 3 The present invention provides a frame extraction system 1 for streaming media monitoring video, comprising: a streaming media server 2, an intelligent frame extraction server 21, an intelligent caching server 22, and a transcoding server 3.

[0075] Among them, streaming media server 2 is used to receive frame extraction requests sent by third-party platforms, acquire and parse monitoring video streams according to the frame extraction requests, extract frames from the monitoring video streams according to the frame extraction parameters of the frame extraction requests, obtain target binary video frame data, and store the target binary video frame data in a data cache queue.

[0076] Transcoding server 3 is used to consume the target binary video frame data from the data cache queue, compress and decode the target binary video frame data according to the output format of the third-party platform to obtain compressed and decoded target binary video frame data, and return the compressed and decoded target binary video frame data to the third-party platform.

[0077] Specifically, the frame extraction parameters include frame extraction interval, frame extraction count, and frame type, and the streaming media server 2 includes:

[0078] The intelligent frame extraction server 21 is used to simultaneously extract frames from the monitoring video stream according to the frame extraction parameters of the frame extraction request, and extract the dynamic feature values ​​of the monitoring video stream in real time; dynamically adjust the frame extraction interval and the number of frame extractions according to the dynamic feature values ​​to obtain the dynamically adjusted frame extraction interval and the dynamically adjusted number of frame extractions; and filter out target binary video frame data that conforms to the frame type from the monitoring video stream according to the dynamically adjusted frame extraction interval and the dynamically adjusted number of frame extractions.

[0079] Specifically, the intelligent frame extraction server 21 is as follows:

[0080] Determine whether the proportion of the motion area exceeds a first threshold. If yes, mark the monitoring video stream as a high dynamic scene; otherwise, mark the monitoring video stream as a static scene.

[0081] The historical frame-sampling effect data of the third-party platform is obtained. Adjustment parameters are generated based on the historical frame-sampling effect data and the reinforcement learning model. The frame-sampling interval and the number of frame-samplings in the high-dynamic scene are automatically shortened according to the adjustment parameters to obtain the automatically shortened frame-sampling interval and the number of frame-samplings. The frame-sampling interval and the number of frame-samplings in the static scene are automatically extended according to the adjustment parameters to obtain the automatically extended frame-sampling interval and the number of frame-samplings.

[0082] Specifically, the streaming media server 2 includes:

[0083] The intelligent caching server 22 is used to store the target binary video frame data into a data cache queue according to the timestamp and the device ID using a time-series index caching mechanism. It also filters out target binary video frame data that conforms to a first time period as hot data based on the timestamp, and selects target binary video frame data that does not conform to the first time period as cold data. The hot data is stored in a memory queue, and the cold data is stored in a disk.

[0084] Specifically, the transcoding server 2 is:

[0085] A transcoding task is created according to the output format of the third-party platform, and the transcoding task is passed to the transcoding thread pool to call the GPU acceleration unit to compress and decode the target binary video frame data.

[0086] Since the systems / devices described in the above embodiments of the present invention are systems / devices used to implement the methods of the above embodiments of the present invention, those skilled in the art can understand the specific structure and modifications of the systems / devices based on the methods described in the above embodiments of the present invention, and therefore will not be repeated here. All systems / devices used in the methods of the above embodiments of the present invention fall within the scope of protection of the present invention.

[0087] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0088] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions.

[0089] It should be noted that any reference numerals placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In claims that enumerate several means, several of these means may be embodied by the same hardware. The use of the terms first, second, third, etc., is merely for convenience of expression and does not indicate any order. These terms can be understood as part of the component names.

[0090] Furthermore, it should be noted that in the description of this specification, the terms "one embodiment," "some embodiments," "embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0091] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the claims should be interpreted to include both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0092] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, then this invention should also include these modifications and variations.

Claims

1. A method for frame extraction in streaming media surveillance video, characterized in that, include: The streaming media server receives a frame extraction request sent by a third-party platform, acquires and parses the monitoring video stream according to the frame extraction request, and extracts frames from the monitoring video stream according to the frame extraction parameters of the frame extraction request to obtain target binary video frame data, and stores the target binary video frame data in the data cache queue. The target binary video frame data is obtained by consuming from the data cache queue, and compressed and decoded according to the output format of the third-party platform to obtain compressed and decoded target binary video frame data. The compressed and decoded target binary video frame data is then sent to the third-party platform.

2. The frame extraction method for streaming media monitoring video as described in claim 1, characterized in that, The frame extraction parameters include frame extraction interval, frame extraction count, and frame type. Simultaneously extracting frames from the monitoring video stream according to the frame extraction parameters of the frame extraction request to obtain target binary video frame data includes: Simultaneously, the monitoring video stream is frame-sampling based on the frame-sampling interval, the number of frame-sampling times, and the frame type, and the dynamic feature values ​​of the monitoring video stream are extracted in real time. The frame-sampling interval and the number of frame-samplings are dynamically adjusted based on the dynamic feature values ​​to obtain the dynamically adjusted frame-sampling interval and the dynamically adjusted number of frame-samplings. Based on the dynamically adjusted frame extraction interval and the dynamically adjusted number of frame extractions, target binary video frame data that conforms to the frame type are filtered from the monitoring video stream.

3. The frame extraction method for streaming media monitoring video as described in claim 2, characterized in that, The dynamic feature value includes the proportion of the motion region. The step of dynamically adjusting the frame skipping interval and the number of frame skipping based on the dynamic feature value to obtain the dynamically adjusted frame skipping interval and the dynamically adjusted number of frame skipping includes: Determine whether the proportion of the motion area exceeds a first threshold. If yes, mark the monitoring video stream as a high dynamic scene; otherwise, mark the monitoring video stream as a static scene. The system acquires historical frame-sampling effect data from the third-party platform, generates adjustment parameters based on the historical frame-sampling effect data and the reinforcement learning model, and automatically shortens the frame-sampling interval and the number of frame-samplings in the high-dynamic scene according to the adjustment parameters to obtain the automatically shortened frame-sampling interval and the automatically shortened number of frame-samplings. The system also automatically extends the frame-sampling interval and the number of frame-samplings in the static scene according to the adjustment parameters to obtain the automatically extended frame-sampling interval and the automatically extended number of frame-samplings.

4. The frame extraction method for streaming media monitoring video as described in claim 1, characterized in that, The target binary video frame data includes a timestamp and a device ID. The step of storing the target binary video frame data in a data cache queue includes: A time-series index caching mechanism is adopted to store the target binary video frame data into a data cache queue according to the timestamp and the device ID. Target binary video frame data that conforms to the first time period is selected as hot data based on the timestamp, while target binary video frame data that does not conform to the first time period is selected as cold data. The hot data is stored in a memory queue, and the cold data is stored on the disk.

5. The frame extraction method for streaming media monitoring video as described in claim 1, characterized in that, The step of compressing and decoding the target binary video frame data according to the output format of the third-party platform includes: A transcoding task is created according to the output format of the third-party platform, and the transcoding task is passed to the transcoding thread pool to call the GPU acceleration unit to compress and decode the target binary video frame data.

6. A frame extraction system for streaming media surveillance video, characterized in that, include: A streaming media server is used to receive frame extraction requests sent by a third-party platform, acquire and parse the monitoring video stream according to the frame extraction request, extract frames from the monitoring video stream according to the frame extraction parameters of the frame extraction request to obtain target binary video frame data, and store the target binary video frame data in a data cache queue. A transcoding server is used to consume the target binary video frame data from the data cache queue, compress and decode the target binary video frame data according to the output format of the third-party platform to obtain compressed and decoded target binary video frame data, and send the compressed and decoded target binary video frame data to the third-party platform.

7. A frame extraction system for streaming media monitoring video as described in claim 6, characterized in that, The frame extraction parameters include frame extraction interval, frame extraction count, and frame type; the streaming media server includes: An intelligent frame extraction server is used to simultaneously extract frames from the monitoring video stream based on the frame extraction interval, the number of frame extractions, and the frame type, and extract dynamic feature values ​​of the monitoring video stream in real time; dynamically adjust the frame extraction interval and the number of frame extractions based on the dynamic feature values ​​to obtain dynamically adjusted frame extraction interval and dynamically adjusted number of frame extractions; and filter target binary video frame data that conforms to the frame type from the monitoring video stream based on the dynamically adjusted frame extraction interval and dynamically adjusted number of frame extractions.

8. The frame extraction system for streaming media monitoring video as described in claim 7, characterized in that, The dynamic feature value includes the proportion of the motion region, and the intelligent frame extraction server is specifically: Determine whether the proportion of the motion area exceeds a first threshold. If yes, mark the monitoring video stream as a high dynamic scene; otherwise, mark the monitoring video stream as a static scene. The system acquires historical frame-sampling effect data from the third-party platform, generates adjustment parameters based on the historical frame-sampling effect data and the reinforcement learning model, and automatically shortens the frame-sampling interval and the number of frame-samplings in the high-dynamic scene according to the adjustment parameters to obtain the automatically shortened frame-sampling interval and the automatically shortened number of frame-samplings. The system also automatically extends the frame-sampling interval and the number of frame-samplings in the static scene according to the adjustment parameters to obtain the automatically extended frame-sampling interval and the automatically extended number of frame-samplings.

9. A frame extraction system for streaming media monitoring video as described in claim 6, characterized in that, The streaming media server includes: The intelligent caching server is used to store the target binary video frame data into a data cache queue according to the timestamp and the device ID using a time-series index caching mechanism. It also filters out target binary video frame data that conforms to a first time period as hot data based on the timestamp, and selects target binary video frame data that does not conform to the first time period as cold data. The hot data is stored in a memory queue, and the cold data is stored in a disk.

10. A frame extraction system for streaming media monitoring video as described in claim 6, characterized in that, The transcoding server is specifically: A transcoding task is created according to the output format of the third-party platform, and the transcoding task is passed to the transcoding thread pool to call the GPU acceleration unit to compress and decode the target binary video frame data.

Images