Video processing method and device, computer equipment, storage medium and computer program product

Abstract

The invention relates to a video processing method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring a plurality of video files of an original video stream; decoding the current video file in the plurality of video files to obtain a decoded video file of the current video file; extracting a plurality of video frames in the decoded video file, and storing each video frame in a target cache queue in sequence, so that a video processing end processes the video frames in sequence; and under the condition that each video frame is processed, taking the next video file corresponding to the current video file as a new current video file, and skipping to the step of decoding the current video file in the plurality of video files to obtain a decoded video file of the current video file. The current video file is the last video file in the plurality of video files. By adopting the method, the video processing efficiency can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a video processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] Currently, in order to meet the ever-increasing demand for video data, efficient video processing has become crucial.

[0003] In traditional technology, video processing is generally done directly through edge computing devices. However, edge computing devices have limited hardware resources and cannot process every frame of the video in a timely manner, which can easily lead to congestion and low video processing efficiency. Summary of the Invention

[0004] Therefore, it is necessary to provide a video processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve the processing efficiency of video in response to the above-mentioned technical problems.

[0005] In a first aspect, this application provides a video processing method applied to an edge computing device, comprising:

[0006] Obtain multiple video files from the raw video stream;

[0007] In response to a video processing request sent by the video processing terminal for the original video stream, the current video file among the plurality of video files is decoded to obtain the decoded video file of the current video file;

[0008] Multiple video frames are extracted from the decoded video file, and each video frame is stored sequentially in the target cache queue of the edge computing device, so that the video processing terminal processes each video frame in the target cache queue sequentially;

[0009] After each video frame is processed, the next video file corresponding to the current video file is taken as the new current video file, and the process jumps to the step of decoding the current video file among the multiple video files to obtain the decoded video file of the current video file, until the current video file is the last video file among the multiple video files.

[0010] In one embodiment, obtaining multiple video files from the original video stream includes:

[0011] Determine the current application scenario of the original video stream;

[0012] Query the correspondence between application scenarios and time segments to obtain the time segment corresponding to the current application scenario, which is used as the target time segment of the original video stream.

[0013] The original video stream is segmented according to the target time interval to obtain the multiple video files.

[0014] In one embodiment, prior to acquiring multiple video files from the original video stream, the method further includes:

[0015] The video stream to be analyzed is obtained from the network video recorder using preset protocol information;

[0016] The current resolution and current frame rate of the video stream to be analyzed are adjusted to obtain an adjusted video stream; the adjusted video stream is used to represent a video stream that simultaneously meets the preset resolution and preset frame rate.

[0017] The adjusted video stream is denoised to obtain a denoised video stream, which is used as the original video stream.

[0018] In one embodiment, before decoding the current video file among the plurality of video files to obtain the decoded video file of the current video file, the method further includes:

[0019] The current video file among the plurality of video files is loaded to obtain the loaded video file of the current video file;

[0020] The step of decoding the current video file among the plurality of video files to obtain the decoded video file of the current video file includes:

[0021] The loaded video file is decoded to obtain the decoded video file of the current video file.

[0022] In one embodiment, the step of decoding the loaded video file to obtain the decoded video file of the current video file includes:

[0023] The encoding format information and file encapsulation format information of the loaded video file are determined and used as the target format information of the loaded video file.

[0024] From multiple candidate decoding tools, a candidate decoding tool that meets the target format information is selected as the target decoding tool for the loaded video file;

[0025] The target decoding tool is used to decode the loaded video file to obtain the decoded video file.

[0026] In one embodiment, extracting multiple video frames from the decoded video file includes:

[0027] Receive processing request information for the original video stream sent by the video processing terminal;

[0028] The processing requirement information is parsed to obtain parsed information;

[0029] Extract the interval information from the parsed information and use it as the current interval information of the decoded video file;

[0030] Based on the current interval information, extract multiple video frames from the decoded video file.

[0031] Secondly, this application also provides a video processing apparatus for use in edge computing devices, comprising:

[0032] The video acquisition module is used to acquire multiple video files from the original video stream;

[0033] The video decoding module is used to respond to a video processing request sent by the video processing terminal for the original video stream, and to decode the current video file among the multiple video files to obtain the decoded video file of the current video file;

[0034] The video storage module is used to extract multiple video frames from the decoded video file and store each video frame sequentially into the target cache queue of the edge computing device, so that the video processing terminal processes each video frame in the target cache queue sequentially.

[0035] The condition judgment module is used to, when each video frame is processed, take the next video file corresponding to the current video file as the new current video file, and jump to the step of decoding the current video file among the multiple video files to obtain the decoded video file of the current video file, until the current video file is the last video file among the multiple video files.

[0036] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0037] Obtain multiple video files from the raw video stream;

[0038] In response to a video processing request sent by the video processing terminal for the original video stream, the current video file among the plurality of video files is decoded to obtain the decoded video file of the current video file;

[0039] Multiple video frames are extracted from the decoded video file, and each video frame is stored sequentially in the target cache queue of the edge computing device, so that the video processing terminal processes each video frame in the target cache queue sequentially;

[0040] After each video frame is processed, the next video file corresponding to the current video file is taken as the new current video file, and the process jumps to the step of decoding the current video file among the multiple video files to obtain the decoded video file of the current video file, until the current video file is the last video file among the multiple video files.

[0041] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0042] Obtain multiple video files from the raw video stream;

[0043] In response to a video processing request sent by the video processing terminal for the original video stream, the current video file among the plurality of video files is decoded to obtain the decoded video file of the current video file;

[0044] Multiple video frames are extracted from the decoded video file, and each video frame is stored sequentially in the target cache queue of the edge computing device, so that the video processing terminal processes each video frame in the target cache queue sequentially;

[0045] After each video frame is processed, the next video file corresponding to the current video file is taken as the new current video file, and the process jumps to the step of decoding the current video file among the multiple video files to obtain the decoded video file of the current video file, until the current video file is the last video file among the multiple video files.

[0046] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0047] Obtain multiple video files from the raw video stream;

[0048] In response to a video processing request sent by the video processing terminal for the original video stream, the current video file among the plurality of video files is decoded to obtain the decoded video file of the current video file;

[0049] Multiple video frames are extracted from the decoded video file, and each video frame is stored sequentially in the target cache queue of the edge computing device, so that the video processing terminal processes each video frame in the target cache queue sequentially;

[0050] After each video frame is processed, the next video file corresponding to the current video file is taken as the new current video file, and the process jumps to the step of decoding the current video file among the multiple video files to obtain the decoded video file of the current video file, until the current video file is the last video file among the multiple video files.

[0051] The aforementioned video processing method, apparatus, computer equipment, storage medium, and computer program product first acquire multiple video files of the original video stream, and in response to a video processing request sent by the video processing end for the original video stream, decode the current video file among the multiple video files to obtain a decoded video file of the current video file. Next, multiple video frames are extracted from the decoded video file, and each video frame is sequentially stored in the target cache queue of the edge computing device. The video processing end processes each video frame in the target cache queue sequentially. When the processing of each video frame is completed, the next video file corresponding to the current video file is taken as the new current video file, and the process jumps to the step of decoding the current video file among the multiple video files to obtain the decoded video file of the current video file, until the current video file is the last video file among the multiple video files. In this way, during video processing, edge computing devices decode and extract multiple video files from the original video stream, storing the processed video frames in a target buffer queue. This allows the video processing end to sequentially retrieve video frames from the queue for processing, eliminating the need to wait for the entire video file to be decoded before starting processing. Instead, it can retrieve processed video frames from the target buffer queue during the decoding process, effectively reducing waiting time and improving video processing efficiency. Furthermore, this process effectively alleviates the problem of limited hardware resources on edge computing devices, which can hinder the timely processing of each frame in the video and lead to congestion, thus further improving video processing efficiency. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is an application environment diagram of a video processing method in one embodiment;

[0054] Figure 2 This is a flowchart illustrating a video processing method in one embodiment;

[0055] Figure 3 This is a flowchart illustrating a low-resource video playback frame extraction method based on segmented storage and on-demand frame extraction in one embodiment.

[0056] Figure 4 This is a flowchart illustrating a video processing method in another embodiment;

[0057] Figure 5 This is a flowchart illustrating a low-resource video playback frame extraction method based on segmented storage and on-demand frame extraction in another embodiment.

[0058] Figure 6 This is a structural block diagram of a video processing device in one embodiment;

[0059] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0061] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0062] The video processing method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, the edge computing device 102 communicates with the video processing terminal 104 via a network. Specifically, refer to... Figure 1The edge computing device 102 acquires multiple video files from the original video stream. In response to a video processing request for the original video stream sent by the video processing terminal 104, it decodes the current video file from the multiple video files to obtain a decoded video file. It then extracts multiple video frames from the decoded video file and stores each video frame sequentially into the target cache queue of the edge computing device. The video processing terminal 104 processes each video frame in the target cache queue sequentially. After each video frame is processed, the next video file corresponding to the current video file is taken as the new current video file, and the process jumps to the step of decoding the current video file from the multiple video files to obtain the decoded video file of the current video file, until the current video file is the last video file among the multiple video files.

[0063] In this context, edge computing device 102 refers to a device used for decoding and extracting video data, also known as an RTSP (Real-Time Streaming Protocol) client, which can be a terminal or a server. Video processing terminal refers to a terminal used for video playback and AI (Artificial Intelligence) recognition, also known as the consumer end or consumer side. Terminals can be, but are not limited to, various personal computers, laptops, smartphones, and tablets. Servers can be implemented using independent servers or server clusters composed of multiple servers.

[0064] In one exemplary embodiment, such as Figure 2 As shown, a video processing method is provided, which is applied to... Figure 1 Taking an edge computing device as an example, this embodiment of the method includes the following steps:

[0065] Step S201: Obtain multiple video files from the original video stream.

[0066] The original video stream refers to the video stream that the video processing end needs to play back and perform AI recognition on.

[0067] Here, video files refer to files obtained by segmenting the original video stream. It should be noted that video files include video file 1, video file 2, and video file n, all of which are stored in the file system of the edge computing device.

[0068] For example, the edge computing device acquires the original video stream from the network video recorder and performs segmentation processing on the original video stream to obtain multiple video files of the original video stream.

[0069] Step S202: In response to the video processing request sent by the video processing terminal for the original video stream, the current video file among multiple video files is decoded to obtain the decoded video file of the current video file.

[0070] Among them, the video processing request is used to indicate the request information for processing the original video stream (video playback and AI recognition).

[0071] The current video file refers to the video file that needs to be decoded within the current time period.

[0072] The decoded video file refers to the current video file after decoding.

[0073] For example, the edge computing device establishes a network path with the video processing end and listens for video processing requests sent by the video processing end through the network path; then, in response to the video processing request for the original video stream sent by the video processing end, the edge computing device uses a decoding tool to decode the current video file among multiple video files to obtain the decoded video file of the current video file.

[0074] Step S203: Extract multiple video frames from the decoded video file and store each video frame sequentially into the target cache queue of the edge computing device, so that the video processing end processes each video frame in the target cache queue sequentially.

[0075] In this context, video frames represent static images of the decoded video file at a specific moment. It should be noted that all video frames are stored in the memory of the edge computing device.

[0076] The target cache queue refers to the cache queue in the edge computing device. In practical scenarios, the target cache queue refers to a cache queue with a capacity of 2 frames, such as... Figure 3 As shown.

[0077] For example, the edge computing device receives processing request information for the original video stream sent by the video processing terminal through a network path between the two terminals. Then, based on this processing request information, the edge computing device generates a video extraction instruction for the decoded video file. Next, based on this video extraction instruction, the edge computing device extracts multiple video frames from the decoded video file. Then, the edge computing device sequentially stores each video frame into a target cache queue, allowing the video processing terminal to process each video frame in the target cache queue sequentially. For example, the edge computing device groups multiple video frames into pairs sequentially to obtain grouped video frames, such as frames 1 and 2 in one group, and frames 3 and 4 in another group, allowing the video processing terminal to process the grouped video frames sequentially.

[0078] It should be noted that the video processing end can process each video frame in the target buffer queue as needed, and can process only one video frame or two video frames at the same time. After the video processing end processes one video frame at a time, the edge computing device will continue to take the next video frame in sequence to supplement the target buffer queue.

[0079] Step S204: After each video frame is processed, the next video file corresponding to the current video file is taken as the new current video file, and the process jumps to the step of decoding the current video file among multiple video files to obtain the decoded video file of the current video file, until the current video file is the last video file among multiple video files.

[0080] Here, "last video file" refers to the video file that appears last in the sequence of multiple video files. In practical scenarios, the last video file is video file n, such as... Figure 3 As shown.

[0081] For example, the edge computing device judges the processing result of each video frame and obtains the judgment result of each video frame; when the judgment result of each video frame indicates that the processing is completed, the edge computing device takes the next video file corresponding to the current video file as the new current video file, and jumps to the step of decoding the current video file among multiple video files to obtain the decoded video file of the current video file, until the current video file is the last video file among multiple video files.

[0082] In the above video processing method, multiple video files of the original video stream are first obtained. In response to the video processing request sent by the video processing end for the original video stream, the current video file among the multiple video files is decoded to obtain the decoded video file of the current video file. Then, multiple video frames are extracted from the decoded video file, and each video frame is stored sequentially in the target cache queue of the edge computing device. The video processing end processes each video frame in the target cache queue sequentially. When the processing of each video frame is completed, the next video file corresponding to the current video file is taken as the new current video file, and the process jumps to the step of decoding the current video file among the multiple video files to obtain the decoded video file of the current video file, until the current video file is the last video file among the multiple video files. In this way, during video processing, edge computing devices decode and extract multiple video files from the original video stream, storing the processed video frames in a target buffer queue. This allows the video processing end to sequentially retrieve video frames from the queue for processing, eliminating the need to wait for the entire video file to be decoded before starting processing. Instead, it can retrieve processed video frames from the target buffer queue during the decoding process, effectively reducing waiting time and improving video processing efficiency. Furthermore, this process effectively alleviates the problem of limited hardware resources on edge computing devices, which can hinder the timely processing of each frame in the video and lead to congestion, thus further improving video processing efficiency.

[0083] In an exemplary embodiment, step S201 above, obtaining multiple video files of the original video stream, specifically includes the following: determining the current application scenario of the original video stream; querying the correspondence between the application scenario and the segmented time to obtain the segmented time corresponding to the current application scenario, which is used as the target segmented time of the original video stream; and segmenting the original video stream according to the target segmented time to obtain multiple video files.

[0084] The current application scenario is used to indicate the application scenario in which the original video stream is located, such as surveillance.

[0085] The correspondence between application scenarios and time segments indicates the association between them. For example, in the case of routine public place monitoring, the corresponding time segment is 5 minutes; in the case of key area monitoring, the corresponding time segment is 2 minutes.

[0086] The target segment time is used to represent the segment time corresponding to the original video stream.

[0087] For example, the edge computing device obtains the identification information (such as the video name) of the original video stream and determines the current application scenario of the original video stream based on the identification information; then, the edge computing device queries the database for the correspondence between the application scenario and the segmented time to obtain the segmented time corresponding to the current application scenario, and uses the segmented time as the target segmented time of the original video stream; then, the original video stream is segmented according to the target segmented time to obtain multiple video files.

[0088] In this embodiment, the target segment time of the original video stream is determined based on the current application scenario of the original video stream. This allows for the determination of a more suitable target segment time for the original video stream, which in turn enables effective segmentation of the original video stream based on the target segment time, thereby improving data processing efficiency.

[0089] In an exemplary embodiment, step S201, before acquiring multiple video files of the original video stream, specifically includes the following: acquiring the video stream to be analyzed from the network video recorder through preset protocol information; adjusting the current resolution and current frame rate of the video stream to be analyzed to obtain an adjusted video stream; using the adjusted video stream to represent a video stream that simultaneously meets the preset resolution and preset frame rate; and performing denoising processing on the adjusted video stream to obtain a denoised video stream, which serves as the original video stream.

[0090] The preset protocol information refers to pre-defined protocol information, such as the RTSP protocol. It should be noted that the preset protocol information depends on the specific circumstances.

[0091] Among them, network video recorder refers to NVR (Network Video Recorder) equipment.

[0092] The video stream to be analyzed is used to represent the video stream in the network video recorder.

[0093] The current resolution is used to represent the resolution of the video stream to be analyzed at the current time.

[0094] The current frame rate is used to represent the frame rate of the video stream being analyzed at the current time.

[0095] The adjusted video stream is used to represent the video stream to be analyzed after adjustment.

[0096] The preset resolution refers to the resolution that is set in advance. It should be noted that the preset resolution depends on the specific circumstances.

[0097] The preset frame rate refers to the frame rate set in advance. It should be noted that the preset frame rate depends on the situation.

[0098] Among them, the denoised video stream refers to the adjusted video stream after denoising processing.

[0099] For example, the edge computing device obtains the video stream to be analyzed from the network video recorder associated with the edge computing device through preset protocol information; then, the edge computing device adjusts the current resolution of the video stream to be analyzed to a preset resolution and the current frame rate of the video stream to be analyzed to a preset frame rate, thus obtaining an adjusted video stream; then, the edge computing device identifies the noise information in the adjusted video stream and generates a denoising instruction corresponding to the noise information; then, the edge computing device performs denoising processing on the adjusted video stream according to the denoising instruction, thus obtaining a denoised video stream as the original video stream.

[0100] In this embodiment, by adjusting the current resolution and frame rate of the video stream to be analyzed to preset values, it can be ensured that the video data entering the subsequent processing flow has a unified standard, so that the processed video stream can better adapt to the subsequent processing flow, which is conducive to improving the accuracy and stability of the analysis results, and thus improving the detection accuracy of the video.

[0101] In an exemplary embodiment, before decoding the current video file among multiple video files to obtain the decoded video file of the current video file, step S202 specifically includes the following: loading the current video file among multiple video files to obtain the loaded video file of the current video file.

[0102] Therefore, step S202 above, which decodes the current video file among multiple video files to obtain the decoded video file of the current video file, specifically includes the following: decoding the loaded video file to obtain the decoded video file of the current video file.

[0103] The loaded video file refers to the current video file after loading processing.

[0104] For example, the edge computing device loads the current video file from among multiple video files stored in the file system of the edge computing device to obtain a loaded video file of the current video file, and stores the loaded video file in the memory of the edge computing device; then, the edge computing device decodes the loaded video file stored in the memory of the edge computing device to obtain a decoded video file of the current video file.

[0105] In this embodiment, by loading the video, the metadata of the video file can be read to understand its format characteristics. By decoding the video, video data of various encoding formats can be converted into raw data formats that the system can directly process, providing a unified data foundation for subsequent video analysis, editing and other operations.

[0106] In an exemplary embodiment, the loaded video file is decoded to obtain a decoded video file. This process includes: determining the encoding format information and file encapsulation format information of the loaded video file as the target format information; selecting a candidate decoding tool that meets the target format information from multiple candidate decoding tools as the target decoding tool for the loaded video file; and decoding the loaded video file using the target decoding tool to obtain the decoded video file.

[0107] The encoding format information is used to indicate the format information corresponding to the compression of the loaded video file, such as H.264.

[0108] The file encapsulation format information is used to indicate the format information corresponding to the integration of the loaded video files, such as MKV (Matroska Video, a multimedia container format).

[0109] Among them, target format information refers to the format information of the loaded video file, including encoding format information and file encapsulation format information.

[0110] Among them, candidate decoding tools refer to the decoding tools to be selected.

[0111] The target decoding tool refers to the decoding tool corresponding to the loaded video file, such as FFmpeg (FastForward Moving Picture Experts Group Audio Video Encoder).

[0112] For example, the edge computing device uses a multimedia analysis tool to determine the encoding format information and file encapsulation format information of the loaded video file, and uses the encoding format information and file encapsulation format information of the loaded video file as the target format information of the loaded video file; then, the edge computing device selects a candidate decoding tool that meets the target format information from multiple candidate decoding tools, and uses the candidate decoding tool as the target decoding tool for the loaded video file; then, the edge computing device uses the target decoding tool to decode the loaded video file to obtain the decoded video file.

[0113] In this embodiment, by determining the target format information and filtering out decoding tools that meet the conditions, it can be ensured that the selected decoding tool is completely compatible with the format of the video file, thereby ensuring the smooth progress of the decoding process, avoiding decoding errors or failures caused by format incompatibility, and improving the accuracy of video decoding.

[0114] In an exemplary embodiment, step S203 above, extracting multiple video frames from the decoded video file, specifically includes the following: receiving processing requirement information for the original video stream sent by the video processing terminal; parsing the processing requirement information to obtain parsing information; extracting the interval information from the parsing information as the current interval information of the decoded video file; and extracting multiple video frames from the decoded video file according to the current interval information.

[0115] The processing requirement information indicates the relevant requirements for processing the original video stream, such as frame extraction interval (e.g., extracting one frame every certain number of frames) and specific frame extraction (specifying the extraction of frames with specific numbers). It should be noted that edge computing devices can perform frame extraction processing according to the processing requirement information from the video processing end.

[0116] Among them, the parsed information is used to represent the information obtained by parsing and processing the processing requirement information.

[0117] The interval information is used to represent the video extraction interval, also known as the frame extraction interval.

[0118] The current interval information is used to represent the interval information corresponding to the decoded video file.

[0119] For example, the edge computing device receives processing request information for the original video stream sent by the video processing terminal through a network path between the two terminals. Then, the edge computing device parses the processing request information to obtain parsed information. Next, the edge computing device extracts the interval information from the parsed information and uses this interval information as the current interval information of the decoded video file. Then, the edge computing device generates a video extraction instruction for the decoded video file based on the current interval information. Finally, the edge computing device extracts multiple video frames from the decoded video file according to the video extraction instruction.

[0120] In this embodiment, by using the processing requirement information based on the original video stream, the current interval information of the decoded video file can be determined, thereby effectively extracting multiple video frames from the decoded video file. This enables the edge computing device to meet video processing requirements of varying complexity, which is beneficial for improving video processing efficiency.

[0121] In one exemplary embodiment, such as Figure 4 As shown, another video processing method is provided, which can be applied to... Figure 1 Taking edge computing devices as an example, the following steps are included:

[0122] Step S401: Obtain the video stream to be analyzed from the network video recorder using preset protocol information.

[0123] Step S402: Adjust the current resolution and current frame rate of the video stream to be analyzed to obtain the adjusted video stream; the adjusted video stream is used to represent the video stream that simultaneously meets the preset resolution and preset frame rate.

[0124] Step S403: Denoise the adjusted video stream to obtain a denoised video stream, which is used as the original video stream.

[0125] Step S404: Determine the current application scenario of the original video stream.

[0126] Step S405: Query the correspondence between application scenarios and segmented time to obtain the segmented time corresponding to the current application scenario, which is used as the target segmented time of the original video stream.

[0127] Step S406: The original video stream is segmented according to the target time segment to obtain multiple video files.

[0128] Step S407: In response to the video processing request sent by the video processing terminal for the original video stream, the current video file among multiple video files is decoded to obtain the decoded video file of the current video file.

[0129] Step S408: Receive processing requirement information for the original video stream sent by the video processing terminal; parse the processing requirement information to obtain parsing information; extract the interval information from the parsing information as the current interval information of the decoded video file; extract multiple video frames from the decoded video file according to the current interval information.

[0130] Step S409: Store each video frame sequentially into the target cache queue of the edge computing device, so that the video processing end processes each video frame in the target cache queue sequentially.

[0131] Step S410: After each video frame is processed, the next video file corresponding to the current video file is taken as the new current video file, and the process jumps to the step of decoding the current video file among multiple video files to obtain the decoded video file of the current video file, until the current video file is the last video file among multiple video files.

[0132] In the aforementioned video processing method, during the video processing, edge computing devices decode and extract multiple video files from the original video stream, and store the processed video frames in a target buffer queue. This allows the video processing end to sequentially retrieve video frames from the queue for processing, eliminating the need to wait for the entire video file to be decoded before starting processing. Instead, the processing end can retrieve processed video frames from the target buffer queue during the decoding process, effectively reducing waiting time and improving video processing efficiency. Furthermore, the entire process effectively alleviates the problem of limited hardware resources on edge computing devices, which can hinder timely processing of each frame in the video and lead to congestion, thus further improving video processing efficiency.

[0133] In an exemplary embodiment, to more clearly illustrate the video processing method provided in this application, the following detailed description uses a specific embodiment. In one embodiment, as... Figure 5 As shown, this application also provides a low-resource video playback frame extraction method based on segmented storage and on-demand frame extraction. During video processing, multiple video files of the original video stream are first acquired. In response to a video processing request sent by the video processing end for the original video stream, the current video file among the multiple video files is decoded to obtain the decoded video file. Then, multiple video frames are extracted from the decoded video file, and each video frame is sequentially stored in the target cache queue of the edge computing device. The video processing end processes each video frame in the target cache queue sequentially. Upon completion of processing each video frame, the next video file corresponding to the current video file is taken as the new current video file, and the process jumps to the step of decoding the current video file among the multiple video files to obtain the decoded video file, until the current video file is the last video file among the multiple video files. Specifically, it includes the following:

[0134] 1. Obtain the raw video stream:

[0135] Obtain the raw video stream using RTSP (Real-Time Streaming Protocol) or other protocols.

[0136] 2. Video stream segmentation:

[0137] The original video stream is divided into segments according to a certain time period (e.g., every 10 seconds) and saved as multiple short video files.

[0138] These short video files are saved to hard drives or storage media.

[0139] 3. Load segmented files into memory:

[0140] When video playback or AI recognition is required, the first video file is loaded into memory.

[0141] Once each video file is loaded, decoding and frame extraction begin immediately, and the frames are pushed into a queue of 2 buffered frames (first-in, first-out) to await consumption by the consumer side (such as an AI recognition system).

[0142] 4. On-demand frame extraction and processing:

[0143] After consuming one frame at a time, the consumer continues to decode from the video buffer and retrieves the next frame to supplement the buffer area.

[0144] When the current video stream ends, read the next video file and repeat the above steps.

[0145] 5. Loop processing:

[0146] The entire process is repeated until all video segments have been processed.

[0147] In the above embodiments, during video processing, the edge computing device decodes and extracts multiple video files from the original video stream and stores the processed video frames in a target cache queue. This allows the video processing end to sequentially retrieve video frames from the queue for processing, eliminating the need to wait for the entire video file to be decoded before starting processing. Instead, the processing end can retrieve processed video frames from the target cache queue during the decoding process. This effectively reduces waiting time during processing and improves video processing efficiency. Furthermore, the entire process effectively alleviates the problem of limited hardware resources on edge computing devices, which can hinder timely processing of each frame in the video and lead to congestion, thus further improving video processing efficiency. Meanwhile, by storing video streams in segments, the amount of data processed per session is reduced, and memory usage is lowered. This enables efficient operation of video playback and AI recognition tasks even on resource-constrained devices. Most small NVR devices do not support configuring output frame rates. Edge computing devices only need NVR devices with basic playback functions to complete AI recognition of video recordings, thus reducing the requirements for NVR devices. The segmentation time and frame extraction strategy can be flexibly adjusted according to different application scenarios to adapt to various video processing needs without causing frame drops or wasting resources while waiting.

[0148] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0149] Based on the same inventive concept, this application also provides a video processing apparatus for implementing the video processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more video processing apparatus embodiments provided below can be found in the limitations of the video processing method described above, and will not be repeated here.

[0150] In one exemplary embodiment, such as Figure 6 As shown, a video processing apparatus is provided, including: a video acquisition module 601, a video decoding module 602, a video storage module 603, and a condition judgment module 604, wherein:

[0151] The video acquisition module 601 is used to acquire multiple video files from the original video stream.

[0152] The video decoding module 602 is used to respond to a video processing request sent by the video processing terminal for the original video stream, and to decode the current video file among multiple video files to obtain the decoded video file of the current video file.

[0153] The video storage module 603 is used to extract multiple video frames from the decoded video file and store each video frame sequentially into the target cache queue of the edge computing device, so that the video processing end processes each video frame in the target cache queue sequentially.

[0154] The condition judgment module 604 is used to, after each video frame is processed, take the next video file corresponding to the current video file as the new current video file, and jump to the step of decoding the current video file among multiple video files to obtain the decoded video file of the current video file, until the current video file is the last video file among multiple video files.

[0155] In an exemplary embodiment, the video acquisition module 601 is further configured to determine the current application scenario of the original video stream; query the correspondence between the application scenario and the segmented time to obtain the segmented time corresponding to the current application scenario, which is used as the target segmented time of the original video stream; and perform segmentation processing on the original video stream according to the target segmented time to obtain multiple video files.

[0156] In an exemplary embodiment, the video processing apparatus further includes a video processing module, configured to acquire a video stream to be analyzed from a network video recorder via preset protocol information; adjust the current resolution and current frame rate of the video stream to be analyzed to obtain an adjusted video stream; use the adjusted video stream to represent a video stream that simultaneously satisfies the preset resolution and preset frame rate; and perform denoising processing on the adjusted video stream to obtain a denoised video stream, which serves as the original video stream.

[0157] In one exemplary embodiment, the video processing apparatus further includes a video loading module for loading the current video file from a plurality of video files to obtain a loaded video file of the current video file; and a video decoding module 602 for decoding the loaded video file to obtain a decoded video file of the current video file.

[0158] In an exemplary embodiment, the video decoding module 602 is further configured to determine the encoding format information and file encapsulation format information of the loaded video file as the target format information of the loaded video file; select a candidate decoding tool that meets the target format information from multiple candidate decoding tools as the target decoding tool of the loaded video file; and perform decoding processing on the loaded video file through the target decoding tool to obtain the decoded video file.

[0159] In an exemplary embodiment, the video storage module 603 is further configured to receive processing requirement information for the original video stream sent by the video processing terminal; parse the processing requirement information to obtain parsing information; extract the interval information in the parsing information as the current interval information of the decoded video file; and extract multiple video frames from the decoded video file according to the current interval information.

[0160] Each module in the aforementioned video processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0161] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores data such as video files and video frames. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network. When the computer program is executed by the processor, it implements a video processing method.

[0162] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0163] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0164] In one exemplary embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above-described method embodiments.

[0165] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0166] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0167] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0168] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A video processing method, characterized in that, Applied to edge computing devices, the method includes: Obtain multiple video files from the raw video stream; In response to a video processing request sent by the video processing terminal for the original video stream, the current video file among the plurality of video files is decoded to obtain the decoded video file of the current video file; Multiple video frames are extracted from the decoded video file, and each video frame is stored sequentially in the target cache queue of the edge computing device, so that the video processing terminal processes each video frame in the target cache queue sequentially; After each video frame is processed, the next video file corresponding to the current video file is taken as the new current video file, and the process jumps to the step of decoding the current video file among the multiple video files to obtain the decoded video file of the current video file, until the current video file is the last video file among the multiple video files.

2. The method according to claim 1, characterized in that, The multiple video files from which the original video stream is obtained include: Determine the current application scenario of the original video stream; Query the correspondence between application scenarios and time segments to obtain the time segment corresponding to the current application scenario, which is used as the target time segment of the original video stream. The original video stream is segmented according to the target time interval to obtain the multiple video files.

3. The method according to claim 1, characterized in that, Before acquiring multiple video files from the raw video stream, the process also includes: The video stream to be analyzed is obtained from the network video recorder using preset protocol information; The current resolution and current frame rate of the video stream to be analyzed are adjusted to obtain an adjusted video stream; the adjusted video stream is used to represent a video stream that simultaneously meets the preset resolution and preset frame rate. The adjusted video stream is denoised to obtain a denoised video stream, which is used as the original video stream.

4. The method according to claim 1, characterized in that, Before decoding the current video file among the plurality of video files to obtain the decoded video file of the current video file, the process further includes: The current video file among the plurality of video files is loaded to obtain the loaded video file of the current video file; The step of decoding the current video file among the plurality of video files to obtain the decoded video file of the current video file includes: The loaded video file is decoded to obtain the decoded video file of the current video file.

5. The method according to claim 4, characterized in that, The step of decoding the loaded video file to obtain the decoded video file of the current video file includes: The encoding format information and file encapsulation format information of the loaded video file are determined and used as the target format information of the loaded video file. From multiple candidate decoding tools, a candidate decoding tool that meets the target format information is selected as the target decoding tool for the loaded video file; The target decoding tool is used to decode the loaded video file to obtain the decoded video file.

6. The method according to any one of claims 1 to 5, characterized in that, Extracting multiple video frames from the decoded video file includes: Receive processing request information for the original video stream sent by the video processing terminal; The processing requirement information is parsed to obtain parsed information; Extract the interval information from the parsed information and use it as the current interval information of the decoded video file; Based on the current interval information, extract multiple video frames from the decoded video file.

7. A video processing apparatus, characterized in that, Applied to edge computing devices, the device includes: The video acquisition module is used to acquire multiple video files from the original video stream; The video decoding module is used to respond to a video processing request sent by the video processing terminal for the original video stream, and to decode the current video file among the multiple video files to obtain the decoded video file of the current video file; The video storage module is used to extract multiple video frames from the decoded video file and store each video frame sequentially into the target cache queue of the edge computing device, so that the video processing terminal processes each video frame in the target cache queue sequentially. The condition judgment module is used to, when each video frame is processed, take the next video file corresponding to the current video file as the new current video file, and jump to the step of decoding the current video file among the multiple video files to obtain the decoded video file of the current video file, until the current video file is the last video file among the multiple video files.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images