Video frame extraction analysis method, device, equipment and storage medium

Abstract

The present application discloses a video frame extraction analysis method, device, equipment and storage medium. The present application extracts multiple video frames from a corresponding video stream based on a video stream address through a video tool, converts the multiple video frames into a picture stream, and transmits the picture stream to a cutting program through a preset pipeline; receives the picture stream from a preset pipeline through a cutting program, and cuts the picture stream into multiple video frames; calls an algorithm analysis application to analyze and process the video frame to obtain the analysis result of the video frame. Through the above technical means, the problem of low efficiency of video frame analysis and affecting the normal use of the hard disk in the prior art is solved, and the efficiency of video frame extraction and video frame analysis is improved.

Description

Technical Field

[0001] The present application relates to the field of video frame extraction technology, and in particular to a video frame extraction analysis method, device, equipment and storage medium. Background Art

[0002] At present, when the video analysis platform performs algorithmic analysis on the video, it extracts the video frames from the video stream through the video frame extraction technology. Performing algorithmic analysis on the extracted video frames can achieve high-speed and effective video analysis without reducing the effect of the analysis algorithm, meeting the video analysis platform's demand for real-time video analysis.

[0003] In the prior art, the video frames extracted from the video stream are stored in the hard disk, and then the video frames stored in the hard disk are subjected to algorithmic analysis, and the video frames that have completed the analysis are cleared from the hard disk. When extracting frames from multiple video streams simultaneously, a large number of video frames will be generated. If the large number of video frames are directly stored in the hard disk, the storage efficiency of the video frames may be affected due to the low hard disk read and write speed, thereby reducing the efficiency of the algorithmic analysis. After analyzing the large number of video frames, the video frames in the hard disk need to be continuously cleared, and repeated read and write operations will affect the service life of the damaged hard disk. Summary of the invention

[0004] The present application provides a video frame extraction and analysis method, device, equipment and storage medium, which solves the problem of low efficiency of video frame analysis and affecting the normal use of the hard disk in the prior art, improves the video frame extraction efficiency and video frame analysis efficiency, reduces the number of times the hard disk is accessed, and extends the service life of the hard disk.

[0005] In a first aspect, the present application provides a video frame analysis method, comprising:

[0006] Extracting multiple video frames from a corresponding video stream based on a video stream address through a video tool, converting the multiple video frames into a picture stream, and transmitting the picture stream to a cutting program through a preset pipeline;

[0007] Receiving the picture stream from the preset pipeline through the cutting program, and cutting the picture stream into a plurality of the video frames;

[0008] An algorithm analysis application is called to analyze and process the video frame to obtain an analysis result of the video frame.

[0009] Furthermore, extracting a plurality of video frames from a corresponding video stream based on a video stream address by using a video tool includes:

[0010] The video stream address of the front-end camera is configured, and the video tool extracts frames of the video stream corresponding to the video stream address according to a preset streaming media protocol to obtain the video frame.

[0011] Furthermore, converting the plurality of video frames into a picture stream includes:

[0012] Converting the encoding of each of the video frames into a binary encoding in a portable network graphics format;

[0013] The binary codes of the plurality of video frames are placed in the preset pipeline to obtain the picture stream.

[0014] Furthermore, dividing the picture stream into a plurality of video frames includes:

[0015] Extracting the binary code of the video frame from the picture stream according to the data format of the portable network image;

[0016] The binary code of the video frame is converted into a standard character code.

[0017] Furthermore, extracting the binary code of the video frame from the picture stream according to the data format of the portable network image includes:

[0018] Verifying the signature information in the picture stream according to the byte arrangement order of the picture stream;

[0019] After verifying the signature information in the picture stream, extracting a plurality of data blocks and an end block from the picture stream, wherein the data block includes a block length, a block type, block data, and a check code;

[0020] After the end block is extracted, the signature information, the plurality of data blocks and the end block are determined as a binary code of one video frame.

[0021] Furthermore, after cutting the picture stream into a plurality of video frames, the method further includes:

[0022] The standard character encodings of the plurality of video frames are pushed to a distributed message queue cluster through the cutting program.

[0023] Furthermore, the calling of the algorithm analysis application to analyze and process the video frame includes:

[0024] A plurality of the video frames are obtained from the distributed message queue cluster through a plurality of the algorithm analysis applications, and the plurality of the video frames are analyzed and processed.

[0025] In a second aspect, the present application provides a video frame extraction analysis device, comprising:

[0026] A pipeline transmission module is configured to extract a plurality of video frames from a corresponding video stream based on a video stream address through a video tool, convert the plurality of video frames into a picture stream, and transmit the picture stream to a cutting program through a preset pipeline;

[0027] A video frame cutting module is configured to receive the picture stream from the preset pipeline through the cutting program, and cut the picture stream into a plurality of video frames;

[0028] The video frame analysis module is configured to call an algorithm analysis application to analyze and process the video frame to obtain an analysis result of the video frame.

[0029] In a third aspect, the present application provides a video frame extraction analysis device, comprising:

[0030] One or more processors; a storage device storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the video frame analysis method as described in the first aspect.

[0031] In a fourth aspect, the present application provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to perform the video frame analysis method as described in the first aspect.

[0032] In this application, a video tool extracts multiple video frames from a corresponding video stream based on a video stream address, converts multiple video frames into a picture stream, and transmits the picture stream to a cutting program through a preset pipeline; a cutting program receives a picture stream from a preset pipeline and cuts the picture stream into multiple video frames; the cutting program is used as a producer of a distributed message queue cluster, and an algorithm analysis application is used as a consumer of a distributed message queue cluster. The cutting program caches the video frames in the distributed message queue cluster, and the algorithm analysis application obtains the video frames from the distributed message queue cluster, and analyzes and processes the video frames to obtain the analysis results of the video frames. Through the above technical means, after the video tool extracts the video frames from the video stream, the video frames are transmitted to the cutting program through the pipeline stream transmission method between applications. The video frames do not need to be written to the hard disk in a file format, which eliminates the process of reading the video frames from the hard disk, shortens the transmission time of the video frames, improves the efficiency of video frame extraction and video frame analysis, and does not need to clean the hard disk after analyzing the video frames, avoiding disk loss caused by repeated addressing operations and extending the service life of the disk. In addition, decoupling the cutting program and algorithm analysis application through a distributed message queue cluster helps to dynamically adjust the number of nodes in the server cluster, balance the processing speed of producing and consuming video frames, avoid the system from completely crashing due to overloaded requests, and improve system stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a flow chart of a video frame analysis method provided by an embodiment of the present application;

[0034] Figure 2 is a schematic diagram of a video frame transmission process provided by an embodiment of the present application;

[0035] Figure 3 This is a flowchart of converting a video stream into a picture stream provided by an embodiment of the present application;

[0036] Figure 4 This is a flow chart of a cutting program processing a picture stream provided in an embodiment of the present application;

[0037] Figure 5 It is a flowchart of extracting binary encoding of video frames from a picture stream provided by an embodiment of the present application;

[0038] Figure 6 It is a structural schematic diagram of a video frame extraction and analysis device provided in an embodiment of the present application;

[0039] Figure 7 It is a structural schematic diagram of a video frame extraction and analysis device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0041] The terms "first", "second", etc. in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0042] The video frame analysis method provided in this embodiment can be executed by a video frame analysis device. The video frame analysis device can be implemented by software and / or hardware. The video frame analysis device can be composed of two or more physical entities or one physical entity.

[0043] The video frame analysis device is installed with at least one type of operating system, and the video frame analysis device can install at least one application based on the operating system, and the application can be an application that comes with the operating system, or an application downloaded from a third-party device or server. In this embodiment, the video frame analysis device is installed with at least an application that can execute the video frame analysis method.

[0044] For ease of understanding, this embodiment is described by taking a video frame extraction analysis device as an example of a subject that executes a video frame extraction analysis method.

[0045] In one embodiment, the video cloud platform is provided with multiple cameras, and the video cloud platform manages the video streams collected by each camera. The video frame extraction and analysis device extracts frames from the multiple video streams collected by the camera at the same time, and stores the extracted video frames as image files in the hard disk, then reads the video frames from the hard disk, performs algorithm analysis on the video frames, and deletes the analyzed video frames from the hard disk. Since the video frame extraction and analysis device can extract frames from hundreds or thousands of video streams at the same time, the massive images generated need to be stored. If the frame extraction server uses a low-speed mechanical hard disk, the number of video frames it processes simultaneously will be limited, affecting the frame extraction efficiency. If the video frame extraction and analysis device uses a high-speed solid-state hard disk, although the influence of the hard disk read and write speed is eliminated, the service life of the solid-state hard disk will be seriously damaged under frequent read and write operations.

[0046] To solve the above problems, this embodiment provides a video frame extraction and analysis method to transmit video frames through a pipeline stream to avoid direct storage of video frames on a disk, thereby improving the efficiency of video frame extraction and reducing the number of hard disk accesses.

[0047] Figure 1A flow chart of a video frame analysis method provided by an embodiment of the present application is given. Figure 1 , the video frame extraction analysis method specifically includes:

[0048] S110, extracting multiple video frames from the corresponding video stream based on the video stream address through a video tool, converting the multiple video frames into a picture stream, and transmitting the picture stream to the cutting program through a preset pipeline.

[0049] The video stream address refers to the real-time live stream address of the video cloud platform, such as the rtsp / rtmp stream address. The video cloud platform binds each front-end camera to the live stream address in advance, so that the video stream collected by the front-end camera can be played on the web page where the corresponding live stream address is located. The video tool obtains the corresponding video stream based on the live stream address of each front-end camera, and extracts frames from the video stream in the background to extract multiple video frames from the video stream.

[0050] In one embodiment, the video tool uses FFmpeg (Fast Forward Mpeg), which has a video capture function, and can extract frames from the video stream collected by the camera through FFmpeg. Parameters such as the ID, frame extraction type and frame extraction frequency of each camera can be configured in FFmpeg, so that FFmpeg extracts video frames from the video stream of the corresponding camera according to the frame extraction frequency and frame extraction type, and marks the video frame with the ID of the corresponding camera, so as to subsequently determine the acquisition source of each video frame. In this embodiment, the video stream address of the front-end camera is configured, and the video tool extracts frames from the video stream corresponding to the video stream address according to the preset streaming media protocol to obtain the video frame. Exemplarily, the rtsp / rtmp stream address of the front-end camera is configured in FFmpeg, and FFmpeg extracts each video frame from the video stream corresponding to the rtsp / rtmp stream address based on mainstream streaming media protocols such as rtsp / rtmp.

[0051] In this embodiment, Figure 2 Schematic diagram of the video frame transmission process provided by the embodiment of the present application. Figure 2 As shown, the video frame extraction and analysis device includes a frame extraction server cluster, and the frame extraction server cluster is configured with multiple FFmpegs. Exemplarily, the live stream addresses of multiple cameras are input into the multiple FFmpegs configured in the frame extraction server cluster, so that multiple video streams are simultaneously extracted through multiple FFmpegs to obtain the video frames corresponding to each video stream. Moreover, the frame extraction server cluster supports dynamic expansion of FFmpeg resources. The frame extraction server cluster can meet the simultaneous frame extraction of video streams of a large number of front-end cameras, and the frame extraction efficiency is greatly improved. Therefore, the frame extraction server cluster can be applied to large-scale video stream frame extraction, and can be dynamically expanded according to the application scale, with high scalability.

[0052] In this embodiment, the image stream refers to the form in which multiple video frames are stored in the transmission pipeline of the application. The preset pipeline refers to the transmission pipeline between the video tool and the cutting program. It can be understood that in order to avoid directly storing the video frames on the disk, the pipeline stream output method of FFmpeg can be used to transmit the video frames through the pipeline to subsequent tools or applications. The video frames do not need to be saved as image files, and then do not need to be written to the hard disk, which effectively improves the transmission efficiency of the video frames. In this embodiment, Figure 3 1 is a flowchart of converting a video stream into a picture stream provided by an embodiment of the present application. Figure 3 As shown, the step of converting the video stream into the picture stream specifically includes S1101-S1102:

[0053] S1101, converting the code of each video frame into a binary code in a portable network graphics format.

[0054] S1102: placing binary codes of multiple video frames in a preset pipeline to obtain a picture stream.

[0055] Exemplarily, the image2pipe function of FFmpeg can specify that the video frames extracted by FFmpeg are output in the form of a pipeline stream. When using image2pipe, FFmpeg must first convert the encoding of each video frame into a binary encoding so that the video frame data can be transmitted in the pipeline. Portable Network Graphics (PNG) is a bitmap format of a lossless compression algorithm, which keeps the image resolution of the video frame unchanged after being converted into a binary encoding, and ensures the accuracy of subsequent analysis results. Further, after converting multiple video frames into binary encoding, FFmpeg outputs the binary encoding of multiple video frames to the pipeline between FFmpeg and the cutting program, so as to transmit the binary encoding of multiple video frames to the cutting program through the pipeline. When FFmpeg outputs the binary encoding of multiple video frames, it is equivalent to placing the binary encoding of multiple video frames in the pipeline, and the binary encoding of multiple video frames forms a string of binary data streams in the pipeline, and the binary data stream is input into the cutting program through the pipeline. Therefore, the binary data stream is the picture stream in this embodiment.

[0056] S120, receiving a picture stream from a preset pipeline through a cutting program, and dividing the picture stream into multiple video frames.

[0057] In this embodiment, the cutting program refers to a processing program for checking, segmenting and encoding the picture stream transmitted by FFmpeg, such as a python program. Figure 2, a python program is configured in the frame extraction server cluster, and the python program is connected to the FFmpeg pipeline. After extracting the video frames, FFmpeg performs binary encoding on the video frames, and transmits the binary encoding of multiple video frames to the python program through the pipeline between FFmpeg and the python program. When FFmpeg outputs the binary encoding of multiple video frames, the binary encoding of multiple video frames is strung into a binary data stream, and the algorithm analysis application cannot directly process the binary data stream. Therefore, the binary data stream needs to be split into the binary encoding of each video frame through a cutting program, and then the video frame is converted into a base64 encoding of the image that can be processed by the algorithm analysis application.

[0058] In one embodiment, Figure 4 Flowchart of the image stream processing by the cutting program provided in the embodiment of the present application. Figure 4 As shown, the steps of the cutting program processing the image stream specifically include S1201-S1202:

[0059] S1201. Extract binary codes of video frames from a picture stream according to a data format of a portable network image.

[0060] Exemplarily, when the binary code of the video frame is in the portable network graphics format, the binary code of the video frame consists of an 8-byte PNG file signature field, multiple data blocks organized according to a specific structure, and an end block (IEND block). The end block is the last data block of the binary code of the video frame, and its structure is the same as the data block. The binary code of the portable network graphics format corresponding to each video frame can be extracted from the binary data stream according to the data stored in each byte in the binary data stream.

[0061] In this embodiment, Figure 5 1 is a flowchart of extracting binary encoding of video frames from a picture stream provided by an embodiment of the present application. Figure 5 The step of extracting binary encoding of video frames from the picture stream specifically includes S12011-S12013:

[0062] S12011. Verify the signature information in the image stream according to the byte arrangement order of the image stream.

[0063] S12012. After verifying the signature information in the picture stream, extract multiple data blocks and an end block from the picture stream, where the data block includes a block length, a block type, block data, and a check code.

[0064] S12013. After extracting the end block, determine the signature information, multiple data blocks and the end block as a binary code of a video frame.

[0065] Exemplarily, when the python program just starts to receive the binary data stream transmitted by FFmpeg, it reads information from the first eight bytes of the binary data stream and verifies whether the read information is a PNG signature. After verifying that the information is a PNG signature, the PNG signature is no longer verified before a video frame is read. According to the byte arrangement order of the binary data stream, 4 bytes of information are read downward to obtain the PNG block length, and then 4 bytes of information are read downward to obtain the PNG block type. According to the latest read PNG block length, the corresponding length byte information is read downward to obtain the PNG block data. For example, if the latest read PNG block length is 8, then after reading the PNG block type, 8 bytes of information are continued to be read downward to obtain the block data. Then 4 bytes of information are read downward to obtain the CRC (cyclic redundancy check) check code until the extraction of a data block is completed. The above-mentioned operation of extracting data blocks is repeated to extract multiple data blocks. When the end block is extracted, it indicates that the binary code extraction of a video frame is completed, and then the extracted PNG signature, multiple data blocks and end block are combined to form the binary code of a video frame. After extracting the binary code of a video frame, continue to extract the next video frame from the remaining binary data stream of the pipeline according to the same steps as above.

[0066] S1202: Convert the binary code of the video frame into a standard character code.

[0067] In this embodiment, the standard character encoding is the base64 encoding of the image. After Python extracts the binary encoding of the video frame from the binary data stream, the video frame is converted into the base64 encoding, so that the algorithm analysis application can directly perform algorithm analysis processing after obtaining the base64-encoded video frame of the image, thereby improving the efficiency of the algorithm analysis.

[0068] S130: Calling an algorithm analysis application to analyze and process the video frame to obtain an analysis result of the video frame.

[0069] For example, after the Python program converts the video frame into a base64 image encoding, the algorithm analysis application is directly called to analyze and process the video frame to obtain the analysis results of each video frame. The video frame is transmitted between various applications throughout the entire process from extraction to transmission to analysis, without writing the video frame to the hard disk and then reading the video frame from the hard disk, which greatly improves the efficiency of video frame extraction, transmission and analysis.

[0070] In one embodiment, a distributed message queue cluster is built between the cutting program and the algorithm analysis application, so that the cutting program is used as the producer of the distributed message queue cluster and the algorithm analysis application is used as the consumer of the distributed message queue cluster. For example, the cutting program can push the standard character encoding of the video frame to each category of the distributed message queue cluster. Figure 2 The distributed message queue cluster in the video frame extraction and analysis device uses a kafka cluster, which includes multiple kafka nodes. Multiple python programs in the frame extraction server cluster push the base64 encoding of the corresponding video frame to the kafka cluster. The kafka node receives the base64 encoding of the image pushed by the python program and writes the base64 encoding of the image into different partitions, and obtains the algorithm analysis application to consume the base64 encoding of the image in each kafka node. Furthermore, the algorithm analysis application obtains the video frame from the distributed message queue cluster and analyzes and processes the video frame. Reference Figure 2 The video frame analysis device is also equipped with an algorithm analysis application. Different algorithm analysis applications can subscribe to different partitions of the Kafka node to call all analysis services of each application to achieve diversified analysis and processing. Moreover, multiple algorithm analysis applications can run on different machines to monitor different partitions, which is conducive to improving the efficiency of video frame analysis. When the number of base64 encoded images generated by the python program is large, the base64 encoded images will be cached in the Kafka cluster, and then the speed of consuming video frames will be accelerated by increasing the number of consumers in the Kafka cluster, solving the problem of inconsistent processing speed of production and consumption messages.

[0071] In summary, the video frame extraction and analysis method provided by the embodiment of the present application extracts multiple video frames from the corresponding video stream based on the video stream address through the video tool, converts the multiple video frames into a picture stream, and transmits the picture stream to the cutting program through a preset pipeline; receives the picture stream from the preset pipeline through the cutting program, and cuts the picture stream into multiple video frames; uses the cutting program as the producer of the distributed message queue cluster, and uses the algorithm analysis application as the consumer of the distributed message queue cluster. The cutting program caches the video frame in the distributed message queue cluster, and the algorithm analysis application obtains the video frame from the distributed message queue cluster, and analyzes and processes the video frame to obtain the analysis result of the video frame. Through the above technical means, after the video tool extracts the video frame from the video stream, it transmits the video frame to the cutting program through the pipeline stream transmission mode between applications. The video frame does not need to be written to the hard disk in file format, which eliminates the process of reading the video frame from the hard disk, shortens the transmission time of the video frame, improves the efficiency of video frame extraction and video frame analysis, and does not need to clean the hard disk after analyzing the video frame, avoiding the disk loss caused by repeated addressing operations and extending the service life of the disk. In addition, decoupling the cutting program and algorithm analysis application through a distributed message queue cluster helps to dynamically adjust the number of nodes in the server cluster, balance the processing speed of producing and consuming video frames, avoid the system from completely crashing due to overloaded requests, and improve system stability.

[0072] Based on the above embodiments, Figure 6 A schematic diagram of the structure of a video frame extraction and analysis device provided in an embodiment of the present application. Figure 6 The video frame extraction and analysis device provided in this embodiment specifically includes: a pipeline transmission module 21, a video frame cutting module 22 and a video frame analysis module 23.

[0073] The pipeline transmission module is configured to extract multiple video frames from the corresponding video stream based on the video stream address through the video tool, convert the multiple video frames into a picture stream, and transmit the picture stream to the cutting program through a preset pipeline;

[0074] A video frame cutting module is configured to receive a picture stream from a preset pipeline through a cutting program and divide the picture stream into a plurality of video frames;

[0075] The video frame analysis module is configured to call an algorithm analysis application to analyze and process the video frame to obtain an analysis result of the video frame.

[0076] Based on the above embodiment, the pipeline transmission module includes: a video frame extraction unit, which is configured to configure the video stream address of the front-end camera, and extracts frames of the video stream corresponding to the video stream address through the video tool according to a preset streaming media protocol to obtain the video frame.

[0077] Based on the above embodiment, the pipeline transmission module includes: a first encoding unit, configured to convert the encoding of each video frame into a binary encoding in a portable network graphics format; a pipeline transmission unit, configured to place the binary encodings of multiple video frames in a preset pipeline to obtain a picture stream.

[0078] Based on the above embodiment, the video frame cutting module includes: a code extraction unit, configured to extract the binary code of the video frame from the picture stream according to the data format of the portable network image; and a second coding unit, configured to convert the binary code of the video frame into a standard character code.

[0079] On the basis of the above embodiment, the encoding extraction unit includes: a signature check subunit, configured to check the signature information in the picture stream according to the byte arrangement order of the picture stream; a data block extraction subunit, configured to extract multiple data blocks and an end block from the picture stream after checking the signature information in the picture stream, the data block including a block length, a block type, a block data and a check code; a combination subunit, configured to determine the signature information, multiple data blocks and the end block as a binary code of a video frame after extracting the end block.

[0080] Based on the above embodiment, the video frame cutting module also includes: a video frame pushing unit, which is configured to push the standard character encoding of multiple video frames to the distributed message queue cluster through a cutting program after dividing the image stream into multiple video frames.

[0081] On the basis of the above-mentioned embodiment, the video frame analysis module includes: a video frame subscription unit, which is configured to obtain multiple video frames from a distributed message queue cluster through multiple algorithm analysis applications, and analyze and process the multiple video frames.

[0082] As mentioned above, the video frame extraction and analysis device provided by the embodiment of the present application extracts multiple video frames from the corresponding video stream based on the video stream address through the video tool, converts the multiple video frames into a picture stream, and transmits the picture stream to the cutting program through a preset pipeline; receives the picture stream from the preset pipeline through the cutting program, and cuts the picture stream into multiple video frames; uses the cutting program as the producer of the distributed message queue cluster, and uses the algorithm analysis application as the consumer of the distributed message queue cluster. The cutting program caches the video frame in the distributed message queue cluster, and the algorithm analysis application obtains the video frame from the distributed message queue cluster, and analyzes and processes the video frame to obtain the analysis result of the video frame. Through the above technical means, after the video tool extracts the video frame from the video stream, it transmits the video frame to the cutting program through the pipeline stream transmission method between applications. The video frame does not need to be written to the hard disk in file format, which saves the process of reading the video frame from the hard disk, shortens the transmission time of the video frame, improves the efficiency of video frame extraction and video frame analysis, and does not need to clean the hard disk after analyzing the video frame, avoiding the disk loss caused by repeated addressing operations and extending the service life of the disk. In addition, decoupling the cutting program and algorithm analysis application through a distributed message queue cluster helps to dynamically adjust the number of nodes in the server cluster, balance the processing speed of producing and consuming video frames, avoid the system from completely crashing due to overloaded requests, and improve system stability.

[0083] The video frame analysis device provided in the embodiment of the present application can be used to execute the video frame analysis method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0084] Figure 7 is a schematic diagram of the structure of a video frame analysis device provided in an embodiment of the present application, with reference to Figure 7The video frame analysis device includes: a processor 31, a memory 32, a communication device 33, an input device 34 and an output device 35. The number of processors 31 in the video frame analysis device can be one or more, and the number of memories 32 in the video frame analysis device can be one or more. The processor 31, memory 32, communication device 33, input device 34 and output device 35 of the video frame analysis device can be connected through a bus or other methods.

[0085] The memory 32, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the video frame extraction analysis method of any embodiment of the present application (for example, the pipeline transmission module 21, the video frame cutting module 22 and the video frame analysis module 23 in the video frame extraction analysis device). The memory 32 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the device, etc. In addition, the memory 32 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0086] The communication device 33 is used for data transmission.

[0087] The processor 31 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 32, that is, implements the above-mentioned video frame extraction analysis method.

[0088] The input device 34 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 35 may include a display device such as a display screen.

[0089] The video frame analysis device provided above can be used to execute the video frame analysis method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0090] An embodiment of the present application also provides a storage medium containing computer executable instructions. When the computer executable instructions are executed by a computer processor, they are used to perform a video frame extraction and analysis method. The video frame extraction and analysis method includes: extracting multiple video frames from the corresponding video stream based on the video stream address through a video tool, converting the multiple video frames into a picture stream, and transmitting the picture stream to a cutting program through a preset pipeline; receiving the picture stream from the preset pipeline through the cutting program, and cutting the picture stream into multiple video frames; calling an algorithm analysis application to analyze and process the video frames to obtain analysis results of the video frames.

[0091] Storage medium - any of various types of memory devices or storage devices. The term "storage medium" is intended to include: installation media, such as CD-ROM, floppy disk or tape device; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory elements, etc. Storage media may also include other types of memory or combinations thereof. In addition, the storage medium may be located in the first computer system in which the program is executed, or may be located in a different second computer system, which is connected to the first computer system via a network (such as the Internet). The second computer system can provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media residing in different locations (for example, in different computer systems connected by a network). The storage medium may store program instructions (for example, embodied as a computer program) that can be executed by one or more processors.

[0092] Of course, the storage medium containing computer executable instructions provided in the embodiment of the present application is not limited to the above-mentioned video frame analysis method, and the computer executable instructions can also execute related operations in the video frame analysis method provided in any embodiment of the present application.

[0093] The video frame analysis device, video frame analysis system, storage medium and video frame analysis equipment provided in the above embodiments can execute the video frame analysis method provided in any embodiment of the present application. For technical details not described in detail in the above embodiments, please refer to the video frame analysis method provided in any embodiment of the present application.

[0094] The above are only preferred embodiments of the present application and the technical principles used. The present application is not limited to the specific embodiments herein, and various obvious changes, readjustments and substitutions that can be made by those skilled in the art will not deviate from the protection scope of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may also include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A video frame analysis method, characterized in that: include: Extracting multiple video frames from the corresponding video stream based on the video stream address through a video tool, converting the multiple video frames into a picture stream, and transmitting the picture stream to a cutting program through a preset pipeline, wherein converting the multiple video frames into a picture stream includes: converting the code of each video frame into a binary code in a portable network graphics format, and placing the binary codes of the multiple video frames in the preset pipeline to obtain the picture stream; Receiving the picture stream from the preset pipeline through the cutting program, and cutting the picture stream into a plurality of the video frames; An algorithm analysis application is called to analyze and process the video frame to obtain an analysis result of the video frame.

2. The video frame extraction analysis method according to claim 1, characterized in that: The extracting a plurality of video frames from the corresponding video stream based on the video stream address by the video tool includes: The video stream address of the front-end camera is configured, and the video tool extracts frames of the video stream corresponding to the video stream address according to a preset streaming media protocol to obtain the video frame.

3. The video frame extraction analysis method according to claim 1, characterized in that: The step of cutting the picture stream into a plurality of video frames comprises: Extracting the binary code of the video frame from the picture stream according to the data format of the portable network image; The binary code of the video frame is converted into a standard character code.

4. The video frame analysis method according to claim 1, characterized in that: The step of extracting the binary code of the video frame from the picture stream according to the data format of the portable network image comprises: Verifying the signature information in the picture stream according to the byte arrangement order of the picture stream; After verifying the signature information in the picture stream, extracting a plurality of data blocks and an end block from the picture stream, wherein the data block includes a block length, a block type, block data, and a check code; After the end block is extracted, the signature information, the plurality of data blocks and the end block are determined as a binary code of one video frame.

5. The video frame extraction analysis method according to claim 1, characterized in that: After dividing the picture stream into a plurality of video frames, the method further includes: The standard character encoding of the video frame is pushed to the distributed message queue cluster through the cutting program.

6. The video frame extraction analysis method according to claim 5, characterized in that: The calling algorithm analysis application to analyze and process the video frame includes: The algorithm analysis application obtains the video frame from the distributed message queue cluster, and analyzes and processes the video frame.

7. A video frame extraction and analysis device, characterized in that: include: The pipeline transmission module is configured to extract multiple video frames from the corresponding video stream based on the video stream address through the video tool, convert the multiple video frames into a picture stream, and transmit the picture stream to the cutting program through a preset pipeline. The pipeline transmission module is specifically configured to: convert the code of each video frame into a binary code in a portable network graphics format, and place the binary codes of the multiple video frames in the preset pipeline to obtain the picture stream; A video frame cutting module is configured to receive the picture stream from the preset pipeline through the cutting program, and cut the picture stream into a plurality of video frames; The video frame analysis module is configured to call an algorithm analysis application to analyze and process the video frame to obtain an analysis result of the video frame.

8. A video frame extraction and analysis device, characterized in that: include: one or more processors; A storage device stores one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the video frame analysis method as described in any one of claims 1-6.

9. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions are used to perform the video frame analysis method as described in any one of claims 1-6 when executed by a computer processor.

Images