Video inference method, device, electronic device and storage medium

Abstract

The present invention provides a video reasoning method, device, electronic device, and storage medium. The method includes: obtaining the total video reasoning throughput corresponding to videos of different resolutions; determining the maximum number of frames extracted for videos of different resolutions based on the total video reasoning throughput and the number of video frames corresponding to the videos of different resolutions; and creating a linear table based on the maximum number of frames extracted for videos of different resolutions; obtaining the resolution of the video to be reasoned; and determining the maximum number of frames extracted for the video to be reasoned from the linear table based on the resolution of the video to be reasoned; determining the number of video frames processed by an artificial intelligence model based on a preset application scenario coefficient and the maximum number of frames extracted for the video to be reasoned; extracting video frames from the video to be reasoned based on the number of video frames, and inputting the extracted video frames into the artificial intelligence model for reasoning. The present invention can efficiently utilize and use computing resources, thereby improving the efficiency of video reasoning.

Description

Technical Field

[0001] The present invention relates to the field of video processing technology, and in particular to a video inference method, device, electronic device and storage medium. Background Art

[0002] With the increasing popularity and development of artificial intelligence (AI), various vertical industries are demanding the application of computer vision analysis (multi-channel audio and video analysis) to their respective scenarios, thereby improving overall productivity and efficiency. However, transmitting video and audio data to cloud platforms for intelligent analysis currently suffers from numerous drawbacks, including high network bandwidth consumption, poor timeliness of AI analysis, large volumes of centralized audio and video data processing, and high cloud platform load and cost. Consequently, AI analysis based on edge and end-to-end computing presents a significant opportunity for development. Computing resources are the most valuable asset in edge and end-to-end computing, and efficient utilization of these resources is a pressing challenge for AI applications. Summary of the Invention

[0003] The present invention provides a video reasoning method, device, electronic device and storage medium for efficiently utilizing and using computing resources to improve the efficiency of video reasoning.

[0004] The present invention provides a video inference method, comprising:

[0005] Obtain the total video inference throughput corresponding to videos of different resolutions. Based on the total video inference throughput and video frame count corresponding to videos of different resolutions, determine the maximum number of frames extracted for videos of different resolutions. Create a linear table based on the maximum number of frames extracted for videos of different resolutions.

[0006] Obtaining the resolution of the video to be inferred, and determining the maximum number of frames corresponding to the video to be inferred from the linear table based on the resolution of the video to be inferred;

[0007] Determine the number of video frames processed by the artificial intelligence model based on a preset application scenario coefficient and a maximum number of frames corresponding to the video to be inferred;

[0008] Video frames are extracted from the video to be inferred based on the number of video frames, and the extracted video frames are input into the artificial intelligence model for inference.

[0009] According to the video inference method provided by the present invention, obtaining the total video inference throughput corresponding to videos of different resolutions includes:

[0010] Determine the computing power consumption of the artificial intelligence model for images with different resolutions, and determine the total video inference throughput corresponding to videos with different resolutions based on the computing power consumption and the total hardware computing power of the artificial intelligence model.

[0011] According to the video inference method provided by the present invention, determining the total video inference throughput corresponding to videos of different resolutions based on the computing power consumption and the total hardware computing power of the artificial intelligence model includes:

[0012] The total computing power of the hardware is divided by the computing power consumption corresponding to images of different resolutions to obtain the total video inference throughput corresponding to videos of different resolutions.

[0013] According to the video inference method provided by the present invention, determining the number of video frames processed by the artificial intelligence model based on a preset application scenario coefficient and the maximum number of frames corresponding to the video to be inferred includes:

[0014] Multiply the preset application scenario coefficient by the maximum number of frames corresponding to the video to be inferred to obtain the number of video frames processed by the artificial intelligence model.

[0015] According to the video inference method provided by the present invention, extracting video frames from the video to be inferred based on the number of video frames includes:

[0016] Based on the number of video frames, intra-frame coded frames, predictive coded frames and bidirectional predictive interpolation coded frames are sequentially extracted from the video to be inferred.

[0017] According to the video inference method provided by the present invention, the maximum number of frames extracted from each of the videos with different resolutions is obtained based on the following steps:

[0018] For multiple online videos of each resolution, multiply the number of online video channels by the video frame rate of a single online video channel to obtain the total video frame rate of each resolution video; wherein the video frame rates of different channels in the multiple online videos are the same;

[0019] Divide the total video inference throughput corresponding to each resolution video by the corresponding total video frame rate to obtain the maximum number of frames corresponding to each resolution video.

[0020] The present invention also provides a video reasoning device, comprising:

[0021] A tabulation module is used to obtain the total video inference throughput corresponding to videos of different resolutions, determine the maximum number of frames extracted based on the total video inference throughput and video frame number corresponding to videos of different resolutions, and create a linear table based on the maximum number of frames extracted for videos of different resolutions;

[0022] A first determination module is configured to obtain a resolution of the video to be inferred, and determine a maximum number of frames corresponding to the video to be inferred from the linear table based on the resolution of the video to be inferred;

[0023] A second determination module is used to determine the number of video frames processed by the artificial intelligence model based on a preset application scenario coefficient and a maximum number of frames corresponding to the video to be inferred;

[0024] An inference module is used to extract video frames from the video to be inferred based on the number of video frames, and input the extracted video frames into the artificial intelligence model for inference.

[0025] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the video inference method described above is implemented.

[0026] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described video inference methods.

[0027] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any one of the above-mentioned video reasoning methods.

[0028] The video inference method, device, electronic device and storage medium provided by the present invention determine the maximum frame extraction number of videos with different resolutions based on the total video inference throughput and video frame number corresponding to videos with different resolutions, and then determine the number of video frames processed by the artificial intelligence model in combination with the resolution of the video to be inferred and the preset application scenario coefficient. Based on the video frame number, video frames are extracted from the video to be inferred and then input into the artificial intelligence model for inference. The present invention comprehensively considers various factors, with the purpose of maximizing computing power utilization, analyzes as many video frames as possible to obtain better inference analysis results, efficiently utilizes and uses computing resources, and improves the efficiency of video inference. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0030] Figure 1 1 is a flow chart of the video inference method provided by the present invention;

[0031] Figure 2 Schematic diagram of the structure of the video reasoning device provided by the present invention;

[0032] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0033] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0034] The following combination Figure 1-Figure 3 The video reasoning method, device, electronic device and storage medium of the present invention are described.

[0035] like Figure 1 As shown, the present invention provides a video reasoning method, comprising:

[0036] Step 110: Obtain the total video inference throughput corresponding to videos of different resolutions. Based on the total video inference throughput and the number of video frames corresponding to videos of different resolutions, determine the maximum number of frames extracted for videos of different resolutions. Create a linear table based on the maximum number of frames extracted for videos of different resolutions.

[0037] It is understood that before executing step 110, the user can pre-set the high-efficiency mode or the high-performance mode, and the system selects the appropriate strategy to complete the video reasoning according to the set mode. When the user selects the high-efficiency mode, the method provided by the present invention begins to be executed.

[0038] The total video inference throughput corresponding to videos of different resolutions may be different. Therefore, the maximum number of frames corresponding to videos of different resolutions may also be different. The linear table contains the maximum number of frames corresponding to videos of different resolutions.

[0039] Step 120: Obtain the resolution of the video to be inferred, and based on the resolution of the video to be inferred, determine the maximum number of frames corresponding to the video to be inferred from the linear table.

[0040] It is understandable that the resolution of the video to be inferred is also the image resolution in the video to be inferred, and the resolutions of different frames of images in the video to be inferred are the same.

[0041] In the linear table, for example, the resolution of 1920*1080 corresponds to a maximum number of frames of 5; the resolution of 1024*768 corresponds to a maximum number of frames of 10; the resolution of 640*640 corresponds to a maximum number of frames of 20.

[0042] If the calculated maximum frame rate is less than 1, the AI ​​model cannot complete inference analysis of the video image within 1 second. In this case, the linear table can be used to calculate an appropriate image resolution. The AI ​​model's pre-processing module can then resize the video image to a maximum frame rate greater than 1. If the linear table does not provide an appropriate resolution, the AI ​​model's performance can be used for video analysis.

[0043] Step 130: Determine the number of video frames processed by the artificial intelligence model based on a preset application scenario coefficient and the maximum number of frames corresponding to the video to be inferred.

[0044] It's understandable that the application scenario coefficient is user-configured. For example, in the geo-fence application scenario, since it primarily detects intruders, it doesn't require processing a large amount of frame data. Based on actual testing experience, it's set to 0.5. The more data the AI ​​model needs to process, the larger the application scenario coefficient.

[0045] The artificial intelligence model can be a neural network model that performs inference on video frames.

[0046] Step 140: extract video frames from the video to be inferred based on the number of video frames, and input the extracted video frames into the artificial intelligence model for inference.

[0047] It is understood that video frames may include intra-frame coded frames, forward-predictive coded frames, and bidirectionally predicted interpolated coded frames. The intra-frame coded frames, forward-predictive coded frames, and bidirectionally predicted interpolated coded frames are respectively input into the artificial intelligence model for inference to obtain the inference results of the artificial intelligence model.

[0048] In video compression, each frame represents a still image. Various algorithms are used to reduce data capacity during actual compression, with IPB frames being the most common.

[0049] Intra-frame coded frames (I-frames) are typically the first frame in each GOP (Group of Pictures, a video compression technique used by the Moving Picture Experts Group). Moderately compressed, they serve as a reference point for random access and can be considered a static image. An I-frame can be considered the product of image compression, achieving a 6:1 compression ratio without any perceptible blur. I-frame compression removes spatial redundancy in the video, while the subsequent P-frames and B-frames are designed to remove temporal redundancy.

[0050] A predictive-coded frame (P-frame) is a coded image that reduces the amount of data transmitted by removing the temporal redundancy of previously coded frames in the image sequence. It is also called a predicted frame. A P-frame represents the difference between this frame and the previous I-frame (or P-frame). During decoding, the difference defined by this frame is superimposed on the previously cached image to generate the final image, known as the difference frame. A P-frame does not contain complete image data, only the difference between the previous frame and the image.

[0051] A bidirectionally interpolated prediction frame (B frame) uses the preceding I or P frame and the following P frame as reference frames to calculate a predicted value and two motion vectors for a specific point in the B frame. The difference between the predicted values ​​and the motion vectors is then transmitted. The receiver calculates the predicted value based on the motion vectors in the two reference frames and sums the difference with the predicted value to obtain the sample value for the specific point in the B frame, thus obtaining the complete B frame. In other words, decoding a B frame requires not only obtaining the previous cached image but also decoding the subsequent image. The final image is obtained by superimposing the preceding and subsequent images with the current frame data. B frames offer high compression rates, but decoding them places a higher CPU load.

[0052] AI model reasoning, a deep learning-based AI, consists of three elements: training, reasoning, and datasets. Reasoning involves applying trained AI models to specific scenarios. Reasoning relies on algorithm models, reasoning frameworks, and hardware reasoning platforms to perform AI analysis and judgment on real-time video data. It is a key step in AI algorithm engineering. Common reasoning frameworks include openvivo, tensorRT, mediapipe, paddleLite, and tengine. These frameworks primarily help users adapt algorithm models to hardware computing platforms.

[0053] AI computing power is the unit of processor computing power (OPS, Operations Per Second): 1TOPS (Tera): 10^12 operations per second; 1GOPS (Giga): 10^9 operations per second; 1MOPS (Million): 10^6 operations per second.

[0054] The computing power estimation of the artificial intelligence model is to calculate the AI ​​computing power required by the model based on the algorithm model and the image format that the model needs to process. Taking pytorch as an example, the framework provides the torchstat tool, which can estimate the computing power of the artificial intelligence model.

[0055] In some embodiments, obtaining the total video inference throughput corresponding to videos of different resolutions includes:

[0056] Determine the computing power consumption of the artificial intelligence model for images with different resolutions, and determine the total video inference throughput corresponding to videos with different resolutions based on the computing power consumption and the total hardware computing power of the artificial intelligence model.

[0057] It's understandable that total hardware computing power refers to the AI ​​computing power provided by the hardware platform running the AI ​​model. Different image resolutions consume different amounts of computing power. Therefore, the corresponding total video inference throughput must be determined based on the computing power consumed by images of different resolutions and the total hardware computing power.

[0058] The computing power consumption of the artificial intelligence model for images of different resolutions, that is, the computing power consumption of the artificial intelligence model for a single image, is different for images of different resolutions.

[0059] In some embodiments, determining the total video inference throughput corresponding to videos of different resolutions based on the computing power consumption and the total hardware computing power of the artificial intelligence model includes:

[0060] The total computing power of the hardware is divided by the computing power consumption corresponding to images of different resolutions to obtain the total video inference throughput corresponding to videos of different resolutions.

[0061] It is understandable that the video resolutions processed by the artificial intelligence model will be different. Obtain the resolutions of all videos that currently need to be processed, divide the total hardware computing power by the computing power consumption corresponding to each resolution, and obtain the total video inference throughput corresponding to each resolution video.

[0062] In some embodiments, determining the number of video frames to be processed by the artificial intelligence model based on a preset application scenario coefficient and a maximum number of frames corresponding to the video to be inferred includes:

[0063] Multiply the preset application scenario coefficient by the maximum number of frames corresponding to the video to be inferred to obtain the number of video frames processed by the artificial intelligence model.

[0064] It can be understood that when the resolution of the video input to the artificial intelligence model changes, the maximum number of frames corresponding to the video to be inferred is determined from the linear table based on the resolution of the video to be inferred, and the preset application scenario coefficient is multiplied by the maximum number of frames corresponding to the video to be inferred to obtain the number of video frames processed by the artificial intelligence model.

[0065] In some embodiments, extracting video frames from the video to be inferred based on the number of video frames includes:

[0066] Based on the number of video frames, intra-frame coded frames, predictive coded frames and bidirectional predictive interpolation coded frames are sequentially extracted from the video to be inferred.

[0067] It is understandable that the intra-frame coding frame has the smallest loss, so the effect of inputting it into the artificial intelligence model for inference is the best, the effect of predictive coding frame for inference is second, and the effect of bidirectional predictive interpolation coding frame for inference is the worst.

[0068] The total number of intra-frame coded frames, predictive coded frames and bidirectional predictive interpolation coded frames finally extracted is the number of video frames.

[0069] In some embodiments, the maximum number of frames extracted for each of the videos with different resolutions is obtained based on the following steps:

[0070] For multiple online videos of each resolution, multiply the number of online video channels by the video frame rate of a single online video channel to obtain the total video frame rate of each resolution video; wherein the video frame rates of different channels in the multiple online videos are the same;

[0071] Divide the total video inference throughput corresponding to each resolution video by the corresponding total video frame rate to obtain the maximum number of frames corresponding to each resolution video.

[0072] It can be understood that the premise for calculating each resolution video based on the method provided in this embodiment is that the resolution of each video channel in each resolution video is the same, and the video frame rate of each video channel is the same.

[0073] For the number of online video channels, for example, if the hardware supports 8-channel video but 4-channel video is actually connected, the number of online video channels is 4.

[0074] In other embodiments, the user needs to pre-set the high-efficiency mode or high-performance mode. The system will select the appropriate strategy to complete video inference based on the configuration. For example, in the high-efficiency mode, the system will first estimate the computing power consumption of the artificial intelligence model for an image, and then calculate the total video inference throughput based on the total hardware computing power. The maximum number of frames extracted for each video channel is calculated based on the actual number of video channels connected and the frame rate of each video channel. The number of video frames that need to be processed by the artificial intelligence model is then calculated by multiplying the application scenario coefficient (configured by the user) by the maximum number of frames extracted. At the same time, the system will make a linear table of the maximum number of frames obtained under videos of different resolutions. In this way, when the resolution of the on-site input video changes, the maximum number of frames extracted is obtained according to the linear table. Finally, the corresponding number of video frames are extracted from the video in the order of I frame, P frame, and B frame for inference analysis. The only difference in the high-performance mode is that the application scenario coefficient is always 1, which meets the maximum demand for video frame inference analysis.

[0075] To sum up, the video inference method provided by the present invention includes: obtaining the total video inference throughput corresponding to videos of different resolutions, determining the maximum frame extraction number of videos of different resolutions based on the total video inference throughput and video frame number corresponding to videos of different resolutions, and making a linear table based on the maximum frame extraction number of videos of different resolutions; obtaining the resolution of the video to be inferred, and determining the maximum frame extraction number corresponding to the video to be inferred from the linear table based on the resolution of the video to be inferred; determining the number of video frames processed by the artificial intelligence model based on a preset application scenario coefficient and the maximum frame extraction number corresponding to the video to be inferred; extracting video frames from the video to be inferred based on the video frame number, and inputting the extracted video frames into the artificial intelligence model for inference.

[0076] In the video inference method provided by the present invention, the maximum number of frames extracted for videos with different resolutions is determined based on the total video inference throughput and the number of video frames corresponding to videos with different resolutions. Then, the number of video frames processed by the artificial intelligence model is determined in combination with the resolution of the video to be inferred and the preset application scenario coefficient. Based on the number of video frames, video frames are extracted from the video to be inferred and then input into the artificial intelligence model for inference. The present invention comprehensively considers various factors, with the purpose of maximizing computing power utilization, analyzes as many video frames as possible, obtains better inference analysis results, efficiently utilizes and uses computing resources, and improves the efficiency of video inference.

[0077] The video reasoning device provided by the present invention is described below. The video reasoning device described below and the video reasoning method described above can be referenced to each other.

[0078] like Figure 2 As shown, the present invention further provides a video inference device 200, comprising:

[0079] Tabulation module 210 is used to obtain the total video inference throughput corresponding to videos of different resolutions, determine the maximum number of frames extracted for videos of different resolutions based on the total video inference throughput and the number of video frames corresponding to the videos of different resolutions, and create a linear table based on the maximum number of frames extracted for videos of different resolutions;

[0080] A first determination module 220 is configured to obtain a resolution of the video to be inferred, and determine a maximum number of frames corresponding to the video to be inferred from the linear table based on the resolution of the video to be inferred;

[0081] A second determination module 230 is configured to determine the number of video frames to be processed by the artificial intelligence model based on a preset application scenario coefficient and a maximum number of frames corresponding to the video to be inferred;

[0082] The inference module 240 is used to extract video frames from the video to be inferred based on the number of video frames, and input the extracted video frames into the artificial intelligence model for inference.

[0083] The electronic device, computer program product, and storage medium provided by the present invention are described below. The electronic device, computer program product, and storage medium described below can be referenced to the video reasoning method described above.

[0084] Figure 3 An example of a physical structure diagram of an electronic device is shown below. Figure 3 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communication interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 may call logic instructions in the memory 330 to execute a video inference method, which includes:

[0085] Obtain the total video inference throughput corresponding to videos of different resolutions. Based on the total video inference throughput and video frame count corresponding to videos of different resolutions, determine the maximum number of frames extracted for videos of different resolutions. Create a linear table based on the maximum number of frames extracted for videos of different resolutions.

[0086] Obtaining the resolution of the video to be inferred, and determining the maximum number of frames corresponding to the video to be inferred from the linear table based on the resolution of the video to be inferred;

[0087] Determine the number of video frames processed by the artificial intelligence model based on a preset application scenario coefficient and a maximum number of frames corresponding to the video to be inferred;

[0088] Video frames are extracted from the video to be inferred based on the number of video frames, and the extracted video frames are input into the artificial intelligence model for inference.

[0089] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0090] In another aspect, the present invention further provides a computer program product, comprising a computer program. The computer program may be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the video inference method provided by each of the above methods, including:

[0091] Obtain the total video inference throughput corresponding to videos of different resolutions. Based on the total video inference throughput and video frame count corresponding to videos of different resolutions, determine the maximum number of frames extracted for videos of different resolutions. Create a linear table based on the maximum number of frames extracted for videos of different resolutions.

[0092] Obtaining the resolution of the video to be inferred, and determining the maximum number of frames corresponding to the video to be inferred from the linear table based on the resolution of the video to be inferred;

[0093] Determine the number of video frames processed by the artificial intelligence model based on a preset application scenario coefficient and a maximum number of frames corresponding to the video to be inferred;

[0094] Video frames are extracted from the video to be inferred based on the number of video frames, and the extracted video frames are input into the artificial intelligence model for inference.

[0095] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the video inference method provided by each of the above methods is implemented. The method includes:

[0096] Obtain the total video inference throughput corresponding to videos of different resolutions. Based on the total video inference throughput and video frame count corresponding to videos of different resolutions, determine the maximum number of frames extracted for videos of different resolutions. Create a linear table based on the maximum number of frames extracted for videos of different resolutions.

[0097] Obtaining the resolution of the video to be inferred, and determining the maximum number of frames corresponding to the video to be inferred from the linear table based on the resolution of the video to be inferred;

[0098] Determine the number of video frames processed by the artificial intelligence model based on a preset application scenario coefficient and a maximum number of frames corresponding to the video to be inferred;

[0099] Video frames are extracted from the video to be inferred based on the number of video frames, and the extracted video frames are input into the artificial intelligence model for inference.

[0100] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0101] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A video inference method, characterized in that: include: Obtain the total video inference throughput corresponding to videos of different resolutions. Based on the total video inference throughput and video frame count corresponding to videos of different resolutions, determine the maximum number of frames extracted for videos of different resolutions. Create a linear table based on the maximum number of frames extracted for videos of different resolutions. Obtaining the resolution of the video to be inferred, and determining the maximum number of frames corresponding to the video to be inferred from the linear table based on the resolution of the video to be inferred; Determine the number of video frames processed by the artificial intelligence model based on a preset application scenario coefficient and a maximum number of frames corresponding to the video to be inferred; Extracting video frames from the video to be inferred based on the number of video frames, and inputting the extracted video frames into the artificial intelligence model for inference; Obtaining the total video inference throughput corresponding to videos of different resolutions includes: Determine the computing power consumption of the artificial intelligence model for images with different resolutions, and determine the total video inference throughput corresponding to videos with different resolutions based on the computing power consumption and the total hardware computing power of the artificial intelligence model; Determining the total video inference throughput corresponding to videos of different resolutions based on the computing power consumption and the total hardware computing power of the artificial intelligence model includes: The total computing power of the hardware is divided by the computing power consumption corresponding to images of different resolutions to obtain the total video inference throughput corresponding to videos of different resolutions.

2. The video inference method according to claim 1, characterized in that The determining the number of video frames processed by the artificial intelligence model based on a preset application scenario coefficient and a maximum number of frames corresponding to the video to be inferred includes: Multiply the preset application scenario coefficient by the maximum number of frames corresponding to the video to be inferred to obtain the number of video frames processed by the artificial intelligence model.

3. The video inference method according to claim 1, characterized in that The extracting video frames from the video to be inferred based on the number of video frames includes: Based on the number of video frames, intra-frame coded frames, predictive coded frames and bidirectional predictive interpolation coded frames are sequentially extracted from the video to be inferred.

4. The video inference method according to any one of claims 1 to 3, characterized in that: The maximum number of frames extracted for each of the videos with different resolutions is obtained based on the following steps: For multiple online videos of each resolution, multiply the number of online video channels by the video frame rate of a single online video channel to obtain the total video frame rate of each resolution video; wherein the video frame rates of different channels in the multiple online videos are the same; Divide the total video inference throughput corresponding to each resolution video by the corresponding total video frame rate to obtain the maximum number of frames corresponding to each resolution video.

5. A video reasoning device, characterized in that: include: A tabulation module is used to obtain the total video inference throughput corresponding to videos of different resolutions, determine the maximum number of frames extracted for videos of different resolutions based on the total video inference throughput and video frame number corresponding to videos of different resolutions, and create a linear table based on the maximum number of frames extracted for videos of different resolutions; A first determination module is configured to obtain a resolution of the video to be inferred, and determine a maximum number of frames corresponding to the video to be inferred from the linear table based on the resolution of the video to be inferred; A second determination module is used to determine the number of video frames processed by the artificial intelligence model based on a preset application scenario coefficient and a maximum number of frames corresponding to the video to be inferred; An inference module, configured to extract video frames from the video to be inferred based on the number of video frames, and input the extracted video frames into the artificial intelligence model for inference; The tabulation module is specifically used for: Determine the computing power consumption of the artificial intelligence model for images with different resolutions, and determine the total video inference throughput corresponding to videos with different resolutions based on the computing power consumption and the total hardware computing power of the artificial intelligence model; The tabulation module is further specifically used for: The total computing power of the hardware is divided by the computing power consumption corresponding to images of different resolutions to obtain the total video inference throughput corresponding to videos of different resolutions.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the video inference method according to any one of claims 1 to 4 is implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the video reasoning method according to any one of claims 1 to 4 is implemented.

8. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the video reasoning method according to any one of claims 1 to 4 is implemented.

Images