Video coding and decoding method, device and equipment based on GPGPU (General Purpose Graphics Processing Unit) and medium
By adopting GPGPU-based parallel computing and deep learning algorithm methods in video encoding and decoding, the problem of video quality degradation when traditional video encoding and decoding is solved when processing complex video content and adapting to network bandwidth fluctuations is achieved, and high-quality video encoding and decoding and optimized user experience is achieved.
Patent Information
- Application Number
- CN202510023153.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-07
- Publication Date
- 2025-04-08
AI Technical Summary
Traditional video encoding and decoding methods seem to be ineffective when dealing with complex and changeable video content, dealing with network bandwidth fluctuations, and adapting to diversified terminal devices, resulting in a decline in video quality.
The video encoding and decoding method based on GPGPU is adopted, and the parallel computing power of GPGPU combined with deep learning algorithms is used to realize intelligent control and optimization of the video encoding and decoding process. The specific steps include: using GPGPU to process video image frames, perform feature analysis based on deep learning models, integrate network environment, terminal equipment performance and video content analysis results, determine the encoding and decoding strategy, allocate GPGPU computing resources, and finally perform video encoding and decoding processing.
By efficiently utilizing the parallel computing power and deep learning algorithms of GPGPU, deep intelligent control and optimization of the video encoding and decoding process is achieved, significantly improving the quality of video encoding and decoding, and optimizing the user experience.
Smart Images
Figure CN119788869A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of video processing technology, and in particular to a GPGPU-based video encoding and decoding method, device, equipment and medium. Background Art
[0002] With the rapid growth of video content and the popularization of HD and UHD video technology, video coding and decoding technology is facing unprecedented challenges. Traditional coding and decoding methods are unable to cope with complex and changing video content, network bandwidth fluctuations, and adapt to diverse terminal devices, which often leads to a decline in video quality. Therefore, how to improve the quality of video coding and decoding is an urgent problem to be solved. Summary of the invention
[0003] In view of this, the purpose of the present invention is to provide a GPGPU-based video encoding and decoding method, device, equipment and medium, which can realize intelligent control and optimization of the video encoding and decoding process by utilizing the parallel computing capability of GPGPU combined with deep learning algorithms to improve the quality of video encoding and decoding. The specific scheme is as follows:
[0004] In a first aspect, the present application provides a GPGPU-based video encoding and decoding method, comprising:
[0005] Using GPGPU to process the video to be coded and decoded to obtain a target video image frame, and performing feature analysis on the target video image frame based on a preset deep learning model to obtain a video content analysis result corresponding to the video to be coded and decoded;
[0006] Input the current network environment, user terminal device performance information and the video content analysis results into the preset deep learning model to obtain the current encoding and decoding strategy and the current GPGPU allocatable computing resources;
[0007] Based on the current encoding and decoding strategy and the current GPGPU allocatable computing resources, video encoding and decoding processing is performed on the video to be encoded and decoded to obtain a target video.
[0008] Optionally, the using GPGPU to process the video to be coded to obtain a target video image frame includes:
[0009] The video to be encoded and decoded is segmented by using GPGPU to obtain corresponding initial video image frames, and each of the initial video image frames is normalized to obtain the target video image frame.
[0010] Optionally, the performing feature analysis on the target video image frame based on a preset deep learning model to obtain a video content analysis result corresponding to the video to be encoded and decoded includes:
[0011] Performing feature extraction on the target video image frame based on a preset neural network model in the preset deep learning model to obtain video content features;
[0012] The video content features are input into the classifier in the preset deep learning model for feature classification processing to obtain the video content analysis result corresponding to the video to be encoded and decoded.
[0013] Optionally, the extracting features of the target video image frame based on a preset neural network model in the preset deep learning model to obtain video content features includes:
[0014] Performing feature extraction on the target video image frame based on a preset convolutional neural network model in the preset deep learning model to obtain a feature image;
[0015] The feature image is input into a preset recurrent neural network model in the preset deep learning model, so that the preset recurrent neural network model processes the feature image based on a time axis to obtain video content features.
[0016] Optionally, the classifier is a classifier obtained by training the classification performance of an initial classifier in the preset deep learning model using historical data; and the historical data is video data that has been labeled based on preset classification labels.
[0017] Optionally, the step of inputting the current network environment, the user terminal device performance information and the video content analysis result into the preset deep learning model to obtain the current encoding and decoding strategy and the current GPGPU allocatable computing resources includes:
[0018] Acquire the current network environment and user terminal device performance information, and integrate the current network environment, the user terminal device performance information and the video content analysis result to obtain a state vector;
[0019] The state vector is input into the preset deep learning model to obtain the current encoding and decoding strategy and the current GPGPU allocatable computing resources.
[0020] Optionally, after performing video encoding and decoding processing on the video to be encoded and decoded based on the current encoding and decoding strategy and the current GPGPU allocatable computing resources, the method further includes:
[0021] Based on a preset real-time monitoring framework, the video encoding and decoding process of the video to be encoded and decoded is monitored in real time to obtain actual encoding and decoding performance data, and the actual encoding and decoding performance data is input into the preset deep learning model for feedback learning by using a preset feedback mechanism to obtain an updated preset deep learning model;
[0022] A new current encoding and decoding strategy and a new current GPGPU allocatable computing resources are generated based on the updated preset deep learning model, and the updated preset deep learning model is determined as the new preset deep learning model, and then the process jumps to the step of performing video encoding and decoding processing on the video to be encoded and decoded based on the current encoding and decoding strategy and the current GPGPU allocatable computing resources until the video encoding and decoding processing operation of the video to be encoded and decoded is completed.
[0023] In a second aspect, the present application provides a GPGPU-based video encoding and decoding device, comprising:
[0024] A video analysis module is used to process the video to be coded and decoded using GPGPU to obtain a target video image frame, and perform feature analysis on the target video image frame based on a preset deep learning model to obtain a video content analysis result corresponding to the video to be coded and decoded;
[0025] An information determination module, used to input the current network environment, user terminal device performance information and the video content analysis result into the preset deep learning model to obtain the current encoding and decoding strategy and the current GPGPU allocatable computing resources;
[0026] The target video acquisition module is used to perform video encoding and decoding processing on the video to be encoded and decoded based on the current encoding and decoding strategy and the current GPGPU allocatable computing resources to obtain the target video.
[0027] In a third aspect, the present application provides an electronic device, including:
[0028] Memory, used to store computer programs;
[0029] The processor is used to execute the computer program to implement the aforementioned GPGPU-based video encoding and decoding method.
[0030] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned GPGPU-based video encoding and decoding method is implemented.
[0031] In the present application, the GPGPU is first used to process the video to be encoded and decoded to obtain a target video image frame, and the target video image frame is feature analyzed based on a preset deep learning model to obtain a video content analysis result corresponding to the video to be encoded and decoded; the current network environment, user terminal device performance information and the video content analysis result are input into the preset deep learning model to obtain the current encoding and decoding strategy and the current GPGPU allocatable computing resources; based on the current encoding and decoding strategy and the current GPGPU allocatable computing resources, the video to be encoded and decoded is subjected to video encoding and decoding processing to obtain the target video. As can be seen from the above, before encoding and decoding the target video, the present application first uses GPGPU and a deep learning model to analyze the video to be encoded and decoded to obtain the corresponding video content analysis result, and then uses the deep learning model to determine the current encoding and decoding strategy based on the video content analysis result and the current network information and the user terminal device performance information, and at the same time allocates the current GPGPU computing resources, and finally encodes and decodes the video to be encoded and decoded based on the current encoding and decoding strategy and the GPGPU computing resources. In this way, by efficiently utilizing the parallel computing power of GPGPU combined with deep learning algorithms, deep intelligent control and optimization of the video encoding and decoding process are achieved, thereby significantly improving the quality of video encoding and decoding and optimizing the user experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0033] Figure 1 A flow chart of a GPGPU-based video encoding and decoding method disclosed in this application;
[0034] Figure 2 A flowchart of a specific GPGPU-based video encoding and decoding method disclosed in this application;
[0035] Figure 3 A flow chart of a GPGPU-based video encoding and decoding method disclosed in this application;
[0036] Figure 4 A schematic diagram of the structure of a GPGPU-based video encoding and decoding device disclosed in this application;
[0037] Figure 5 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION
[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0039] With the rapid growth of video content and the popularization of high-definition and ultra-high-definition video technology, video coding and decoding technology is facing unprecedented challenges. Traditional coding and decoding methods are unable to cope with complex and changing video content, cope with network bandwidth fluctuations, and adapt to diverse terminal devices, which often leads to a decline in video quality. Therefore, this application will specifically introduce a video coding and decoding method that can solve the above problems.
[0040] See also Figure 1 As shown, the embodiment of the present invention discloses a GPGPU-based video encoding and decoding method, which may include:
[0041] Step S11: Use GPGPU to process the video to be encoded and decoded to obtain a target video image frame, and perform feature analysis on the target video image frame based on a preset deep learning model to obtain a video content analysis result corresponding to the video to be encoded and decoded.
[0042] In this embodiment, before using GPGPU (General-Purpose computing on Graphics Processing Units) to process the encoded and decoded video, the system is first initialized. Specifically, the system is started and the preset deep learning model, encoding and decoding algorithm and GPGPU driver are loaded to ensure the normal operation of each component.
[0043] In this embodiment, in order to ensure that the pixel value range of the target video image frame is suitable for the input of the preset deep learning model, the use of GPGPU to process the video to be coded to obtain the target video image frame may include: using GPGPU to segment the video to be coded to obtain the corresponding initial video image frame, and normalizing each of the initial video image frames to obtain the target video image frame. That is, in this embodiment, the parallel computing capability of GPGPU is first used to segment the video to be coded into separate image frames to obtain a corresponding series of separate initial video image frames, and then each of the separate initial video image frames is normalized to obtain the target video image frame. It should be noted that the normalization process can adjust the pixel value of the corresponding separate initial video image data in each of the separate initial video image frames to a uniform range, and the range can be between 0 and 1, or between -1 and 1. In this way, the brightness and contrast inconsistency problems caused by factors such as parameter differences between different initial video image frames are eliminated, so that the target video image frame obtained after processing is more suitable for subsequent preset deep learning model input. In addition, image data enhancement technology can be used to increase the generalization ability of the preset deep learning model. The image data enhancement technology includes but is not limited to rotating, scaling, and cropping the initial video image frame.
[0044] In this embodiment, in order to obtain the video content analysis result corresponding to the video to be encoded and decoded, it is necessary to use the preset neural network model in the preset deep learning model to extract features of the target video image frame to obtain video content features, and then obtain the video content analysis result; it should be pointed out that the preset neural network model may include a convolutional neural network model, a recurrent neural network model, a feedforward neural network model, a generative adversarial network model, etc.
[0045] Step S12: input the current network environment, user terminal device performance information and the video content analysis results into the preset deep learning model to obtain the current encoding and decoding strategy and the current GPGPU allocatable computing resources.
[0046] In this embodiment, the current network environment and user terminal device performance information are obtained, and the current network environment, the user terminal device performance information and the video content analysis results are integrated to obtain a state vector; the state vector is input into the preset deep learning model to obtain the current codec strategy and the current GPGPU allocatable computing resources. Among them, the current network environment includes the current bandwidth, the current packet loss rate, the current network delay, etc., the user terminal device performance information includes the CPU (Central Processing Unit, i.e., central processing unit) load, GPU (Graphics Processing Unit, i.e., graphics processor) capability, screen resolution, etc. of the user terminal device, and the video content analysis results include but are not limited to scene complexity and motion intensity. It can be understood that the preset deep learning model outputs the current codec strategy or the current codec parameter adjustment suggestion according to the state vector, and the current encoding parameters may include quantization step size, encoding mode and frame rate. The process of selecting the current codec strategy involves multi-objective optimization, such as balancing video quality and transmission efficiency to ensure the best viewing experience under limited resources. At the same time, according to the complexity and real-time requirements of the video content to be encoded, the current GPGPU computing resources are allocated on different time scales and spatial scales to obtain the current GPGPU allocatable computing resources. In this way, efficient resource utilization and load balancing are achieved through the allocation of GPGPU computing resources. It should be noted that in the process of allocating GPGPU computing resources, this embodiment supports multi-task parallel processing to improve the overall throughput and response speed of the system.
[0047] Step S13: performing video encoding and decoding processing on the video to be encoded and decoded based on the current encoding and decoding strategy and the current GPGPU allocatable computing resources to obtain a target video.
[0048] In this embodiment, the current encoding and decoding strategy and the current GPGPU allocatable computing resources are used to perform video encoding and decoding processing on the video to be encoded and decoded, aiming to generate a target video that meets the expected quality standards and format requirements. In a specific implementation, based on the HEVC (High Efficiency Video Coding, i.e. H.265, high-efficiency video coding) encoding and decoding strategy and Tesla GPGPU (Tesla General-Purpose Graphics Processing Unit, i.e. Tesla general-purpose image processing unit) allocatable computing resources, a video to be encoded and decoded with a step of 4K resolution and 60 frames per second is encoded and decoded. During the encoding and decoding process, the floating-point computing power and parallel processing power of Tesla GPGPU are used, while the HEVC encoding and decoding strategy is followed to optimize the encoding and decoding efficiency of the video to be encoded and decoded. In this way, the initial 4K video file is significantly compressed, while maintaining high-definition picture quality and smooth playback effects, and finally the target video is generated. This process not only improves the efficiency of video processing, but also ensures the quality of the target video. In this embodiment, in order to complete the adjustment of the current encoding and decoding strategy and the current GPGPU allocatable computing resources, and complete the video encoding and decoding processing of the video to be encoded and decoded, after the video encoding and decoding processing is performed on the video to be encoded and decoded based on the current encoding and decoding strategy and the current GPGPU allocatable computing resources, it also includes: based on a preset real-time monitoring framework, the video encoding and decoding process of the video to be encoded and decoded is monitored in real time to obtain actual encoding and decoding performance data, and the actual encoding and decoding performance data is input into the preset deep learning model for feedback learning by using a preset feedback mechanism to obtain an updated preset deep learning model; based on the updated preset deep learning model, a new current encoding and decoding strategy and a new current GPGPU allocatable computing resources are generated, and the updated preset deep learning model is determined as a new preset deep learning model, and then the step of performing video encoding and decoding processing on the video to be encoded and decoded based on the current encoding and decoding strategy and the current GPGPU allocatable computing resources is jumped to, until the video encoding and decoding processing operation of the video to be encoded and decoded is completed.
[0049] It can be understood that this embodiment pre-designs a real-time monitoring framework that can capture key performance indicators such as encoding rate, decoding delay, resource utilization, etc. during the encoding and decoding process, and based on the real-time monitoring framework, the video encoding and decoding process of the video to be encoded and decoded is monitored in real time to obtain actual encoding and decoding performance data such as actual packet loss rate, delay, buffer time, etc., and then the actual encoding and decoding performance data is input into the preset deep learning model through a preset feedback mechanism. It should be pointed out that the preset feedback mechanism can perform feedback learning on the preset deep learning model according to the actual encoding and decoding performance data to obtain an updated preset deep learning model, and generate a new current encoding and decoding strategy and a new current GPGPU allocatable computing resource based on the updated preset deep learning model. Furthermore, the preset feedback mechanism has real-time response capability, and can adjust the current encoding and decoding strategy in time according to the fluctuation of the current network environment and the changes in the performance of the user terminal device to ensure the smoothness and stability of video transmission, for example, when the bandwidth is reduced, the resolution or frame rate of the video to be encoded and decoded is reduced, when the network condition improves, the quality of the video to be encoded and decoded is improved, and the current encoding and decoding complexity is adjusted according to the changes in the performance of the user terminal device to avoid overload. Next, the updated preset deep learning model is determined as a new preset deep learning model, and then the video to be encoded and decoded is continued to be processed based on the current encoding and decoding strategy and the current GPGPU allocatable computing resources, and the above process is repeated until the video encoding and decoding processing operation of the video to be encoded and decoded is completed.
[0050] In this way, this embodiment uses a preset feedback mechanism to prompt the preset deep learning model to adjust the current encoding and decoding strategy and the current GPGPU allocatable computing resources according to the actual encoding and decoding performance data, and continuously optimizes the current encoding and decoding strategy and the current GPGPU allocatable computing resources through iterative learning, thereby adapting to the ever-changing current network environment and user terminal device performance.
[0051] As can be seen from the above, in the present application, GPGPU is first used to process the video to be encoded and decoded to obtain a target video image frame, and feature analysis is performed on the target video image frame based on a preset deep learning model to obtain a video content analysis result corresponding to the video to be encoded and decoded; the current network environment, user terminal device performance information and the video content analysis result are input into the preset deep learning model to obtain the current encoding and decoding strategy and the current GPGPU allocatable computing resources; based on the current encoding and decoding strategy and the current GPGPU allocatable computing resources, video encoding and decoding processing is performed on the video to be encoded and decoded to obtain the target video. As can be seen from the above, before encoding and decoding the target video, the present application first uses GPGPU and a deep learning model to analyze the video to be encoded and decoded to obtain the corresponding video content analysis result, and then uses the deep learning model to determine the current encoding and decoding strategy based on the video content analysis result and the current network information and the user terminal device performance information, and at the same time allocates the current GPGPU computing resources, and finally encodes and decodes the video to be encoded and decoded based on the current encoding and decoding strategy and the GPGPU computing resources. In this way, by efficiently utilizing the parallel computing power of GPGPU combined with deep learning algorithms, deep intelligent control and optimization of the video encoding and decoding process are achieved, thereby significantly improving the quality of video encoding and decoding and optimizing the user experience.
[0052] See also Figure 2 As shown, in order to quickly and accurately obtain the video content analysis result corresponding to the video to be encoded and decoded, the embodiment of the present application further provides a specific GPGPU-based video encoding and decoding method, which may include:
[0053] Step S21: Use GPGPU to process the video to be encoded and decoded to obtain a target video image frame, and extract features of the target video image frame based on a preset convolutional neural network model in a preset deep learning model to obtain a feature image.
[0054] In this embodiment, a preset convolutional neural network model is used to extract features from the target video image frame to obtain a feature image. It should be noted that the preset convolutional neural network model can automatically learn high-level, abstract spatial feature representations from the original pixels, and the feature representations include edges, textures, shapes, etc. Common convolutional neural network model architectures include AlexNet (i.e., deep convolutional neural network), VGG (Visual Geometry Group Network), and ResNet (Residual Network). The convolutional neural network model architecture is constructed by stacking multiple convolutional layers, activation layers such as RELU (Rectified Linear Unit, i.e., linear rectifier function), pooling layers such as maximum pooling layers, and fully connected layers. It should be noted that each convolutional layer in the convolutional neural network model will output a series of feature maps, which contain spatial information of the input image at different scales. As the depth of the network increases, the resolution of the feature map gradually decreases, but the semantic information contained is richer.
[0055] Step S22: input the feature image into the preset recurrent neural network model in the preset deep learning model, so that the preset recurrent neural network model processes the feature image based on the time axis to obtain video content features.
[0056] In this embodiment, since the video is a time series data, it is necessary to use a preset convolutional neural network model or its variant to capture the time dependency between the target video image frames. Specifically, the feature image extracted by the preset convolutional neural network model is input into the preset recurrent neural network model, and each time step corresponds to each frame in the video to be encoded and decoded, so that the preset recurrent neural network model processes the feature image based on the time axis, obtains the dynamic changes in the video to be encoded and decoded, and obtains the video content features. It should be pointed out that in order to meet the real-time processing requirements, the preset recurrent neural network model needs to be compressed and accelerated, such as pruning, quantization, distillation, etc.
[0057] Step S23: input the video content features into the classifier in the preset deep learning model for feature classification processing to obtain a video content analysis result corresponding to the video to be encoded and decoded.
[0058] It should be noted that the classifier in this embodiment is a classifier obtained by training the classification performance of the initial classifier in the preset deep learning model using historical data; the historical data is the video data after labeling based on the preset classification labels. The historical data includes scene types such as outdoor and indoor, action categories such as running and dancing, and emotional labels such as happy and sad.
[0059] Step S24 inputs the current network environment, user terminal device performance information and the video content analysis results into the preset deep learning model to obtain the current encoding and decoding strategy and the current GPGPU allocatable computing resources.
[0060] Step S25: Perform video encoding and decoding processing on the video to be encoded and decoded based on the current encoding and decoding strategy and the current GPGPU allocatable computing resources to obtain a target video.
[0061] Among them, the specific implementation process of steps S24 and S25 can refer to the corresponding content disclosed in the above-mentioned embodiments, and will not be repeated here.
[0062] As can be seen from the above, in this embodiment, the GPGPU is first used to process the video to be encoded and decoded to obtain a target video image frame, and the target video image frame is feature extracted based on the preset convolutional neural network model in the preset deep learning model to obtain a feature image; the feature image is input into the preset recurrent neural network model in the preset deep learning model, so that the preset recurrent neural network model processes the feature image based on the time axis to obtain video content features; the video content features are input into the classifier in the preset deep learning model for feature classification processing to obtain the video content analysis result corresponding to the video to be encoded and decoded; the current network environment, user terminal device performance information and the video content analysis result are input into the preset deep learning model to obtain the current encoding and decoding strategy and the current GPGPU allocatable computing resources; based on the current encoding and decoding strategy and the current GPGPU allocatable computing resources, the video to be encoded and decoded is subjected to video encoding and decoding processing to obtain the target video. As can be seen from the above, this embodiment uses a preset convolutional neural network model to extract features from the processed target video image frame to obtain a feature image, then uses a preset recurrent neural network model to process the feature image to obtain video content features, and then uses a classifier in a preset deep learning model to process the video content features to obtain corresponding video content analysis results. In this way, by performing feature analysis on the target video image frame using a preset deep learning model, the video content analysis results corresponding to the video to be encoded and decoded are accurately and efficiently obtained, thereby significantly improving the efficiency and quality of video transmission.
[0063] refer to Figure 3As shown, the present application discloses a specific GPGPU-based video encoding and decoding method, the process of which is as follows: first, the system is initialized, and then the GPGPU and deep learning model are used to perform video content analysis on the video to be encoded and decoded to obtain the video content analysis result, and then based on the video content analysis result and the current network information and the user terminal device performance information, the deep learning model is used to determine the current encoding and decoding strategy, and the current GPGPU allocatable computing resources are allocated at the same time, and finally, based on the current encoding and decoding strategy and the allocatable computing resources of the GPGPU, the video to be encoded and decoded is encoded and decoded. It should be pointed out that during the video encoding and decoding process, it is necessary to use a preset real-time monitoring framework to monitor the process in real time, in order to continuously optimize the current encoding and decoding strategy and the current GPGPU allocatable computing resources, and obtain the final target video through iterative updates.
[0064] Accordingly, see Figure 4 As shown, the embodiment of the present application also provides a GPGPU-based video encoding and decoding device, which may include:
[0065] The video analysis module 11 is used to process the video to be coded and decoded by using GPGPU to obtain a target video image frame, and perform feature analysis on the target video image frame based on a preset deep learning model to obtain a video content analysis result corresponding to the video to be coded and decoded;
[0066] An information determination module 12, configured to input the current network environment, user terminal device performance information and the video content analysis result into the preset deep learning model to obtain the current encoding and decoding strategy and the current GPGPU allocatable computing resources;
[0067] The target video acquisition module 13 is used to perform video encoding and decoding processing on the video to be encoded and decoded based on the current encoding and decoding strategy and the current GPGPU allocatable computing resources to obtain the target video.
[0068] As can be seen from the above, in the present application, GPGPU is first used to process the video to be encoded and decoded to obtain a target video image frame, and feature analysis is performed on the target video image frame based on a preset deep learning model to obtain a video content analysis result corresponding to the video to be encoded and decoded; the current network environment, user terminal device performance information and the video content analysis result are input into the preset deep learning model to obtain the current encoding and decoding strategy and the current GPGPU allocatable computing resources; based on the current encoding and decoding strategy and the current GPGPU allocatable computing resources, video encoding and decoding processing is performed on the video to be encoded and decoded to obtain the target video. As can be seen from the above, before encoding and decoding the target video, the present application first uses GPGPU and a deep learning model to analyze the video to be encoded and decoded to obtain the corresponding video content analysis result, and then uses the deep learning model to determine the current encoding and decoding strategy based on the video content analysis result and the current network information and the user terminal device performance information, and at the same time allocates the current GPGPU computing resources, and finally encodes and decodes the video to be encoded and decoded based on the current encoding and decoding strategy and the GPGPU computing resources. In this way, by efficiently utilizing the parallel computing power of GPGPU combined with deep learning algorithms, deep intelligent control and optimization of the video encoding and decoding process are achieved, thereby significantly improving the quality of video encoding and decoding and optimizing the user experience.
[0069] In some specific implementations, the video analysis module 11 includes:
[0070] The video processing unit is used to use GPGPU to segment the video to be encoded and decoded to obtain corresponding initial video image frames, and to normalize each of the initial video image frames to obtain the target video image frame.
[0071] In some specific implementations, the video analysis module 11 includes:
[0072] A first feature extraction unit, configured to extract features of the target video image frame based on a preset neural network model in the preset deep learning model to obtain video content features;
[0073] A feature classification unit is used to input the video content features into the classifier in the preset deep learning model for feature classification processing to obtain a video content analysis result corresponding to the video to be encoded and decoded.
[0074] In some specific implementations, the video analysis module 11 includes:
[0075] A second feature extraction unit, configured to extract features of the target video image frame based on a preset convolutional neural network model in the preset deep learning model to obtain a feature image;
[0076] A feature processing unit is used to input the feature image into a preset recurrent neural network model in the preset deep learning model, so that the preset recurrent neural network model processes the feature image based on a time axis to obtain video content features.
[0077] In some specific implementations, the information determination module 12 includes:
[0078] A state vector obtaining unit, used to obtain the current network environment and user terminal device performance information, and integrate the current network environment, the user terminal device performance information and the video content analysis result to obtain a state vector;
[0079] An information determination unit is used to input the state vector into the preset deep learning model to obtain the current encoding and decoding strategy and the current GPGPU allocatable computing resources.
[0080] In some specific implementations, the target video acquisition module 13 further includes:
[0081] A model updating unit, configured to monitor the video encoding and decoding process of the video to be encoded and decoded in real time based on a preset real-time monitoring framework to obtain actual encoding and decoding performance data, and input the actual encoding and decoding performance data into the preset deep learning model for feedback learning using a preset feedback mechanism to obtain an updated preset deep learning model;
[0082] An operation jump unit is used to generate a new current codec strategy and a new current GPGPU allocatable computing resources based on the updated preset deep learning model, and determine the updated preset deep learning model as the new preset deep learning model, and then jump to the step of performing video codec processing on the video to be encoded and decoded based on the current codec strategy and the current GPGPU allocatable computing resources until the video codec processing operation of the video to be encoded and decoded is completed.
[0083] Furthermore, the present application also discloses an electronic device. Figure 5 It is a structural diagram of an electronic device 20 shown according to an exemplary embodiment, and the content in the figure cannot be regarded as any limitation on the scope of use of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input and output interface 25 and a communication bus 26. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the GPGPU-based video encoding and decoding method disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.
[0084] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.
[0085] In addition, the memory 22 as a carrier for resource storage may be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon may include an operating system 221, a computer program 222, etc., and the storage method may be temporary storage or permanent storage.
[0086] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the GPGPU-based video encoding and decoding method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.
[0087] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned GPGPU-based video encoding and decoding method is implemented. The specific steps of the method can refer to the corresponding contents disclosed in the aforementioned embodiments, and will not be repeated here.
[0088] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
[0089] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0090] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
[0091] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0092] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technical personnel in this field, according to the idea of the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.
Claims
1. A video encoding and decoding method based on GPGPU, characterized in that: include: Using GPGPU to process the video to be coded and decoded to obtain a target video image frame, and performing feature analysis on the target video image frame based on a preset deep learning model to obtain a video content analysis result corresponding to the video to be coded and decoded; Input the current network environment, user terminal device performance information and the video content analysis results into the preset deep learning model to obtain the current encoding and decoding strategy and the current GPGPU allocatable computing resources; Based on the current encoding and decoding strategy and the current GPGPU allocatable computing resources, video encoding and decoding processing is performed on the video to be encoded and decoded to obtain a target video.
2. The GPGPU-based video encoding and decoding method according to claim 1, characterized in that: The method of using GPGPU to process the video to be coded and decoded to obtain a target video image frame includes: The video to be encoded and decoded is segmented by using GPGPU to obtain corresponding initial video image frames, and each of the initial video image frames is normalized to obtain the target video image frame.
3. The GPGPU-based video encoding and decoding method according to claim 1, characterized in that: The performing feature analysis on the target video image frame based on a preset deep learning model to obtain a video content analysis result corresponding to the video to be encoded and decoded includes: Performing feature extraction on the target video image frame based on a preset neural network model in the preset deep learning model to obtain video content features; The video content features are input into the classifier in the preset deep learning model for feature classification processing to obtain the video content analysis result corresponding to the video to be encoded and decoded.
4. The GPGPU-based video encoding and decoding method according to claim 3, characterized in that: The extracting features of the target video image frame based on the preset neural network model in the preset deep learning model to obtain video content features includes: Performing feature extraction on the target video image frame based on a preset convolutional neural network model in the preset deep learning model to obtain a feature image; The feature image is input into a preset recurrent neural network model in the preset deep learning model, so that the preset recurrent neural network model processes the feature image based on a time axis to obtain video content features.
5. The GPGPU-based video encoding and decoding method according to claim 3, characterized in that: The classifier is a classifier obtained by training the classification performance of the initial classifier in the preset deep learning model using historical data; the historical data is video data that has been labeled based on preset classification labels.
6. The GPGPU-based video encoding and decoding method according to claim 1, characterized in that: The step of inputting the current network environment, the user terminal device performance information and the video content analysis result into the preset deep learning model to obtain the current encoding and decoding strategy and the current GPGPU allocatable computing resources includes: Acquire the current network environment and user terminal device performance information, and integrate the current network environment, the user terminal device performance information and the video content analysis result to obtain a state vector; The state vector is input into the preset deep learning model to obtain the current encoding and decoding strategy and the current GPGPU allocatable computing resources.
7. The GPGPU-based video encoding and decoding method according to any one of claims 1 to 6, characterized in that: After performing video encoding and decoding processing on the video to be encoded and decoded based on the current encoding and decoding strategy and the current GPGPU allocatable computing resources, the method further includes: Based on a preset real-time monitoring framework, the video encoding and decoding process of the video to be encoded and decoded is monitored in real time to obtain actual encoding and decoding performance data, and the actual encoding and decoding performance data is input into the preset deep learning model for feedback learning by using a preset feedback mechanism to obtain an updated preset deep learning model; A new current encoding and decoding strategy and a new current GPGPU allocatable computing resources are generated based on the updated preset deep learning model, and the updated preset deep learning model is determined as the new preset deep learning model, and then the process jumps to the step of performing video encoding and decoding processing on the video to be encoded and decoded based on the current encoding and decoding strategy and the current GPGPU allocatable computing resources until the video encoding and decoding processing operation of the video to be encoded and decoded is completed.
8. A video encoding and decoding device based on GPGPU, characterized in that: include: A video analysis module is used to process the video to be coded and decoded using GPGPU to obtain a target video image frame, and perform feature analysis on the target video image frame based on a preset deep learning model to obtain a video content analysis result corresponding to the video to be coded and decoded; An information determination module, used to input the current network environment, user terminal device performance information and the video content analysis result into the preset deep learning model to obtain the current encoding and decoding strategy and the current GPGPU allocatable computing resources; The target video acquisition module is used to perform video encoding and decoding processing on the video to be encoded and decoded based on the current encoding and decoding strategy and the current GPGPU allocatable computing resources to obtain the target video.
9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the GPGPU-based video encoding and decoding method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: Used to store a computer program; wherein, when the computer program is executed by a processor, the GPGPU-based video encoding and decoding method according to any one of claims 1 to 7 is implemented.