Processing method, device and equipment

Abstract

The invention discloses a processing method, device and equipment, and the method comprises the steps: analyzing a video frame outputted when electronic equipment operates a target application; if the target video frame meeting the first condition exists, processing the target video frame based on a first processor to generate a target video; the first processor is one of a plurality of processors in the electronic equipment, and when the target application runs, the resource consumption of the first processor is lower than that of other processors; in the process of generating the target video, the output quality of the output video frame corresponding to the target application meets a second condition.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of video processing, and relate to but are not limited to a processing method, device and equipment. Background Art

[0002] Game videos in the gaming industry are also a very popular section on major websites, and the number of user views of highlight short videos is very high. The solution of automatically generating highlight videos based on artificial intelligence (AI) computer vision recognition is a favorite tool for most players. Although the automatic generation solution can reduce user operations and lower the threshold for generating highlight videos, the resource consumption of each processor in the electronic device during the video generation process also has a great impact on the frame rate and fluency of the game. More and more users have reported that it will cause game freezes and problems. How to reduce the impact of collecting highlight short videos on the fluency of the game has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] In view of this, embodiments of the present application provide a processing method, device and equipment.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] In a first aspect, an embodiment of the present application provides a processing method, including:

[0006] Parsing video frames output when the electronic device runs the target application;

[0007] If there is a target video frame that meets the first condition, the target video is generated based on the first processor processing the target video frame; the first processor is one of multiple processors in the electronic device, and the resource consumption of the first processor when the target application is running is lower than the resource consumption of other processors;

[0008] During the process of generating the target video, the output quality of the output video frame corresponding to the target application satisfies the second condition.

[0009] In a second aspect, an embodiment of the present application provides a processing device, including:

[0010] A parsing module, used for parsing video frames output when the electronic device runs the target application;

[0011] a processing module, configured to generate a target video based on a first processor processing the target video frame if there is a target video frame that meets a first condition; the first processor is one of a plurality of processors in the electronic device, and resource consumption of the first processor when the target application is running is lower than resource consumption of other processors;

[0012] During the process of generating the target video, the output quality of the output video frame corresponding to the target application satisfies the second condition.

[0013] In a third aspect, an embodiment of the present application provides an electronic device comprising multiple processors, a display and a memory, wherein the memory stores a computer program that can be run on the processor, and the processor is used to parse the video frames provided to the display when the electronic device runs a target application; if there is a target video frame that meets a first condition, a target video is generated based on processing the target video frame by a first processor; the first processor is one of multiple processors, and the resource consumption of the first processor when the target application is running is lower than the resource consumption of other processors; in the process of generating the target video, the output quality of the output video frame corresponding to the target application meets the second condition.

[0014] In a fourth aspect, an embodiment of the present application provides a storage medium storing executable instructions for implementing the above method when executed by a processor.

[0015] In a fifth aspect, a computer program product includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the steps in the above method are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of an implementation flow of a processing method provided in an embodiment of the present application;

[0017] Figure 2 A schematic diagram of an implementation process of generating a target video provided in an embodiment of the present application;

[0018] Figure 3A A schematic diagram of frame rates provided in an embodiment of the present application;

[0019] Figure 3B A schematic diagram of frame rates provided in an embodiment of the present application;

[0020] Figure 4A A schematic diagram of a GPU occupancy rate provided in an embodiment of the present application;

[0021] Figure 4B A flow chart of an equalization processing method provided in an embodiment of the present application;

[0022] Figure 4C A schematic diagram of time and processor usage provided in an embodiment of the present application;

[0023] Figure 4D A schematic diagram of time and processor usage provided in an embodiment of the present application;

[0024] Figure 4E A schematic diagram of a processor usage rate provided in an embodiment of the present application;

[0025] Figure 4F A schematic diagram of a processor usage rate provided in an embodiment of the present application;

[0026] Figure 5 A schematic diagram of the structure of a processing device provided in an embodiment of the present application;

[0027] Figure 6 A hardware entity schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the specific technical solution of the embodiments of the present application will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application, but are not used to limit the scope of the present application.

[0029] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0030] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0032] The present application embodiment provides a processing method, such as Figure 1 As shown, the method includes:

[0033] Step S110, parsing the video frames output when the electronic device runs the target application;

[0034] Here, the target application can output video frames when running, that is, the target application can display changing screen content on the display of the electronic device when running. For example, the target application can be a game application, and the game screen can be displayed on the display when the game application is running; the target application can also be a video live broadcast application, and the live broadcast screen can be displayed on the display when the video live broadcast application is running.

[0035] Video can be considered as a series of continuous image frames. Each frame represents a time point in the video and contains rich visual information. Parsing video frames can be the understanding and extraction of the corresponding displayed image of the video frame. By parsing video frames, key information in the video can be obtained.

[0036] During implementation, parsing the video frame output when the target application is running may be identifying the screen content corresponding to the video frame to determine whether the screen content corresponding to the video frame meets the first condition.

[0037] Step S120: if there is a target video frame that meets the first condition, generate a target video based on the first processor processing the target video frame; the first processor is one of a plurality of processors in the electronic device, and the resource consumption of the first processor when the target application is running is lower than the resource consumption of other processors;

[0038] During the process of generating the target video, the output quality of the output video frame corresponding to the target application satisfies the second condition.

[0039] Here, the first condition can be preset according to the need to obtain the target video frame. For example, when the target video frame is a video frame of a game application, the first condition can be set to the video frame of the highlight moment output by the game application, wherein the electronic device can automatically capture the video frame of the player's wonderful operation during the game as the video frame of the highlight moment, such as the video frame corresponding to the kill, assist, and combo operation. When the target video frame is a video frame of a live broadcast application, the first condition can be set to the video frame with the most participants in the live broadcast application.

[0040] The target video may be a video obtained by processing a plurality of target video frames. For example, when the target application is a game application, the target video may be a video of a game highlight moment.

[0041] An electronic device may have two or more processors with similar or different functions, i.e., a multi-processor system of the electronic device. During the operation of the electronic device, these processors can exchange data with each other and work independently. The multi-processor system can significantly improve the overall performance of the electronic device through parallel processing.

[0042] When an electronic device runs a target application, multiple processors may correspond to different resource consumption. For example, an electronic device may have both a central processing unit (CPU) and a graphics processing unit (GPU), where the CPU is the final execution unit for information processing and program execution. The GPU is a microprocessor used to process image and graphics-related operations.

[0043] When running a target application, the resource consumption of the CPU is different from that of the GPU. For example, a target application with high requirements for image rendering consumes more GPU resources during operation; a target application with many logical operations consumes more CPU resources during operation.

[0044] During implementation, a first processor may be determined, and when a target application is running, resource consumption of the first processor is lower than resource consumption of other processors. The first processor is used to process a target video frame to generate a target video. For example, a CPU with low resource consumption may be determined to process a video frame with a highlight moment when running a game application to generate a highlight moment video.

[0045] Here, the output quality of the video frame is a key factor affecting the entire video viewing experience. Frame rate is also called FPS (Frames Per Second), which is the number of picture frames refreshed per second. It determines the smoothness of the video and is a key factor affecting the output quality of the video frame.

[0046] The second condition can be preset according to actual needs. For example, the second condition can be set to a target frame rate, and in the process of generating the target video, the frame rate of the output video can be kept not lower than the target frame rate to achieve the effect of not affecting the output quality of the output video.

[0047] In an embodiment of the present application, the video frames output when the electronic device runs the target application are parsed; if there is a target video frame that meets the first condition, the target video is generated based on the first processor processing the target video frame; the first processor is one of multiple processors in the electronic device, and the resource consumption of the first processor when the target application runs is lower than the resource consumption of other processors; in the process of generating the target video, the output quality of the corresponding output video frame of the target application meets the second condition. In this way, using the first processor with lower resource consumption to process the target video frame can reduce the impact on the computing power of the target application, and solve the problem of target application running freezes and reduced output quality caused by the process of processing the target video frame.

[0048] In some embodiments, in the above step S120, “processing the target video frame based on the first processor to generate the target video”, for example Figure 2 As shown, this can be achieved by following the steps below:

[0049] Step S210: determining a second duration based on the first duration; the first duration is a set playback duration of the target video frame, the first duration is related to the running characteristics of the target application, and the second duration is greater than the first duration;

[0050] Here, the operating characteristics of the target application refer to the key attributes and behavior patterns exhibited during the software operation process. For example, if the target application is a game application, the operating characteristics of the game application may refer to the target behaviors triggered by the user during the game, such as using a big move, character sacrifice, victory celebration, or important dialogue.

[0051] The first duration is related to different operation characteristics. For example, when the target application is a game application, the target video frame corresponds to the action of performing a special move, and the first duration is the duration required for performing the special move.

[0052] During implementation, the second duration is determined based on the first duration, and the value of the second duration is greater than the first duration.

[0053] Step S220: Within the second time length, use the first processor to process the target video frame to generate the target video.

[0054] During implementation, the target video frame may be processed by the first process to generate the target video within a second duration that is greater than the first duration.

[0055] For example, the first processor can be used to acquire and process target video frames, such as the highlight moment video frame of the game, the playback duration (first duration) of the generated target video is 3 seconds, and the video frame rate is 60FPS. When the first processor recognizes the highlight moment, the first processor acquires 60 frames of images per second, and continues to acquire for 3 seconds, and a total of 180 frames of images can be acquired; if the first processor processes 60 frames of images per second and generates a 3-second target video, it must process 180 frames of images within 3 seconds, where the processing methods include format conversion, scaling, cropping, and encoding. If 60 frames of images are processed in 1 second, the occupancy rate of the first processor is about 60%. Here, corresponding to the first duration of 3 seconds, the second duration can be set to 30 seconds, then 180 frames of images are processed in 30 seconds, and 6 frames are processed in 1 second, which can reduce the occupancy rate of the first processor to about 6%. Obviously, within the second duration of 30 seconds, which is longer than the first duration of 3 seconds, using the first processor to process the target video frames to generate the target video can significantly reduce the occupancy rate of the first processor, thereby releasing part of the computing power to process other computing power requirements of the target application.

[0056] In the embodiment of the present application, the second duration is determined based on the first duration; the first duration is the set playback duration of the target video frame, the first duration is related to the operation characteristics of the target application, and the second duration is greater than the first duration; within the second duration, the target video frame is processed by the first processor to generate the target video. In this way, processing the target video frame at the second duration that is greater than the first duration used to play the target video frame can significantly reduce the occupancy rate of the first processor, so as to release part of the computing power for processing other computing power requirements of the target application, and reduce the impact of processing the target video on the smoothness of the operation of the target application.

[0057] In some embodiments, the above step S220 "using the first processor to process the target video frame to generate the target video within the second time duration" can be implemented by the following steps:

[0058] Step 221, obtaining a first parameter; the first parameter represents the frame rate of the target video when it is played;

[0059] Here, the frame rate of video playback refers to the number of frames played per second, which is an important indicator for measuring the smoothness and visual effects of the video. The frame rate refers to the number of still images displayed per second in a video file. The frame rate determines the smoothness and visual effects of the video. A higher frame rate will make the action smoother, while a lower frame rate may cause incoherent action or obvious stuttering. The first parameter may be the frame rate of the target video playback. For example, if the target video is a game highlight moment video, the frame rate of the video can be set to 60FPS.

[0060] During implementation, the first parameter may be preset based on the characteristics of the target video, that is, the frame rate of the target video when it is played may be acquired in advance.

[0061] Step 222: Based on the first parameter and the first duration, divide the second duration into a plurality of continuous or discontinuous second sub-durations, so that the first processor processes the target video frame in each of the second sub-durations to generate the target video.

[0062] Here, the second duration may be divided into a plurality of continuous or discontinuous second sub-durations with reference to the playback frame rate of the target video (first parameter) and the playback duration of the target video frame (first duration).

[0063] For example, the second duration is divided into multiple consecutive second sub-durations, and the second sub-duration can be set to 1 second. Then, the target video frame can be processed in the consecutive second sub-durations, that is, the target video frame is processed every second of the second duration.

[0064] The second duration is divided into a plurality of discontinuous second sub-durations, and the second sub-duration can be set to 1 second, and the interval duration is 2 seconds, and the target video frame is processed within 1 second of the first and second sub-durations with an interval of 2 seconds.

[0065] In some embodiments, the processor may also process the target video frame within a continuous second sub-duration within a second duration.

[0066] In the embodiment of the present application, a first parameter is first obtained; the first parameter represents the frame rate of the target video when it is played; then based on the first parameter and the first duration, the second duration is divided into a plurality of continuous or discontinuous second sub-durations, so that the first processor processes the target video frame in each second sub-duration respectively to generate the target video. In this way, the first processor processes the target video frame in each continuous or discontinuous second sub-duration respectively, that is, the task of processing the target video is evenly distributed over a period of time, which can effectively reduce the utilization rate of the first processor and reduce the jamming and unsmoothness of the target application during operation.

[0067] In some embodiments, the first processor generates multiple target videos according to a target period, and the end time of the second duration is earlier than the start time of the target video of the next period; the target period is related to the operating characteristics of the object running the target application.

[0068] Here, the multiple target videos may be video highlights obtained by combining multiple target videos. For example, when the target video is a game highlight moment clip, multiple game highlight moment clips may be combined to obtain a video highlight of the highlight moment.

[0069] In the implementation process, the video collection can be generated in a discontinuous time, that is, the generated target video can be generated periodically by the installation target. The end time of the second duration is earlier than the start time of generating the target video of the next cycle, that is, the processing of the target video of the previous cycle is completed before the target video of the next cycle starts processing.

[0070] The operation characteristics of the object of the target application can be different objects corresponding to different operation characteristics. For example, when the target application is a game application, different users have different operation times for the use of the big move. User 1 may trigger the big move operation once every 2 minutes, and user 2 may trigger the big move operation once every 5 minutes.

[0071] During implementation, the target cycle can be determined based on learning the user's operations on the target application over a period of time. For example, if it can be determined that the user triggers the ultimate move once every 2 minutes during the 10-minute learning period, the target cycle can be determined to be 30 seconds.

[0072] For example, the target period may be 2 minutes. If it takes 3 seconds to generate a 3-second target video, then after generating the target video of the current period, the next target video is generated after waiting for 1 minute and 57 seconds according to the target period of 2 minutes. That is, the occupancy rate of the first processor in the first 3 seconds due to processing the target video is 60%, and the occupancy rate of the first processor in the next 1 minute and 57 seconds due to not processing any target video is 0%. In the case of an uneven distribution of the occupancy rate of the first processor, the 3 seconds of processing the target video can be extended to within a target period of 2 minutes.

[0073] In the embodiment of the present application, the first processor generates multiple target videos according to the target period, and the end time of the second duration is earlier than the start time of generating the target video of the next period; the target period is related to the operation characteristics of the object running the target application. In this way, the set second period time does not affect the processing of the target video by the first processor in the next period, and it can achieve balanced use of the first processor to process the target video in the second duration.

[0074] In some embodiments, in the above step S120, "generating a target video based on processing the target video frame by the first processor" further includes the following steps:

[0075] Step 121: obtaining the running data of the target task on the first processor in the process of generating the target video; the target task includes tasks other than generating the target video;

[0076] Here, the target task includes tasks other than generating the target video, that is, the first processor, in addition to the task of generating the target video, also has tasks such as acquiring video frame images and analyzing the video frame images.

[0077] During the implementation process, the operation data of the target task is obtained, that is, the video frame image is obtained, and the image processing model is used to determine whether the image content in the video frame image meets the preset standard. For example, the operation data can be obtained by obtaining image data and using the image processing model to infer whether the image is a highlight moment image.

[0078] Step 122: determine the second duration or determine a division method of the second duration according to the running data of the target task, so that the first processor performs the task of generating the target video at a different time than the time of performing the target task.

[0079] Here, since the second duration is the duration of the first processor processing the target video frame to generate the target video, it can be different from the time when the target task is executed. In the implementation process, the second duration can be determined according to the running data of the target task or the division method of the second duration can be determined, that is, the execution time of the first processor executing the task of generating the target video within the second duration can be separated from the time when the target task is executed.

[0080] For example, the time when the first processor generates a highlight moment video based on the target video frame can be separated from the reasoning time when the first processor determines whether the acquired video frame is a highlight moment, that is, the reasoning time of the model is separated from the video processing time.

[0081] In the embodiment of the present application, the operation data of the target task on the first processor in the process of generating the target video is first obtained; the target task includes tasks other than generating the target video; and then the second duration is determined or the division method of the second duration is determined according to the operation data of the target task. In this way, the first processor can execute the task of generating the target video at a different time than the time of executing the target task, effectively reducing the occupancy rate of the first processor for executing the task of generating the target video and executing the target task at the same time.

[0082] In some embodiments, the above step S110 “parsing the video frames output when the electronic device runs the target application” can be implemented by the following steps:

[0083] Step S111, identifying whether the target behavior exists in the output video frame;

[0084] Here, the image recognition model can be used to identify the content of the video frame to determine whether the person in the frame has the target behavior. In the implementation process, the target behavior can be pre-set based on different requirements for obtaining the target video frame.

[0085] For example, when the target application is a game application, behaviors such as using a big move, killing, assisting, and combo can be pre-set as target behaviors. The image recognition model is used to identify the game screen to determine whether there is a big move, killing, assisting, and combo in the game screen.

[0086] Step S112: when it is determined that the target behavior exists, determining that the target video frame satisfies the first condition;

[0087] During the implementation process, when the image recognition model is used to recognize that the target behavior exists in the picture of the video frame, it can be determined that the target video frame meets the first condition and the target video can be generated.

[0088] or,

[0089] Step S113: when the electronic device runs the target application, determining whether a target operation is triggered corresponding to outputting the video frame;

[0090] During the implementation process, the user can also trigger the target operation to display the corresponding target behavior on the output video frame. For example, the user can trigger the corresponding video frame including the target operation of releasing the big move, killing, assisting, combo, etc.

[0091] Step S114: When determining that the target operation is triggered, determine that the target video frame meets the first condition.

[0092] During implementation, when the user triggers the target operation, it can be determined that the target video frame meets the first condition, and the target video can be generated. For example, when it is determined that the user triggers the ultimate move operation, the target video can be determined to be a highlight moment video.

[0093] In the embodiment of the present application, it is identified whether there is a target behavior in the output video frame; when it is determined that the target behavior exists, it is determined that the target video frame satisfies the first condition

[0094] In the case where the electronic device runs the target application, it is determined whether the target operation is triggered and the corresponding video frame is output; in the case where it is determined that the target operation is triggered, it is determined that the target video frame meets the first condition.

[0095] In some embodiments, the above processing method further includes any of the following steps:

[0096] Step S150, respectively determining that each processor for processing the target video frame, when processing the target video frame, correspondingly obtains a target frame rate of the output video frame corresponding to the target application, and determining the processor with the highest target frame rate as the first processor;

[0097] Here, since multiple processors are provided in the electronic device, the target frame rate of the target application output video frame can be determined respectively when the processors with different functions process the target video frame, that is, when the processor outputs the target video frame, a processor with the highest target frame rate of the output video corresponding to the target application is determined. The processor includes at least one of the following: a central processing unit, an image processor, and a neural network processor.

[0098] For example, Figure 3A A schematic diagram of the frame rate provided in the embodiment of the present application, such as Figure 3A As shown in the figure, a frame rate test of a game using GPU for AI reasoning and other operations shows that the average frame rate of the video is 120FPS.

[0099] Figure 3B A schematic diagram of the frame rate provided in the embodiment of the present application, such as Figure 3B As shown in the figure, a frame rate test of a certain game using the CPU to enter AI reasoning and other operations shows that the average frame rate of the video is 130.4FPS.

[0100] Based on the above Figure 3A and Figure 3B Comparing the frame rates, it is obvious that the average frame rate of using CPU for AI reasoning is 130.4FPS, which is higher than the average frame rate of 120FPS when using GPU for AI reasoning. CPU can be used for AI reasoning to improve the average frame rate of the video.

[0101] Step S160, obtaining the resource consumption of each processor when the target application is running, and determining the processor with the lowest resource consumption as the first processor;

[0102] Here, the resource consumption of the processor involves the time occupancy, main frequency, number of cores of the processor, and the allocation and utilization of computing resources related thereto. During the implementation process, the resource consumption of each processor can be determined. Since the processor with the lowest resource consumption has the most idle computing resources, the processor with the lowest resource consumption can be determined as the first processor. For example, the resource consumption corresponding to the GPU and the CPU can be obtained respectively. If the GPU resource consumption is low, the GPU can be determined as the first processor.

[0103] Step S170: When it is determined that the target application is a logic operation application, a processor not related to logic operation is determined as the first processor; when it is determined that the target application is a screen rendering application, a processor not used for screen rendering is determined as the first processor.

[0104] Here, different types of target applications consume different resources for different processors. When the target application requires more logical operations, it consumes more CPU computing power, and a GPU that is not related to logical operations can be determined as the first processor; when the target application is an application that requires more screen rendering, it consumes more GPU computing power, and a CPU that is not used for screen rendering can be determined as the first processor.

[0105] For example, when the target application is a gaming application, different types and styles of games have different requirements for the CPU and GPU. Based on whether the current game consumes the CPU or GPU, the processor that the game uses less can be selected to perform operations such as reasoning and encoding during highlight moments. Using this processor does not compete with the running of the game itself for resources.

[0106] The CPU is mainly responsible for processing game logic, handling game network communications, and communicating data with the GPU. For example, in some games with many operations and requiring a lot of calculations, the running speed is very critical. Therefore, the task undertaken by the CPU is very important. It is necessary to calculate the physical effects of a large number of players fighting on the same screen. Such games usually have relatively low requirements for individuals and scenes, so such games pay more attention to the performance of the CPU.

[0107] The GPU is responsible for most of the rendering. The more exquisite, complex, and high-definition the game screen is, the higher the demand for the graphics card. For example, some games have extremely fine graphics, various particle effects, full rendering, and a large number of anti-aliasing, depth of field, shadow and other computing tasks, which places high demands on the graphics card.

[0108] Although existing AI models can choose to use CPU or GPU, or use both CPU and GPU at the same time during reasoning, these are just choices of their own reasoning methods, and do not take into account the impact on the game frame rate and smoothness. Therefore, using CPU or GPU for reasoning and recognition using AI is a dynamic process, and the reasoning method can be selected dynamically according to the different games or scenes. For example, in games that consume a lot of GPU computing resources in the long term and have a low CPU usage rate, you can choose to perform reasoning on the CPU. This can ensure the game quality and frame rate, and capture exciting moments. For another example, for some games that consume more CPU and have a low GPU usage rate, you can run reasoning and encoding processing logic on the GPU. The purpose is to minimize the competition for resources with the game as much as possible, and minimize the impact on the game smoothness when making exciting video processing solutions.

[0109] In the embodiment of the present application, each processor for processing the target video frame is respectively determined to obtain the target frame rate of the output video frame corresponding to the target application when processing the target video frame, and the processor with the highest corresponding target frame rate is determined as the first processor. In this way, when the first processor processes the target video, the target frame rate of the output video frame corresponding to the target application is minimally affected, that is, the output video frame can be played at the highest target frame rate.

[0110] The resource consumption of each processor when the target application is running is obtained, and the processor with the lowest resource consumption is determined as the first processor. In this way, a processor with the lowest resource consumption can be determined as the first processor to process the target video frame without affecting the work of other processors with larger resource consumption, thereby achieving the minimum impact on the operation of the target application.

[0111] When the target application is determined to be a logic operation application, a processor not related to logic operation is determined as the first processor. When the target application is determined to be a screen rendering application, a processor not used for screen rendering is determined as the first processor. In this way, the operation of the target application can be minimized.

[0112] In some embodiments, after determining the first processor, the above processing method further includes the following steps:

[0113] Step S180: adding the first processor to a preset processor list, so as to utilize the first processor to process the target video frame again.

[0114] Here, a processor list can be preset, and the most reasonable processor that has been tested and verified can be added to the processor list as the first processor, and the processor list can be stored in the electronic device. Among them, different first processors corresponding to different target applications can be set in the processor name, that is, the mapping relationship between the identifier of the target application and the corresponding first processor can be set. For example, the first processor corresponding to the first application can be set to the CPU, and the first processor corresponding to the second application can be set to the GPU.

[0115] In the embodiment of the present application, adding the first processor to the preset processor list can achieve the goal of reusing the first processor to process the target video frame, effectively reducing the time to re-determine the first processor and improving the efficiency of acquiring the target video.

[0116] Game applications have very high requirements for GPU or CPU. Although hardware capabilities are constantly improving, the quality of game images is not far behind. Many popular games now consume a lot of CPU or GPU resources. Figure 4A A schematic diagram of a GPU occupancy rate provided in an embodiment of the present application, such as Figure 4A As shown, it can be seen that the GPU occupancy rate of a certain game can reach 95.8%. As long as the GPU computing power is sufficient, the game can also run at a higher frame rate.

[0117] Figure 4B A flow chart of an equalization processing method provided in an embodiment of the present application is shown as follows: Figure 4B As shown, the method can be implemented by the following steps:

[0118] Step S410, capturing the running game screen content;

[0119] In the implementation process, a processor set in an electronic device can be used to capture the running game screen content to determine whether the screen content meets the preset standard of the highlight moment. The processor includes at least one of the following: a central processing unit, an image processor, and a neural network processor.

[0120] Step S420: Select a reasoning method;

[0121] In the implementation process, the selection of reasoning mode is divided into two modes: supported whitelist games and dynamic selection. If it is determined to be a supported whitelist game, step S430 is executed; if it is determined that the game is not in the whitelist, step S440 is executed if the dynamic selection reasoning mode is selected.

[0122] Step S430: static reasoning method;

[0123] In the implementation process, the static reasoning method can be to set up whitelist games. Among them, the whitelist in the whitelist game is tested and verified, and a reasonable reasoning processor is selected according to whether the game consumes CPU or GPU.

[0124] Step S440: dynamic reasoning method;

[0125] Here, games not supported in the whitelist can choose dynamic reasoning. During the game, you can dynamically decide whether to use CPU reasoning or GPU reasoning based on the impact of CPU reasoning and GPU reasoning on the game frame rate. Start the CPU and GPU reasoning objects at the same time, use them for a period of time, and decide on a reasoning method that has less impact on the game frame rate based on their impact on the game frame rate.

[0126] Step S450, selecting a coding segment file after reasoning;

[0127] During implementation, whether it is a fragment file to be processed can be determined based on the user's triggering operation and the screen content of the game.

[0128] Step S460: Perform an equalization operation on the segment file using the equalization measurement.

[0129] Here, the core feature of the game highlight video solution is to save the operation screen of the exciting moments of the game, so the encoding and other operations are multiple fragmented operations, not a long continuous process. Taking advantage of this discontinuous feature, the operations in a short period of time are stretched and averaged to a longer period of time, which can reduce the power consumption of the processor and improve the smoothness of the game frame rate.

[0130] In the embodiment of the present application, the wonderful moments using the highlight solution are fragments of game operations rather than features of the entire process. By adopting an equalization operation, the tasks that are concentrated within a period of time are dispersed and completed over a longer period of time, which will reduce the CPU / GPU occupancy rate and thus improve the smoothness of the game.

[0131] Depending on the characteristics of different game styles (CPU or GPU consumption), the highlight processing solution tries to minimize competition with the game for CPU / GPU resources, thereby reducing the impact on game smoothness.

[0132] Dynamically select the inference scheme, start the inference instance on the CPU / GPU side at the same time, constantly monitor the impact of CPU / GPU inference on the game smoothness and frame rate, and dynamically select the inference scheme with less impact on the frame rate.

[0133] Figure 4C A schematic diagram of time and processor usage provided by an embodiment of the present application, such as Figure 4C As shown, the horizontal axis of the schematic diagram is the time axis, and the vertical axis is the processor usage rate. 41 and 42 are schematic diagrams of the processor usage rate for processing two wonderful moment videos in two concentrated time periods.

[0134] Figure 4D A schematic diagram of time and processor usage provided by an embodiment of the present application, such as Figure 4D As shown, the horizontal axis of the schematic diagram is the time axis, the vertical axis is the processor usage rate, and 43 is a schematic diagram of the processor usage rate of processing two wonderful moment videos in a balanced manner within a period of time.

[0135] Figure 4E A schematic diagram of a processor usage rate provided in an embodiment of the present application, such as Figure 4E As shown, the processor usage is 4.21%.

[0136] Figure 4F A schematic diagram of a processor usage rate provided in an embodiment of the present application, such as Figure 4F As shown, the processor usage is 1.45%.

[0137] in, Figure 4C and Figure 4E Correspondingly, it shows that the processor usage rate for processing two highlight moment videos in two concentrated time periods is 4.21%. Figure 4D and Figure 4F Correspondingly, it shows that the processor usage rate of processing two highlight moment videos in a balanced manner over a period of time is 1.45%. Figure 4E and 4F It can be seen that after using balanced encoding, the processor utilization rate is reduced, which is very helpful in reducing game lag and unsmoothness during exciting moments.

[0138] For example, the characteristic of generating a video of the highlights of a game is that during the game, only the images of the highlights need to be converted and encoded and saved as a video file. Therefore, the conversion and encoding operations can be set to be processed in discontinuous time periods, that is, the image conversion, encoding and saving operations are not completed in a centralized manner, but are evenly distributed over a period of time, such as Figure 4D shown.

[0139] Based on the foregoing embodiments, the embodiments of the present application provide a processing device, which includes the modules included, each module includes each sub-module, each sub-module includes a unit, which can be implemented by a processor in an electronic device; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA), etc.

[0140] Figure 5 A schematic diagram of the structure of the processing device provided in the embodiment of the present application is shown in FIG. Figure 5 As shown, the device 500 includes:

[0141] The parsing module 510 is used to parse the video frames output when the electronic device runs the target application;

[0142] The processing module 520 is configured to generate a target video by processing the target video frame based on a first processor if there is a target video frame that meets a first condition; the first processor is one of a plurality of processors in the electronic device, and resource consumption of the first processor when the target application is running is lower than resource consumption of other processors;

[0143] During the process of generating the target video, the output quality of the output video frame corresponding to the target application satisfies the second condition.

[0144] In some embodiments, the processing module 520 includes a setting submodule and a processing submodule, wherein the setting submodule is used to determine a second duration based on a first duration; the first duration is the set playback duration of the target video frame, the first duration is related to the operating characteristics of the target application, and the second duration is greater than the first duration; the processing submodule is used to use the first processor to process the target video frame to generate the target video within the second duration.

[0145] In some embodiments, the processing submodule includes an acquisition unit and a division unit, wherein the acquisition unit is used to obtain a first parameter; the first parameter represents the frame rate of the target video when it is played; the division unit is used to divide the second duration into multiple continuous or discontinuous second sub-durations based on the first parameter and the first duration, so that the first processor processes the target video frame in each of the second sub-durations respectively to generate the target video.

[0146] In some embodiments, the first processor generates multiple target videos according to a target period, and the end time of the second duration is earlier than the start time of the target video of the next period; the target period is related to the operating characteristics of the object running the target application.

[0147] In some embodiments, the processing module 520 also includes an acquisition submodule and a first determination submodule, wherein the acquisition submodule is used to obtain the running data of the target task on the first processor in the process of generating the target video; the target task includes tasks other than generating the target video; the first determination submodule is used to determine the second duration or determine the division method of the second duration according to the running data of the target task, so that the first processor performs the task of generating the target video at a different time than the time of performing the target task.

[0148] In some embodiments, the parsing module 510 includes an identification submodule and a second determination submodule, or a third determination submodule and a fourth determination submodule, wherein the identification submodule is used to identify whether there is a target behavior in the output video frame; the second determination submodule is used to determine that when the target behavior exists, the target video frame satisfies the first condition; the third determination submodule is used to determine whether the target operation is triggered and the corresponding video frame is output when the electronic device runs the target application; the fourth determination submodule is used to determine that when the target operation is triggered, the target video frame satisfies the first condition.

[0149] In some embodiments, the processing device also includes any one of the following modules: a first determination module, a second determination module and a third determination module, wherein the first determination module is used to respectively determine that each processor used to process the target video frame obtains the target frame rate of the output video frame corresponding to the target application when processing the target video frame, and determines the processor with the highest target frame rate as the first processor; the second determination module is used to obtain the resource consumption of each processor when the target application is running, and determines the processor with the lowest resource consumption as the first processor; the third determination module is used to determine that when the target application is a logical operation application, the processor related to non-logical operation is determined as the first processor, and when the target application is determined to be a screen rendering application, the processor not used for screen rendering is determined as the first processor.

[0150] In some embodiments, the apparatus further comprises an adding module, configured to add the first processor to a preset processor list, so as to utilize the first processor to process the target video frame again.

[0151] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0152] It should be noted that in the embodiment of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable an electronic device (which can be a mobile phone, a tablet computer, a laptop computer, a desktop computer, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0153] Correspondingly, an embodiment of the present application provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the processing method provided in the above embodiment are implemented.

[0154] Correspondingly, an embodiment of the present application provides an electronic device, Figure 6 A hardware entity diagram of an electronic device provided in an embodiment of the present application, such as Figure 6As shown, the hardware entity of the device 600 includes: a memory 601 and a processor 602, wherein the memory 601 stores a computer program that can be run on the processor 602, and the processor 602 implements the steps in the processing method provided in the above embodiment when executing the program.

[0155] The memory 601 is configured to store instructions and applications executable by the processor 602, and can also cache data to be processed or processed by the processor 602 and various modules in the electronic device 600 (for example, image data, audio data, voice communication data, and video communication data), which can be implemented through flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0156] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0157] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0158] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0159] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0160] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0161] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0162] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0163] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can be essentially or partly reflected in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions to enable an electronic device (which can be a mobile phone, a tablet computer, a laptop computer, a desktop computer, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0164] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0165] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0166] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0167] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A processing method, the method comprising: Parsing video frames output when the electronic device runs the target application; If there is a target video frame that meets the first condition, generating a target video based on the first processor processing the target video frame; The first processor is one of a plurality of processors in the electronic device, and resource consumption of the first processor when the target application is running is lower than resource consumption of other processors; During the process of generating the target video, the output quality of the output video frame corresponding to the target application satisfies the second condition.

2. The method according to claim 1, wherein the step of generating a target video based on processing the target video frame by the first processor comprises: determining a second duration based on the first duration; The first duration is a set playback duration of the target video frame, the first duration is related to the running characteristics of the target application, and the second duration is greater than the first duration; During the second time period, the target video frame is processed by the first processor to generate the target video.

3. The method according to claim 2, wherein, within the second duration, processing the target video frame by the first processor to generate the target video comprises: Get the first parameter; The first parameter represents the frame rate of the target video when it is played; Based on the first parameter and the first duration, the second duration is divided into a plurality of continuous or discontinuous second sub-durations, so that the first processor processes the target video frame in each of the second sub-durations to generate the target video.

4. According to the method as claimed in claim 2, the first processor generates multiple target videos according to a target period, and the end time of the second duration is earlier than the start time of the target video of the next period generated; the target period is related to the operating characteristics of the object running the target application.

5. The method according to claim 3 or 4, further comprising: Obtaining running data of a target task on the first processor during the process of generating the target video; The target task includes tasks other than generating the target video; The second duration is determined according to the running data of the target task, or a division method of the second duration is determined, so that the first processor performs the task of generating the target video at a different time than the time of performing the target task.

6. The method according to claim 1, wherein parsing the video frames output when the electronic device runs the target application comprises: Identify whether there is a target behavior in the output video frame; When it is determined that the target behavior exists, determining that the target video frame satisfies the first condition; or, In the case where the electronic device runs the target application, determining whether triggering a target operation corresponds to outputting the video frame; When it is determined that the target operation is triggered, it is determined that the target video frame meets the first condition.

7. The method according to claim 1, further comprising any of the following: Determine respectively when each processor for processing the target video frame processes the target video frame, obtain a target frame rate of the output video frame corresponding to the target application, and determine the processor with the highest target frame rate as the first processor; Obtaining resource consumption of each processor when the target application is running, and determining the processor with the lowest resource consumption as the first processor; When it is determined that the target application is a logic operation application, a processor not related to logic operation is determined as the first processor; when it is determined that the target application is a screen rendering application, a processor not used for screen rendering is determined as the first processor.

8. The method of claim 7, after determining the first processor, the method further comprises: The first processor is added to a preset processor list so as to utilize the first processor to process the target video frame again.

9. A processing device comprising: A parsing module, used for parsing video frames output when the electronic device runs the target application; a processing module, configured to generate a target video by processing the target video frame based on a first processor if there is a target video frame that meets a first condition; the first processor is one of a plurality of processors in the electronic device, and resource consumption of the first processor when the target application is running is lower than resource consumption of other processors; During the process of generating the target video, the output quality of the output video frame corresponding to the target application satisfies the second condition.

10. An electronic device comprising: A plurality of processors, displays and memories, wherein the memory stores a computer program executable on the processor, the processor is used to parse a video frame provided to the display when the electronic device runs a target application; if there is a target video frame that satisfies a first condition, a target video is generated based on processing the target video frame by a first processor; the first processor is one of the plurality of processors, and resource consumption of the first processor when the target application is running is lower than resource consumption of other processors; in the process of generating the target video, the output quality of the output video frame corresponding to the target application satisfies a second condition.

Images