Image processing method, device and electronic equipment

Abstract

The present application discloses a video image processing method, apparatus, and electronic device. When processing a target video frame of a video stream to be processed, the method determines a second scene recognition result for the target video frame by integrating the first scene recognition results of the target video frame and multiple reference video frames preceding the target video frame. Based on the second scene recognition result, the method then selects a corresponding image enhancement processing method to perform image enhancement processing on the target video frame. By incorporating temporal accumulation into the scene recognition process of the target video frame, the method avoids frequent changes in the display effect of the processed video image due to frequent scene changes in the video stream to be processed.

Description

Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to an image processing method, device and electronic device. Background Art

[0002] Image enhancement technology is a technology that adjusts the image brightness peak, color gamut, and dynamic range to improve the visual effect of video images. For example, a high-dynamic range (HDR) image can be obtained by enhancing a general image.

[0003] Typically, image enhancement technology is based on image color, brightness, and other data. Applying this technology to video streams with large continuous image changes can result in poor overall visual quality. For example, different enhancement models may be used to process video images from different scenes, resulting in significant differences in peak brightness after processing. If these two scenes occur frequently in a video stream, the entire video stream may experience excessive flickering, impacting the user's visual experience. Summary of the Invention

[0004] In order to overcome the above-mentioned deficiencies in the prior art, the present application aims to provide a video image processing method, the method comprising:

[0005] Get the target video frame in the video stream to be processed;

[0006] Inputting the target video frame into a pre-trained scene recognition model for processing to obtain a first scene recognition result of the target video frame;

[0007] Obtaining first scene recognition results of a plurality of reference video frames preceding the target video frame;

[0008] determining a second scene recognition result of the target video frame according to the first scene recognition result of the target video frame and the first scene recognition results of the plurality of reference video frames;

[0009] According to the second scene recognition result, a corresponding image enhancement processing method is selected to perform image enhancement processing on the target video frame to obtain a processed target video frame.

[0010] In a possible implementation, the step of inputting the target video frame into a pre-trained scene recognition model for processing to obtain a first scene recognition result of the target video frame includes:

[0011] Inputting the target video frame into a pre-trained scene recognition model for processing, and obtaining a probability value that the target video frame belongs to a preset scene as the first scene recognition result;

[0012] The step of determining the second scene recognition result of the target video frame according to the first scene recognition result of the target video frame and the first scene recognition results of the plurality of reference video frames comprises:

[0013] An average value of the first scene recognition results of the target video frame and the multiple reference video frames is calculated, and the average value is used as the second scene recognition result of the target video frame.

[0014] In a possible implementation, the step of inputting the target video frame into a pre-trained scene recognition model for processing to obtain a first scene recognition result of the target video frame includes:

[0015] inputting the target video frame into a pre-trained scene recognition model for processing, and obtaining a probability value that the target video frame belongs to a preset scene as the first scene recognition result;

[0016] The step of determining the second scene recognition result of the target video frame according to the first scene recognition result of the target video frame and the first scene recognition results of the plurality of reference video frames comprises:

[0017] Calculating a weighted average of first scene recognition results of the target video frame and the multiple reference video frames based on a preset first weight parameter of the target video frame and a second weight parameter of the multiple reference video frames; wherein the first weight parameter is greater than the second weight parameter;

[0018] The weighted average value is used as a second scene recognition result of the target video frame.

[0019] In a possible implementation, the step of selecting a corresponding image enhancement processing method to perform image enhancement processing on the target video frame according to the second scene recognition result includes:

[0020] According to the second scene recognition result, a corresponding brightness enhancement peak is selected, and high dynamic range enhancement processing is performed on the target video frame.

[0021] In a possible implementation, the step of selecting a corresponding brightness enhancement peak according to the second scene recognition result and performing high dynamic range enhancement processing on the target video frame includes:

[0022] Detecting whether different brightness enhancement peaks have been used to perform high dynamic range enhancement processing on the image of the video stream to be processed within a preset time period;

[0023] If so, performing high dynamic range enhancement processing on the target video frame using the brightness enhancement peak value used when processing a previous frame of the target video frame;

[0024] If not, performing high dynamic range enhancement processing on the target video frame using the brightness enhancement peak value corresponding to the second scene recognition result of the target video frame.

[0025] In a possible implementation, the step of selecting a corresponding brightness enhancement peak according to the second scene recognition result and performing high dynamic range enhancement processing on the target video frame includes:

[0026] If the second scene recognition result indicates that the target video frame belongs to a non-game interface scene, performing high dynamic range enhancement processing on the target video frame using the first brightness enhancement peak value;

[0027] If the second scene recognition result indicates that the target video frame belongs to a game interface scene, a second brightness enhancement peak is used to perform high dynamic range enhancement processing on the target video frame; wherein, the first brightness enhancement peak is smaller than the second brightness enhancement peak.

[0028] In a possible implementation, the step of selecting a corresponding brightness enhancement peak according to the second scene recognition result and performing high dynamic range enhancement processing on the target video frame includes:

[0029] If the second scene recognition result indicates that the target video frame belongs to a non-game interface scene, performing high dynamic range enhancement processing on the target video frame using the first brightness enhancement peak value;

[0030] If the second scene recognition result indicates that the target video frame belongs to a game interface scene, a second brightness enhancement peak is used to perform high dynamic range enhancement processing on the game screen area in the target video frame; and a third brightness enhancement peak is used to perform high dynamic range enhancement processing on the non-game screen area in the target video frame; wherein, the first brightness enhancement peak and the third brightness enhancement peak are smaller than the second brightness enhancement peak.

[0031] Another object of the present application is to provide an electronic device, comprising a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are executed by the processor, the image enhancement method provided in the present application is implemented.

[0032] Another object of the present application is to provide a machine-readable storage medium, characterized in that the machine-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are executed by one or more processors, the image enhancement method provided by the present application is implemented.

[0033] Compared with the prior art, this application has the following beneficial effects:

[0034] The video image processing method, device, and electronic device provided by the present application, when processing a target video frame of a video stream to be processed, determines a second scene recognition result for the target video frame by integrating the first scene recognition results of the target video frame and multiple reference video frames preceding the target video frame, and then selects a corresponding image enhancement processing method based on the second scene recognition result to perform image enhancement processing on the target video frame. In this way, time series accumulation is added to the scene recognition process of the target video frame, which can avoid frequent changes in the display effect of the processed video image due to frequent scene changes in the video stream to be processed. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 A schematic flow chart of the steps of the image processing method provided in an embodiment of the present application;

[0037] Figure 2 A data diagram of the scene recognition results provided in an embodiment of the present application;

[0038] Figure 3 A schematic diagram of an electronic device provided in an embodiment of the present application;

[0039] Figure 4 A schematic diagram of the steps of the image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] To make the objectives, technical solutions, and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0041] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in the present application without making any creative efforts shall fall within the scope of protection of the present application.

[0042] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0043] In the description of this application, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, or are the orientations or positional relationships in which the inventive product is typically placed when in use. These terms are intended only to facilitate the description of this application and simplify the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limiting this application. Furthermore, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0044] Furthermore, terms such as "horizontal," "vertical," and "overhanging" do not necessarily imply that a component must be absolutely horizontal or overhanging, but rather that it can be slightly tilted. For example, "horizontal" simply means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but rather that it can be slightly tilted.

[0045] It should also be noted that, in the description of this application, unless otherwise expressly specified or limited, the terms "disposed," "installed," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to internal connections between two components. Those skilled in the art will understand the specific meanings of the above terms in this application based on the specific circumstances.

[0046] See Figure 1 , Figure 1 This is a schematic diagram of a video image processing method provided by this embodiment. The following describes each step of the method in detail.

[0047] Step S110: Acquire a target video frame in the video stream to be processed.

[0048] In this embodiment, the video stream to be processed can have a variety of different scene pictures. Taking the live game video as an example, the video stream to be processed can include a game interface scene that displays the game interface, and a non-game interface scene that displays the game players or live audiences.

[0049] Step S120 : Input the target video frame into a pre-trained scene recognition model for processing to obtain a first scene recognition result of the target video frame.

[0050] In this embodiment, a scene recognition model may be pre-trained to identify the scene to which a video frame belongs. In one implementation, each video frame in the to-be-processed video stream may be processed by the scene recognition model to obtain the corresponding first scene recognition result.

[0051] Step S130 : Acquire first scene recognition results of a plurality of reference video frames preceding the target video frame.

[0052] In this embodiment, the scene recognition model can continuously and sequentially process the video frames in the to-be-processed video stream to obtain corresponding first scene recognition results, and record at least the first scene recognition results corresponding to a plurality of reference video frames within a preset number or a preset time period before the currently processed target video frame. For example, the first scene recognition results for at least 30 video frames starting from the current frame are recorded.

[0053] Step S140 : determining a second scene recognition result of the target video frame according to the first scene recognition result of the target video frame and the first scene recognition results of the plurality of reference video frames.

[0054] In this embodiment, when determining the second scene recognition result of the target video frame, in addition to considering the first scene recognition result of the target video frame, it is also necessary to combine the first scene recognition results of the multiple reference video frames. For example, the second scene recognition result of the target frame can be determined based on the first scene recognition results of a total of 30 video frames from the target video frame forward. In this way, the time domain accumulation feature is added to the second scene recognition result, which can reduce the impact of frequent changes in the first scene recognition result on the second scene recognition result.

[0055] Step S150 : Selecting a corresponding image enhancement processing method to perform image enhancement processing on the target video frame according to the second scene recognition result to obtain a processed target video frame.

[0056] In this embodiment, a corresponding image enhancement processing method can be selected according to the second scene recognition result of the target video frame. For example, different adjustment parameters are used to adjust items such as brightness peak, color gamut and dynamic range.

[0057] Based on the above design, the video image processing method provided in this embodiment combines the first scene recognition results of the multiple reference video frames and the target video frame to determine the second scene recognition result of the target video frame. By incorporating the characteristic of time domain accumulation, the impact of frequent scene changes on the second scene recognition result can be reduced. In this way, when selecting the corresponding image enhancement method for image enhancement processing based on the second scene recognition result of each video frame, the problem of frequent switching of image enhancement methods as the image scene changes is reduced, thereby reducing the frequent sudden changes in the overall brightness or overall color gamut in the processed video stream.

[0058] In one possible implementation, in step S120, the target video frame may be input into a pre-trained scene recognition model for processing, and a probability value that the target video frame belongs to a preset scene may be obtained as the first scene recognition result. Specifically, the scene recognition model may be trained to identify the target video frame as a preset scene or a non-preset scene, and the first scene recognition result output by the scene recognition model may be the probability that the target video frame is determined to be a preset scene. For example, the scene recognition model outputs a probability value between 0 and 1, where a higher probability value indicates a higher probability that the target video frame is determined to be a preset scene.

[0059] In step S130, the average value of the first scene recognition results of the target video frame and the multiple reference video frames can be calculated, and the average value can be used as the second scene recognition result of the target video frame. For example, the first scene recognition results of 30 consecutive video frames including the target video frame can be obtained, and then the average value of these 30 first scene recognition results can be calculated as the second scene recognition result of the target video frame. In this way, the second scene recognition result is obtained by cumulatively calculating multiple first scene recognition results in the time domain. Even if the first scene recognition results of multiple video frames are briefly switched, it will not cause an immediate change in the second scene recognition result, thereby reducing the problem of frequent jumps in the second scene recognition result, and further reducing the frequent changes in the image effect of the entire video stream caused by subsequent frequent switching of image enhancement processing methods.

[0060] In another possible implementation, in step S120, the target video frame may be input into a pre-trained scene recognition model for processing to obtain a probability value that the target video frame belongs to a preset scene as the first scene recognition result.

[0061] In step S130, the weighted average of the first scene recognition results of the target video frame and the multiple reference video frames can be calculated based on the preset first weight parameter of the target video frame and the second weight parameter of the multiple reference video frames. The first weight parameter is greater than the second weight parameter. The weighted average is then used as the second scene recognition result of the target video frame. In this way, using the weighted average as the second scene recognition result can increase the degree of influence of the first scene recognition result of the target video frame itself on its second scene recognition result, thereby reducing the second scene recognition result from frequently changing, and making the second scene recognition result as close as possible to the first scene recognition result of the target video frame itself.

[0062] In one possible implementation, in step S140, a corresponding brightness enhancement peak value may be selected based on the second scene recognition result, and high-dynamic range (HDR) enhancement processing may be performed on the target video frame. Specifically, for certain scenes with relatively simple or dark colors, HDR enhancement processing with a relatively low brightness enhancement peak value may be used; for certain scenes with relatively rich or bright colors, HDR enhancement processing with a relatively high brightness enhancement peak value may be used.

[0063] Furthermore, in one example, in step S140, it may be first detected whether different brightness enhancement peaks have been used to perform high dynamic range enhancement processing on the image of the video stream to be processed within a preset time period. For example, it may be detected whether the action of switching the brightness enhancement peak for HDR enhancement has been performed within 3 seconds.

[0064] If different brightness enhancement peaks have not been used to perform high dynamic range enhancement processing on the image of the video stream to be processed within the preset time period, the brightness enhancement peak corresponding to the second scene recognition result of the target video frame is used to perform high dynamic range enhancement processing on the target video frame.

[0065] If different brightness enhancement peak values ​​have been used to perform high dynamic range enhancement processing on the image of the video stream to be processed within the preset time period, the brightness enhancement peak value used when processing the previous frame of the target video frame is used to perform high dynamic range enhancement processing on the target video frame, thereby avoiding frequent sudden changes in the brightness of the processed video stream due to frequent switching of brightness enhancement peak values ​​in a short period of time.

[0066] In one possible implementation, since the images in the game interface scene are usually virtual synthesized images and do not have rich color toning, while the images of the non-game interface scenes (such as players, spectators, etc.) are usually real-shot images and have richer color toning, if the game interface scene and the non-game interface scene use the same brightness enhancement peak for HDR processing, it will cause the image frames of the non-game interface scene to be too bright compared to the image frames of the game interface scene. Therefore, in this embodiment, in step S140, when it is detected that the second scene recognition result indicates that the target video frame belongs to the non-game interface scene, the first brightness enhancement peak can be used to perform high dynamic range enhancement processing on the target video frame.

[0067] When the second scene recognition result indicates that the target video frame belongs to a game interface scene, the target video frame is subjected to high dynamic range enhancement using the second brightness enhancement peak value. Furthermore, the first brightness enhancement peak value is smaller than the second brightness enhancement peak value. This allows the peak brightness of non-game interface scenes to be lowered while maintaining the high brightness of the game interface scene, thereby reducing the degree of sudden brightness changes when switching between scenes and improving the viewing experience.

[0068] In another possible implementation, there may be game screen areas and non-game screen areas (such as static text areas) in the image frame of the game interface scene. Therefore, in step S140, in addition to using different brightness enhancement peaks for game scenes and non-game scenes, different brightness enhancement peaks can also be used for game screen areas and non-game screen areas in the game scene.

[0069] Specifically, if the second scene recognition result indicates that the target video frame belongs to a non-game interface scene, the first brightness enhancement peak is used to perform high dynamic range enhancement processing on the target video frame.

[0070] If the second scene recognition result indicates that the target video frame belongs to a game interface scene, high dynamic range enhancement processing is performed using the second brightness enhancement peak value for the game screen area in the target video frame. Simultaneously, high dynamic range enhancement processing is performed using the third brightness enhancement peak value for the non-game screen area in the target video frame. The first and third brightness enhancement peak values ​​are smaller than the second brightness enhancement peak value.

[0071] Furthermore, in step S140, when the value of the second scene recognition result is greater than a preset threshold, the target video frame is determined to belong to a game interface scene; when the value of the second scene recognition result is not greater than the preset threshold, the target video frame is determined to belong to a non-game interface scene. For example, if the second scene recognition result is greater than 0.5, the target video frame is determined to be a game interface scene; if it is not greater than 0.5, the target video frame is determined to be a non-game interface scene. Figure 2 , Figure 2 The hollow circles represent the distribution of the first scene recognition results of each image frame, and the thick solid lines represent the distribution of the first brightness enhancement peak value or the second brightness enhancement peak value used by each image frame. Figure 2 It can be seen that the solution provided by this embodiment effectively suppresses frequent switching of image enhancement modes.

[0072] In some possible implementations, the SDR2HDR AI model can be used to process the target image frame in step S140. The processing content includes increasing the brightness range to 0.01nits-1000nits, converting the color space from the BT.709 color gamut to the BT.2020 color gamut, converting the color depth from 8bit to 10bit, etc. In addition, the details of the darker areas of the standard dynamic range (SDR) video frame can be enhanced to enhance the brightness peak, and the contrast and saturation of the SDR video frame can be enhanced to enhance the color expression, thereby improving the overall visual effect of the SDR video.

[0073] Typically, the color depth of an image refers to the number of colors a display can display in its palette. The unit of color depth is bit. Common color depths are 8-bit and 10-bit. The higher the number, the more colors can be displayed, i.e., the richer the color. Typically, an 8-bit color depth will have 256 primary colors, which translates to 16,777,216 (i.e., 256 to the power of 3) colors. Similarly, a 10-bit single primary color has 1,024 colors, and a 10-bit three-primary color depth has 1,073,741,824 colors.

[0074] Furthermore, HDR-enhanced video data is typically stored in 10-bit memory. Therefore, after enhancing the image frame into an HDR image, the HDR image storage format must be met. In 8-bit YUV data, each pixel occupies 8 bits, which is stored byte by byte in memory. Similarly, 16-bit data uses two bytes per pixel. However, 10-bit data requires consideration of processing performance, which can be achieved through sacrificial compression, where each pixel occupies 16 bits (two bytes), with the trailing 6 bits being padding (0).

[0075] Furthermore, in this embodiment, since each frame of image enhancement processing depends on the second scene recognition result, to fully utilize the resources of the processing device (e.g., GPU), a number of threads equivalent to the number of GPU cores can be enabled to simultaneously process the scene recognition task and the image enhancement task. Semaphores and a waiting queue are also required to synchronize these two tasks.

[0076] Specifically, when performing a scene recognition task, the second scene recognition result of each video frame can be stored in a preset semantic information queue pool. When the image enhancement task requires the second scene recognition result of the current image frame, it is attempted to be obtained from the semantic information queue. If available, the image enhancement process is started directly; if not, the current image frame is added to the queue waiting for image enhancement processing. After the scene recognition task processes the current video frame, if the video frame is in the waiting queue, the image enhancement task is awakened. If an image frame in the queue waiting for image enhancement processing times out while waiting for the scene recognition task, the average confidence level of the 30 image frames before the image frame will be used as the second scene recognition result of the image frame to continue the image enhancement processing of the current frame.

[0077] Based on the same inventive concept, this embodiment also provides an electronic device for executing the image processing method provided in this embodiment. The electronic device can be an electronic device with data processing capabilities, such as a server, a personal computer, or a workstation.

[0078] Please refer to Figure 3 , is a block diagram of the electronic device 100. The electronic device 100 includes an image enhancement device 110, a machine-readable storage medium 120, and a processor 130.

[0079] The machine-readable storage medium 120, processor 130, and communication unit 140 are electrically connected to each other, directly or indirectly, to enable data transmission or interaction. For example, these components may be electrically connected to each other via one or more communication buses or signal lines. The image enhancement device 110 includes at least one software function module that can be stored in the machine-readable storage medium 120 in the form of software or firmware or embedded in the operating system (OS) of the electronic device 100. The processor 130 is configured to execute the executable modules stored in the machine-readable storage medium 120, such as the software function modules and computer programs included in the image enhancement device 110.

[0080] The machine-readable storage medium 120 may be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc. The machine-readable storage medium 120 is used to store a program, and the processor 130 executes the program after receiving an execution instruction.

[0081] The processor 130 may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a graphics processing unit (GPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The various methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0082] Please refer to Figure 4 This embodiment further provides an image enhancement device 110, which includes at least one functional module that can be stored in a machine-readable storage medium 120 in software form. Functionally, the image enhancement device 110 can include a first data acquisition module 111, a first type identification module 112, a second data acquisition module 113, a second type identification module 114, and an image enhancement processing module 115.

[0083] The first data acquisition module 111 is used to acquire a target video frame in a video stream to be processed.

[0084] In this embodiment, the first data acquisition module 111 can be used to perform Figure 1 As shown in step S110 , for a detailed description of the first data acquisition module 111 , please refer to the description of step S110 .

[0085] The first type recognition module 112 inputs the target video frame into a pre-trained scene recognition model for processing to obtain a first scene recognition result of the target video frame.

[0086] In this embodiment, the first type identification module 112 may be used to perform Figure 1 As shown in step S120 , for a detailed description of the first type identification module 112 , please refer to the description of step S120 .

[0087] The second data acquisition module 113 is configured to acquire first scene recognition results of a plurality of reference video frames preceding the target video frame.

[0088] In this embodiment, the second data acquisition module 113 can be used to perform Figure 1 As shown in step S130 , for a detailed description of the second data acquisition module 113 , please refer to the description of step S130 .

[0089] The second type identification module 114 is configured to determine a second scene recognition result of the target video frame according to the first scene recognition result of the target video frame and the first scene recognition results of the plurality of reference video frames.

[0090] In this embodiment, the second type identification module 114 can be used to perform Figure 1 As shown in step S140 , for a detailed description of the second type identification module 114 , please refer to the description of step S140 .

[0091] The image enhancement processing module 115 is configured to select a corresponding image enhancement processing method to perform image enhancement processing on the target video frame according to the second scene recognition result, so as to obtain a processed target video frame.

[0092] In this embodiment, the image enhancement processing module 115 can be used to perform Figure 1 As shown in step S150, for a detailed description of the image enhancement processing module 115, please refer to the description of step S150.

[0093] In summary, the video image processing method, device, and electronic device provided by the present application, when processing a target video frame of a video stream to be processed, determines a second scene recognition result for the target video frame by integrating the first scene recognition results of the target video frame and multiple reference video frames preceding the target video frame, and then selects a corresponding image enhancement processing method based on the second scene recognition result to perform image enhancement processing on the target video frame. In this way, a temporal accumulation feature is added to the scene recognition process of the target video frame, which can avoid frequent changes in the display effect of the processed video image due to frequent scene changes in the video stream to be processed.

[0094] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0095] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0096] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0097] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0098] The above descriptions are merely examples of various embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any modifications or substitutions that can be readily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included within the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A video image processing method, characterized in that: The method comprises: Get the target video frame in the video stream to be processed; Inputting the target video frame into a pre-trained scene recognition model for processing to obtain a first scene recognition result of the target video frame; Obtaining first scene recognition results of a plurality of reference video frames preceding the target video frame; determining a second scene recognition result of the target video frame based on the first scene recognition result of the target video frame and the first scene recognition results of the multiple reference video frames; wherein the second scene recognition result is a cumulative calculation result of the multiple first scene recognition results in the time domain; According to the second scene recognition result, a corresponding image enhancement processing method is selected to perform image enhancement processing on the target video frame to obtain a processed target video frame.

2. The method according to claim 1, characterized in that The step of inputting the target video frame into a pre-trained scene recognition model for processing to obtain a first scene recognition result of the target video frame includes: Inputting the target video frame into a pre-trained scene recognition model for processing, and obtaining a probability value that the target video frame belongs to a preset scene as the first scene recognition result; The step of determining the second scene recognition result of the target video frame according to the first scene recognition result of the target video frame and the first scene recognition results of the plurality of reference video frames comprises: An average value of the first scene recognition results of the target video frame and the plurality of reference video frames is calculated, and the average value is used as a second scene recognition result of the target video frame.

3. The method according to claim 1, characterized in that The step of inputting the target video frame into a pre-trained scene recognition model for processing to obtain a first scene recognition result of the target video frame includes: inputting the target video frame into a pre-trained scene recognition model for processing, and obtaining a probability value that the target video frame belongs to a preset scene as the first scene recognition result; The step of determining the second scene recognition result of the target video frame according to the first scene recognition result of the target video frame and the first scene recognition results of the plurality of reference video frames comprises: Calculating a weighted average of first scene recognition results of the target video frame and the multiple reference video frames based on a preset first weight parameter of the target video frame and a second weight parameter of the multiple reference video frames; wherein the first weight parameter is greater than the second weight parameter; The weighted average value is used as a second scene recognition result of the target video frame.

4. The method according to claim 1, wherein The step of selecting a corresponding image enhancement processing method to perform image enhancement processing on the target video frame according to the second scene recognition result includes: According to the second scene recognition result, a corresponding brightness enhancement peak is selected, and high dynamic range enhancement processing is performed on the target video frame.

5. The method according to claim 4, characterized in that The step of selecting a corresponding brightness enhancement peak according to the second scene recognition result and performing high dynamic range enhancement processing on the target video frame includes: Detecting whether different brightness enhancement peaks have been used to perform high dynamic range enhancement processing on the image of the video stream to be processed within a preset time period; If so, performing high dynamic range enhancement processing on the target video frame using the brightness enhancement peak value used when processing a previous frame of the target video frame; If not, performing high dynamic range enhancement processing on the target video frame using the brightness enhancement peak value corresponding to the second scene recognition result of the target video frame.

6. The method according to claim 4, characterized in that The step of selecting a corresponding brightness enhancement peak according to the second scene recognition result and performing high dynamic range enhancement processing on the target video frame includes: If the second scene recognition result indicates that the target video frame belongs to a non-game interface scene, performing high dynamic range enhancement processing on the target video frame using the first brightness enhancement peak value; If the second scene recognition result indicates that the target video frame belongs to a game interface scene, a second brightness enhancement peak is used to perform high dynamic range enhancement processing on the target video frame; wherein, the first brightness enhancement peak is smaller than the second brightness enhancement peak.

7. The method according to claim 4, characterized in that The step of selecting a corresponding brightness enhancement peak according to the second scene recognition result and performing high dynamic range enhancement processing on the target video frame includes: If the second scene recognition result indicates that the target video frame belongs to a non-game interface scene, performing high dynamic range enhancement processing on the target video frame using the first brightness enhancement peak value; If the second scene recognition result indicates that the target video frame belongs to a game interface scene, a second brightness enhancement peak is used to perform high dynamic range enhancement processing on the game screen area in the target video frame; and a third brightness enhancement peak is used to perform high dynamic range enhancement processing on the non-game screen area in the target video frame; wherein, the first brightness enhancement peak and the third brightness enhancement peak are smaller than the second brightness enhancement peak.

8. An image processing device, characterized in that The image processing device comprises: A first data acquisition module is used to acquire a target video frame in a video stream to be processed; a first type recognition module, inputting the target video frame into a pre-trained scene recognition model for processing to obtain a first scene recognition result of the target video frame; A second data acquisition module is used to obtain first scene recognition results of multiple reference video frames before the target video frame; a second-type recognition module, configured to determine a second scene recognition result of the target video frame based on the first scene recognition result of the target video frame and the first scene recognition results of the plurality of reference video frames; the second scene recognition result being a cumulative calculation result of the plurality of first scene recognition results in the time domain; The image enhancement processing module is used to select a corresponding image enhancement processing method to perform image enhancement processing on the target video frame according to the second scene recognition result to obtain a processed target video frame.

9. An electronic device, characterized in that: The method comprises a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A machine-readable storage medium, characterized in that The machine-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are executed by one or more processors, the method according to any one of claims 1 to 7 is implemented.

Images