Method, device, storage medium and processor for processing images in a video

Abstract

The application discloses a kind of processing method, device, storage medium and processor of image in video, wherein the method includes: show the original image to be processed, wherein the original image is the multimedia data collected in the process of playing video;High-frequency layer image in original image is identified;Strong light fusion is carried out to high-frequency layer image, and the preprocessed image of original image is obtained based on the generated strong light fusion result;The preprocessed image is fused with original image, and initial result image is obtained.The application solves the technical problem that the picture exists the difference in definition after the object in video is processed.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more specifically, to a method, apparatus, storage medium, and processor for processing images in a video. Background Technology

[0002] Currently, optimization tools are available to smooth the skin of the target object's face in a video.

[0003] However, when optimization tools perform skin smoothing on the target object's face, due to time and power consumption limitations in video scenes, they mostly choose simple filtering operations, lacking functions for removing blemishes and acne, and failing to adequately protect skin details and backgrounds. This results in the loss of information from the entire image, leading to technical issues such as poor image clarity after image processing.

[0004] There is currently no effective solution to the technical problem of poor image clarity after image processing of the objects in the aforementioned video. Summary of the Invention

[0005] This invention provides a method, apparatus, storage medium, and processor for processing images in a video, to at least solve the technical problem of poor image clarity after image processing of objects in a video.

[0006] According to one aspect of the present invention, a method for processing images in a video is provided. The method may include: displaying an original image to be processed, wherein the original image is multimedia data acquired during video playback; identifying high-frequency layer images in the original image; performing intensity fusion on the high-frequency layer images; and fusing the pre-processed image with the original image to obtain an initial result image.

[0007] According to another aspect of the present invention, another method for processing images in a video is also provided. The method may include: displaying the original image to be processed on a client's display interface, wherein the original image is multimedia data acquired during video playback; performing intensity fusion on the original image and obtaining a preprocessed image of the original image based on the generated intensity fusion result; fusing the preprocessed image with the original image to obtain an initial result image; and sharpening the skin mask in the initial result image to generate a target image, wherein a skin mask for the facial skin is generated based on key information of the face in the original image.

[0008] According to another aspect of the present invention, another method for processing images in a video is also provided. The method may include: displaying the original image to be processed on a client's display interface, wherein the original image is multimedia data acquired during video playback; responding to an image processing instruction received by the client, displaying the image processing result of executing the image processing instruction on the original image on the client's display interface, wherein the image processing instruction is used to perform intense light fusion on the high-frequency layer image of the original image; fusing the image processing result with the original image, and rendering the fused initial result image on the display interface.

[0009] According to another aspect of the present invention, another method for processing images in a video is also provided. The method may include: displaying the original image to be processed in the video on a display interface; determining the high-frequency layer image and the low-frequency layer image in the original image in response to a recognition operation on the original image; performing strong light fusion on the high-frequency layer image and obtaining a preprocessed image of the original image based on the generated strong light fusion result; fusing the preprocessed image with the original image; and rendering the initial result image obtained by fusion on the display interface.

[0010] According to another aspect of the present invention, an image processing apparatus for a video is also provided. The apparatus may include: a first display unit for displaying an original image to be processed, wherein the original image is multimedia data acquired during video playback; a recognition unit for recognizing a high-frequency layer image in the original image; a first processing unit for performing strong light fusion on the high-frequency layer image and obtaining a preprocessed image of the original image based on the generated strong light fusion result; and a first fusion unit for fusing the preprocessed image with the original image to obtain an initial result image.

[0011] According to another aspect of the present invention, another image processing apparatus for video is also provided. The apparatus may further include: a second display unit for displaying the original image to be processed on a client's display interface, wherein the original image is multimedia data acquired during video playback; a second processing unit for performing intensity fusion on the original image and obtaining a preprocessed image of the original image based on the generated intensity fusion result; a second fusion unit for fusing the preprocessed image with the original image to obtain an initial result image; a third processing unit for performing smooth step filtering on the intensity fusion result to generate a filtered image, and brightening the dark areas in the initial result image to generate a secondary brightened image; and a third fusion unit for fusing the initial result image, the filtered image, and the secondary brightened image to generate a secondary result image, and applying a skin mask to the secondary result image, wherein a skin mask for facial skin is generated based on key information of the face in the original image.

[0012] According to another aspect of the present invention, another processing apparatus for images in a video is also provided. The apparatus may further include: a third display unit for displaying the original image to be processed on a client's display interface, wherein the original image is multimedia data acquired during video playback; a first display unit for responding to an image processing instruction received by the client and displaying the image processing result of executing the image processing instruction on the original image on the client's display interface, wherein the image processing instruction is used to perform strong light fusion on the high-frequency layer image of the original image; and a fourth fusion unit for fusing the image processing result with the original image and rendering the fused initial result image on the display interface.

[0013] According to another aspect of the present invention, another processing apparatus for images in a video is also provided. The apparatus may further include: a second display unit for displaying the original image to be processed in the video on a display interface; a determining unit for determining a high-frequency layer image and a low-frequency layer image in the original image in response to a recognition operation on the original image; a fourth processing unit for performing strong light fusion on the high-frequency layer image and obtaining a preprocessed image of the original image based on the generated strong light fusion result; a fifth fusion unit for fusing the preprocessed image with the original image; and a rendering unit for rendering the initial result image obtained by fusion on the display interface.

[0014] According to another aspect of the present invention, a computer-readable storage medium is also provided. The computer-readable storage medium includes a stored program, wherein, when the program is run by a processor, it controls the device where the computer-readable storage medium is located to execute the method for processing images in a video according to the embodiments of the present invention.

[0015] According to another aspect of the present invention, a processor is also provided. The processor is used to run a program, wherein the program, when run by the processor, performs the method for processing images in a video according to the embodiments of the present invention.

[0016] According to another aspect of the present invention, a system for processing images in a video is also provided. The system includes: a processor; and a memory connected to the processor, configured to provide the processor with instructions to perform the following processing steps: displaying the original image to be processed, wherein the original image is multimedia data acquired during video playback; identifying high-frequency layer images in the original image; performing strong light fusion on the high-frequency layer images, and obtaining a preprocessed image of the original image based on the generated strong light fusion result; and fusing the preprocessed image with the original image to obtain an initial result image.

[0017] In the video image processing method of this embodiment, the original image to be processed is displayed, wherein the original image is multimedia data collected during video playback; high-frequency layer images in the original image are identified; strong light fusion is performed on the high-frequency layer images, and a pre-processed image of the original image is obtained based on the generated strong light fusion result; the pre-processed image is fused with the original image to obtain an initial result image. This application identifies high-frequency layer images from the original image, performs strong light fusion on the high-frequency layer images, brightens the original image, and then fuses it with the original image. This achieves the goal of preserving details and color uniformity while smoothing the skin of the original image, making the image appear clean and transparent, improving the texture and clarity of the image, and avoiding the loss of information of the entire image during the skin smoothing process of the optimization tool. This solves the technical problem of poor image clarity after image processing of objects in the video, and achieves the technical effect of improving the image clarity of objects in the video after image processing. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0019] Figure 1 This is a hardware structure block diagram of a computer terminal (or mobile device) for implementing a method for processing images in a video according to an embodiment of the present invention.

[0020] Figure 2 This is a flowchart of a method for processing images in a video according to an embodiment of the present invention;

[0021] Figure 3 This is a flowchart of another method for processing images in a video according to an embodiment of the present invention;

[0022] Figure 4 This is a flowchart of another method for processing images in a video according to an embodiment of the present invention;

[0023] Figure 5 This is a flowchart of a method for processing images in a video according to an embodiment of the present invention;

[0024] Figure 6 This is a schematic diagram of a skin smoothing algorithm applicable to real-time video streams according to an embodiment of the present invention;

[0025] Figure 7 This is a scene diagram illustrating a method for processing images in a video according to an embodiment of the present invention;

[0026] Figure 8This is a schematic diagram of a scene illustrating another method for processing images in a video according to an embodiment of the present invention;

[0027] Figure 9 This is a comparative schematic diagram of the processing results of images in a video according to an embodiment of the present invention;

[0028] Figure 10 This is a schematic diagram of an image processing apparatus in a video according to an embodiment of the present invention;

[0029] Figure 11 This is a schematic diagram of another video image processing apparatus according to an embodiment of the present invention;

[0030] Figure 12 This is a schematic diagram of another video image processing apparatus according to an embodiment of the present invention;

[0031] Figure 13 This is a schematic diagram of another video image processing apparatus according to an embodiment of the present invention;

[0032] Figure 14 This is a structural block diagram of a computer terminal according to an embodiment of the present invention. Detailed Implementation

[0033] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0034] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0035] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows:

[0036] Edge-preserving filters are a special type of filter that can effectively preserve edge information in an image during the filtering process.

[0037] The high-contrast filter is a filtering and skin-smoothing algorithm based on high-contrast preservation.

[0038] Gaussian filtering is a linear smoothing filter suitable for eliminating Gaussian noise and is used in the noise reduction process of image processing.

[0039] Skin mask, also known as skin area mask, determines the area of ​​skin that is displayed;

[0040] Dark areas of the skin refer to areas on the skin where blemishes such as pimples, freckles, and pores are located. These areas have lower pixel values ​​than the surrounding pixels.

[0041] Adjusting the curves of an image is one way to adjust its brightness.

[0042] Example 1

[0043] According to an embodiment of the present invention, an embodiment of a method for processing images in a video is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0044] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 This is a hardware structure block diagram of a computer terminal (or mobile device) for implementing a method for processing images in a video, according to an embodiment of the present invention. Figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0045] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0046] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the video image processing method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implementing the video image processing method of the above-mentioned application. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0047] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0048] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0049] It should be noted here that, in some optional embodiments, the above... Figure 1The computer device (or mobile device) shown may include hardware elements (including circuitry), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. It should be noted that... Figure 1 This is only one instance of a specific particular instance and is intended to illustrate the types of components that may exist in the aforementioned computer device (or mobile device).

[0050] exist Figure 1 Under the operating environment shown, this application provides the following: Figure 2 The method for processing images in the video shown is illustrated. It should be noted that the method for processing images in the video in this embodiment can be derived from… Figure 1 The mobile terminal of the illustrated embodiment is executed.

[0051] Figure 2 This is a flowchart of a method for processing images in a video according to an embodiment of the present invention. Figure 2 As shown, the method may include the following steps:

[0052] Step S202: Display the original image to be processed, wherein the original image is multimedia data acquired during video playback.

[0053] In the technical solution provided in step S202 of the present invention, the original image to be processed may include the portrait of the target object to be processed, such as a facial image, which can be displayed on the client's display interface. The original image may be multimedia data collected during video playback on the client in a video scene. For example, the multimedia data may include video image data, where the video may be a real-time video stream, and the video scene may be a scene of playing live video, short video, or recorded video on the client, without specific limitations.

[0054] Step S204: Identify the high-frequency layer image in the original image.

[0055] In the technical solution provided by step S204 of the present invention, after displaying the original image to be processed, the high-frequency layer image in the original image can be identified.

[0056] In this embodiment, the original image can be layered to identify the high-frequency layer image in the original image. The high-frequency layer image can be an image of the high-frequency components in the original image.

[0057] Optionally, this embodiment can identify the high-frequency components in the original image and further form the high-frequency layer image using the images of these high-frequency components.

[0058] Step S206: Perform strong light fusion on the high-frequency layer image, and obtain a preprocessed image of the original image based on the generated strong light fusion result.

[0059] In the technical solution provided by step S206 of the present invention, after identifying the high-frequency layer image in the original image, strong light fusion is performed on the high-frequency layer image, and a preprocessed image of the original image is obtained based on the generated strong light fusion result.

[0060] In this embodiment, strong light fusion can be performed on the high-frequency layer image to selectively filter out imperfections in the high-frequency layer image, thereby preserving the detailed features in the high-frequency layer image. Imperfections can be spots or pimples on a person's face, and detailed features can be skin details, thereby obtaining a filtered image. This filtered image can be the result of strong light fusion, and then a preprocessed image of the original image can be obtained based on the generated strong light fusion result.

[0061] Optionally, this embodiment performs strong light fusion on the high-frequency layer image using (A<0.5f)? (2*A*A):(1.0f-2*(1.0fA)*(1.0fA)), where A can be used to represent the pixel value, f can be used to represent the data corresponding to the high-frequency layer image during strong light fusion, and the result of strong light fusion on the high-frequency layer image can be represented by mask2.

[0062] Step S208: The preprocessed image is fused with the original image to obtain the initial result image.

[0063] In the technical solution provided by step S208 of the present invention, after performing strong light fusion on the high-frequency layer image and obtaining a preprocessed image of the original image based on the generated strong light fusion result, the preprocessed image can be fused with the original image to obtain an initial result image.

[0064] In this embodiment, when fusing the preprocessed image and the original image, the fusing can be done using certain parameters. For example, the pixels of the preprocessed image and the pixels of the original image can be fused using certain parameters to obtain the pixels of the initial result image. This initial result image can be represented by ret1, where the aforementioned certain parameters can be skin smoothing intensity parameters. Optionally, the pixel values ​​of the initial result image piecelC = piecelA * alpha + piecelB * (1.0 - alpha), where piecelA can be used to represent the pixels of the preprocessed image, piecelB can be used to represent the pixels of the original image, and alpha is between [0, 1].

[0065] Through steps S202 to S208 of this application, the original image to be processed is displayed, wherein the original image is multimedia data collected during video playback; high-frequency layer images in the original image are identified; strong light fusion is performed on the high-frequency layer images, and a pre-processed image of the original image is obtained based on the generated strong light fusion result; the pre-processed image is fused with the original image to obtain an initial result image. In other words, this application identifies high-frequency layer images from the original image, performs strong light fusion on the high-frequency layer images, brightens the original image, and then fuses it with the original image. This achieves the goal of preserving details and color uniformity while smoothing the skin of the original image, resulting in a clean and transparent image, improving image texture and clarity, and avoiding the loss of information in the entire image during the smoothing process by optimization tools. This solves the technical problem of poor image clarity after image processing of objects in the video, achieving the technical effect of improving the image clarity of objects in the video after image processing.

[0066] The method described in this embodiment will be further described below.

[0067] As an optional implementation, step S206, obtaining a preprocessed image of the original image based on the generated strong light fusion result, includes: brightening the original image to obtain a first brightened image; and generating a preprocessed image based on the strong light fusion result and the first brightened image.

[0068] Optionally, in addition to strong light fusion of the high-frequency layer image, the original image can also be brightened. This can be done by adjusting the curve of the original image and then brightening it to obtain a first-level brightened image. For example, the pixel values ​​of the dark areas of the original image can be stretched using a curve to achieve the effect of acne removal and skin cleansing, resulting in a first-level brightened image. This first-level brightened image can be represented by lut1(image). Thus, the above method can preserve skin details and ensure uniform skin tone, making the skin appear clean and translucent. The curve can be an S-curve, and curve adjustment is a method of image brightness adjustment.

[0069] This embodiment can determine the image after strong light fusion and the image after one brightening as the preprocessed image of the original image.

[0070] As an optional implementation, brightening the original image to obtain a first-brightened image includes: brightening the low-frequency layer image of the original image to obtain a first-brightened image.

[0071] In this embodiment, when brightening the original image, the low-frequency layer image identified from the original image can be brightened to obtain a brightened image. Alternatively, the low-frequency layer image can be adjusted by curve adjustment to achieve the purpose of brightening the low-frequency layer image, thereby preserving skin details and skin tone uniformity while performing skin smoothing.

[0072] As an optional implementation, step S204, identifying the high-frequency layer image in the original image, includes: performing Gaussian filtering on the G-channel grayscale image of the original image to obtain the low-frequency layer image of the original image; and calculating a high-contrast layer based on the G-channel grayscale image and the low-frequency layer image to obtain the high-frequency layer image.

[0073] In this embodiment, when identifying the high-frequency layer image in the original image, the G-channel grayscale image of the original image can be obtained first. This G-channel grayscale image can also be called the G-channel image. Gaussian filtering is then applied to it, and the resulting image is the low-frequency layer image, which can be represented by gauss(G). After applying Gaussian filtering to the G-channel grayscale image of the original image to obtain the low-frequency layer image, a high-contrast layer can be calculated based on the G-channel grayscale image and the low-frequency layer image, thus obtaining the high-frequency layer image. Optionally, in this embodiment, the high-contrast layer is the sum of the difference between the G-channel grayscale image and the low-frequency layer image and 128, that is, high-contrast layer hp = G - gauss(G) + 128.

[0074] As an optional implementation, step S206, performing strong light fusion on the high-frequency layer image, includes: performing strong light fusion on the high-frequency layer image to obtain a filtered result after filtering out defects in the high-frequency layer image.

[0075] In this embodiment, strong light fusion is performed on the high-frequency layer image (high-contrast layer) to obtain a filtered result that targets the defects in the high-frequency layer image.

[0076] As an optional implementation, in step S208, during the process of fusing the preprocessed image with the original image, the level of the initial result image is adjusted based on the received parameters.

[0077] This embodiment can receive parameters when acquiring the original image to be processed. These parameters can be skin smoothing intensity parameters, for example, represented by alpha. In the process of fusing the preprocessed image and the original image, the received parameters can be used to fuse the preprocessed image and the original image to obtain an initial result image. The received parameters affect the level of the initial result image. Therefore, in the process of fusing the preprocessed image and the original image, the level of the initial result image can be adjusted based on the received parameters. This level can also be called the skin smoothing level.

[0078] As an optional implementation, after obtaining the initial result image in step S208, the method further includes: performing a smooth step filter on the strong light fusion result to generate a filtered image; brightening the dark areas in the initial result image to generate a secondary brightened image; and fusing the initial result image, the filtered image, and the secondary brightened image to generate a secondary result image.

[0079] In this embodiment, after obtaining the initial result image, the generated strong light fusion result can be smoothed with a step filter, for example, by smoothing and step filtering mask2 to generate a filtered image, which can be represented by mask3. Dark areas in the initial result image can also be identified and then brightened, for example, by curve adjustment, such as curve stretching of the dark area. That is, the pixel values ​​of the dark area are calculated according to a curve formula to obtain the calculated pixel values, thereby generating a secondary brightened image. This secondary brightened image can be simultaneously represented by lut2 (ret1), achieving acne removal and moisturizing effects on the face. After generating the filtered image and the secondary brightened image, this embodiment can merge the initial result image, the filtered image, and the secondary brightened image. This can be done by fusing the pixels of the initial result image, the filtered image, and the secondary brightened image using certain parameters to generate a secondary result image, which can be represented by merge.

[0080] As an optional implementation, after displaying the original image to be processed in step S202, the method further includes: detecting key facial information from the original image based on facial features; and generating a skin mask for the facial skin based on the key facial information.

[0081] In this embodiment, after displaying the original image to be processed, facial features can be determined from the original image, and then key facial information can be detected from the original image based on these facial features. That is, face detection is performed on the original image. The aforementioned key facial information can be the key facial points obtained from the face detection. Then, a skin mask for the facial skin is generated based on the key facial information. This skin mask can be a facial skin region mask mask1. The pixels of the preprocessed image, the pixels of the original image, and the region mask are fused together using certain parameters to obtain the pixels of the initial result image.

[0082] As an optional implementation, after generating the secondary result image, the method further includes: sharpening the secondary result image using a skin mask to generate a target image; and outputting the target image.

[0083] In this embodiment, after generating the secondary result image, the secondary result image can be sharpened using a skin mask, while the skin mask and the original image do not need to be sharpened, thereby generating the target image, and then outputting the target image, which can be represented by out.

[0084] As an optional implementation, image sharpening of the secondary result image using a skin mask to generate a target image includes: using a skin mask and different preset adjustment parameters to sharpen the face region in the secondary result image to generate a target image, wherein the background region in the secondary result image remains unchanged, or the sharpening level of the background region is lower than the sharpening level of the face region.

[0085] In this embodiment, when using a skin mask to sharpen the secondary result image and generate the target image, the skin mask and different preset adjustment parameters can be used to sharpen the face region in the secondary result image. The different preset adjustment parameters can be skin smoothing intensity parameters, which are used to sharpen the face region in the secondary result image while the background region in the secondary result image remains unchanged, or the sharpening level of the background region is lower than that of the face region. That is, the background region remains unchanged or is slightly sharpened, which can compensate for the problem of more blurred highlights introduced by filtering, thereby achieving the purpose of making the image clearer.

[0086] In this embodiment, image sharpening can be performed by overlaying gradient maps, where the gradient map is the difference between the center pixel and the surrounding pixels. This embodiment sharpens the image and the face separately. Here, a skin mask can be used to distinguish the face area from the background, and different skin smoothing parameters or methods are used to sharpen them respectively.

[0087] This invention also provides another method for processing images in a video.

[0088] Figure 3 This is a flowchart of another method for processing images in a video according to an embodiment of the present invention. Figure 3 As shown, the method may include:

[0089] Step S302: Display the original image to be processed on the client's display interface, wherein the original image is multimedia data acquired during video playback.

[0090] In the technical solution provided in step S302 of the present invention, the original image to be processed is acquired, for example, by an image acquisition device. This original image can then be further displayed on the client's display interface. The original image to be processed may include the image of the target object to be processed, such as a facial image. The original image may also be multimedia data collected during video playback on the client in a video scene. For example, the multimedia data may include video image data. The video scene may be a scenario where live video, short video, or recorded video is played on the client.

[0091] Step S304: Perform strong light fusion on the original image and obtain a preprocessed image of the original image based on the generated strong light fusion result.

[0092] In the technical solution provided by step S304 of the present invention, after the original image to be processed is displayed on the client's display interface, strong light fusion can be performed on the original image, and a preprocessed image of the original image can be obtained based on the generated strong light fusion result.

[0093] In this embodiment, the original image can be layered to identify the high-frequency layer image, which can be an image of the high-frequency components in the original image. This embodiment can then perform strong light fusion on the high-frequency layer image to selectively filter imperfections while preserving detailed features. Imperfections include blemishes and pimples on a person's face, while detailed features can be skin details. Based on the generated strong light fusion result, a preprocessed image of the original image is obtained.

[0094] Optionally, this embodiment can further brighten the original image by adjusting the curve of the original image and then brightening the original image after curve adjustment to obtain a brightened image, thereby preserving skin details and ensuring uniform skin tone, making the skin appear clean and translucent.

[0095] This embodiment can determine the image after strong light fusion and the image after brightening as the preprocessed image of the original image.

[0096] Step S306: Fuse the preprocessed image with the original image to obtain the initial result image.

[0097] In the technical solution provided by step S306 of the present invention, after performing strong light fusion on the original image and obtaining a preprocessed image of the original image based on the generated strong light fusion result, the preprocessed image can be fused with the original image to obtain an initial result image.

[0098] This embodiment can be achieved by fusing the pixels of the preprocessed image with the pixels of the original image using certain parameters, thereby obtaining the pixels of the initial result image.

[0099] Step S308: Perform smooth step filtering on the strong light fusion result to generate a filtered image, and brighten the dark areas in the initial result image to generate a secondary brightened image.

[0100] In the technical solution provided by step S308 of the present invention, after obtaining the initial result image, the generated strong light fusion result can be smoothed by step filtering to generate a filtered image; the dark areas in the initial result image can also be determined, and then the dark areas can be brightened, for example, by adjusting the curve of the dark areas, such as by stretching the curve of the dark areas, to obtain the calculated pixel values, and then generating a secondary brightened image through the calculated pixel values, which can achieve acne removal and moisturizing effects on the face.

[0101] Step S310: The initial result image, the filtered image, and the secondary brightening image are fused to generate a secondary result image, and a skin mask is used to sharpen the secondary result image to generate the target image.

[0102] In the technical solution provided by step S310 of the present invention, after brightening the dark areas in the initial result image to generate a secondary brightened image, the initial result image, the filtered image, and the secondary brightened image can be fused to generate a secondary result image. A skin mask is then used to sharpen the secondary result image to generate the target image. A skin mask for the facial skin is generated based on key information about the face in the original image. After generating the secondary result image, the skin mask can be used to sharpen the secondary result image, while the skin mask and the original image do not require sharpening, thereby generating the target image and outputting it.

[0103] This invention also provides another method for processing images in a video.

[0104] Figure 4 This is a flowchart of another method for processing images in a video according to an embodiment of the present invention. Figure 4 As shown, the method may include the following steps:

[0105] Step S402: Display the original image to be processed on the client's display interface, wherein the original image is multimedia data acquired during video playback.

[0106] In the technical solution provided by step S402 of the present invention, the original image to be processed is acquired by an image acquisition device and displayed on the client's display interface. The original image to be processed may include the image of the target object to be processed, such as a facial image. The original image may be multimedia data acquired during video playback on the client in a video scene.

[0107] Step S404: In response to the image processing instruction received by the client, display the image processing result of the image processing instruction on the original image on the client's display interface.

[0108] In the technical solution provided by step S404 of the present invention, after displaying the original image to be processed on the client's display interface, the image processing result of executing the image processing instruction on the original image is displayed on the client's display interface in response to the image processing instruction received by the client. The image processing instruction is used to perform strong light fusion on the high-frequency layer image of the original image, and can also brighten the original image.

[0109] In this embodiment, the user can trigger an image processing command on the client's display interface. The client receives the image processing command and then responds to it to perform strong light fusion on the high-frequency layer image of the original image, and can further brighten the original image.

[0110] This embodiment performs strong light fusion on the high-frequency layer image to achieve targeted filtering of imperfections in the high-frequency layer image, thereby preserving the detailed features in the high-frequency layer image and obtaining a filtered image. Optionally, this embodiment brightens the original image, which can be done by adjusting the curve of the original image and then brightening the original image after curve adjustment, thereby obtaining a brightened image, which can preserve skin details and ensure uniform skin tone, making the skin appear clean and translucent.

[0111] In this embodiment, the image after strong light fusion and the image after brightening are determined as the preprocessed image of the original image. The preprocessed image can be the image processing result of performing image processing instructions on the original image, and then it is displayed on the client's display interface.

[0112] Step S406: The image processing result is fused with the original image, and the fused initial result image is rendered on the display interface.

[0113] In the technical solution provided by step S406 of the present invention, after displaying the image processing result of the image processing instruction executed on the original image on the client's display interface, the image processing result can be fused with the original image, and the initial result image obtained by fusion can be rendered on the display interface.

[0114] In this embodiment, when fusing the image processing result with the original image, the image processing result and the original image can be fused using certain parameters. Alternatively, the pixels of the image processing result and the pixels of the original image can be fused using certain parameters to obtain the pixels of the initial result image. Then, the initial result image is rendered and displayed on the client's display interface.

[0115] This invention also provides another method for processing images in a video.

[0116] Figure 5 This is a flowchart of a method for processing images in a video according to an embodiment of the present invention. Figure 5 As shown, the method may include the following steps:

[0117] Step S502: Display the original image to be processed in the video on the display interface.

[0118] In the technical solution provided in step S502 of the present invention, the video can be a live video played on the client, a short video played, a recorded video, etc. This embodiment can obtain the original image to be processed from the video, which may include the image of the target object to be processed, such as a facial image, and then display the original image to be processed in the video on the client's display interface.

[0119] Step S504: In response to the recognition operation on the original image, the high-frequency layer image in the original image is determined.

[0120] In the technical solution provided by step S504 of the present invention, after the original image to be processed in the video is displayed in the display interface, the high-frequency layer image in the original image can be determined in response to the recognition operation of the original image.

[0121] In this embodiment, the skin smoothing operation on the original image can be a skin smoothing-related operation performed by the user on the client's display interface. The skin smoothing operation can include a recognition operation, which can identify the high-frequency layer image in the original image in response to the recognition operation on the original image.

[0122] Optionally, this embodiment can identify the high-frequency components in the original image and further form a high-frequency layer image from the images of these high-frequency components.

[0123] Step S506: Perform strong light fusion on the high-frequency layer image, and obtain a preprocessed image of the original image based on the generated strong light fusion result.

[0124] In the technical solution provided by step S506 of the present invention, after determining the high-frequency layer image and the low-frequency layer image in the original image, strong light fusion can be performed on the high-frequency layer image, and a preprocessed image of the original image can be obtained based on the generated strong light fusion result.

[0125] In this embodiment, strong light fusion is performed on the high-frequency layer image to achieve targeted filtering of defects in the high-frequency layer image, thereby preserving the detailed features in the high-frequency layer image, and obtaining a preprocessed image of the original image based on the generated strong light fusion result.

[0126] Optionally, this embodiment further brightens the original image by adjusting the curve of the original image and then brightening the original image after curve adjustment to obtain a brightened image, thereby preserving skin details and ensuring uniform skin tone, making the skin appear clean and translucent.

[0127] This embodiment can determine the image after strong light fusion and the image after brightening as the preprocessed image of the original image.

[0128] Step S508: Fuse the preprocessed image with the original image.

[0129] In the technical solution provided by step S508 of the present invention, after performing strong light fusion on the high-frequency layer image and obtaining a preprocessed image of the original image based on the generated strong light fusion result, the preprocessed image can be fused with the original image.

[0130] In this embodiment, when fusing the preprocessed image and the original image, the preprocessed image and the original image can be fused using certain parameters to obtain the initial result image. Alternatively, the pixels of the preprocessed image and the pixels of the original image can be fused using certain parameters to obtain the pixels of the initial result image. The certain parameters can be skin smoothing intensity parameters.

[0131] Step S510: Render the initial result image obtained by fusion in the display interface.

[0132] In the technical solution provided by step S510 of the present invention, after the preprocessed image and the original image are fused, the initial result image obtained by fusion can be rendered in the display interface.

[0133] In this embodiment, the initial result image can be rendered, and the image result obtained by rendering the initial result image can be displayed on the client's display interface, that is, the target image can be displayed.

[0134] In this embodiment, when rendering the initial result image, the dark areas in the initial result image may be brightened to generate a secondary brightened image, the strong light fusion result may be smoothed by step filtering to generate a filtered image, the initial result image, the filtered image and the secondary brightened image may be fused together, and a skin mask may be used to sharpen the secondary result image to generate the target image.

[0135] The image processing method in this embodiment can be applied to portrait beautification in live streaming, conference, mobile camera, and short video applications. It divides the original image into high-frequency and low-frequency layers, allowing for targeted filtering of imperfections in the high-frequency layer and curve adjustment of the low-frequency layer. This achieves skin smoothing while preserving skin details and even skin tone, resulting in clean and clear skin after filtering. This embodiment can also detect dark areas in the skin and combine curve adjustments to achieve acne removal and hydration effects. Furthermore, this embodiment can combine a skin mask to sharpen both the original and initial result images separately, compensating for the increased blurring caused by filtering and making the image clearer. This avoids the loss of overall image information during the skin smoothing process, thus solving the technical problem of poor image clarity after image processing of video objects, achieving the technical effect of improving image clarity after image processing of video objects.

[0136] Example 2

[0137] The technical solution of the present invention will be illustrated below with reference to preferred embodiments.

[0138] With the rise of live streaming and short video industries, portrait beautification technology has been widely used, with skin smoothing being the most important function. However, due to time and power consumption limitations in video scenarios, skin smoothing often involves simple filtering operations, such as using edge-preserving filters (bilateral filters, guided filters, etc.) or high-contrast filters. These two methods result in the loss of skin details and textures (such as pores and facial contours), uneven skin tone, and a lack of blemish and acne removal capabilities, making the skin appear unclean. They also fail to adequately protect skin details and background, leading to a blurry and unclear image. The processed skin may be smooth, but it lacks hydration and texture.

[0139] This embodiment proposes a new skin smoothing algorithm suitable for real-time video streams, which can beautify the skin while preserving the skin's detailed information, thereby improving skin texture and portrait clarity.

[0140] Figure 6This is a schematic diagram of a skin smoothing algorithm applicable to real-time video streams according to an embodiment of the present invention. Figure 6 As shown, this method involves the following processing steps:

[0141] S1, input the original image and the skin smoothing intensity parameter alpha.

[0142] S2 performs face detection on the input original image to obtain facial key points, and then draws the facial skin region mask mask1 based on the facial key points.

[0143] S3. Obtain the G channel image of the input original image, and perform Gaussian filtering on the G channel image to obtain gauss(G). Then calculate the high contrast layer hp = G - gauss(G) + 128.

[0144] S4, perform strong light fusion on the high contrast layer to obtain mask2.

[0145] Alternatively, this embodiment can perform intense light fusion in the following manner:

[0146] (A<0.5f)? (2*A*A):(1.0f-2*(1.0fA)*(1.0fA)), where A represents the pixel value and f represents the data corresponding to the high-contrast layer.

[0147] S5 performs curve adjustments on the input original image to obtain the brightened image lut1(image).

[0148] In this embodiment, the curve can be an S-curve, which is a way to adjust image brightness.

[0149] S6 performs image fusion on the input original image, mask2 after strong light fusion, lut1 after curve adjustment, and input parameters to obtain the filtered image ret1.

[0150] S7. Perform a smoothing step filter on the result mask2 from S4 to obtain the filtered result mask3.

[0151] S8, stretch the curve of the result ret1 from S6 to obtain the brightened image lut2(ret1).

[0152] In this embodiment, when performing curve stretching, the pixel-by-pixel values ​​of the image can be calculated and transformed according to the curve formula.

[0153] S9. Merge the image ret1 from S6, the filtered result mask3 from S7, and the image lut2 from S8 to obtain the merged result merge.

[0154] S10 sharpens the image by combining the result mask1 from S2, the input parameters, and the merge from S9, resulting in the output image out.

[0155] In this embodiment, sharpening can be performed by overlaying gradient maps, where the gradient map is the difference between the center pixel and the surrounding pixels. This embodiment can sharpen the image and the face separately. Here, a skin mask is used to distinguish the facial skin from the background, and different skin smoothing parameters or methods are used to sharpen them respectively.

[0156] Figure 7 This is a schematic diagram illustrating a method for processing images in a video according to an embodiment of the present invention. Figure 7 As shown, the raw image to be processed is input into the computing device, and the raw image can be further displayed on the display interface of the computing device. The raw image to be processed may include the image of the target object to be processed, such as a facial image. The raw image may be multimedia data collected in a video scene during video playback, such as video image data. The video scene may be a scene of playing live video, short video, or recorded video on the client.

[0157] In this embodiment, the computing device can identify the high-frequency layer image from the original image. The computing device in this embodiment can then perform intensity fusion on the high-frequency layer image to selectively filter imperfections while preserving detailed features. Optionally, the computing device in this embodiment can brighten the original image by adjusting its curve and then brightening the low-frequency layer image after curve adjustment to obtain a brightened image.

[0158] The computing device in this embodiment can determine the image after strong light fusion and the image after brightening as the preprocessed image of the original image.

[0159] The computing device in this embodiment can fuse the pixels of the preprocessed image and the pixels of the original image using certain parameters to obtain the pixels of the initial result image.

[0160] After obtaining the initial result image, a smooth step filter can be applied to the strong light fusion result to generate a filtered image. The dark areas in the initial result image are then brightened to generate a secondary brightened image. The initial result image, the filtered image, and the secondary brightened image are then fused to generate a secondary result image. Finally, a skin mask is used to sharpen the secondary result image to generate the target image, which can be displayed on the display interface of the computing device.

[0161] Figure 8This is a scene illustration of another method for processing images in a video according to an embodiment of the present invention. Figure 8 As shown, the original image to be processed from the video is added and displayed in the display interface. This video can be a live stream, a short video, a recorded video, etc., played on the client. After displaying the original image to be processed in the display interface, the high-frequency and low-frequency layers in the original image can be identified in response to the recognition operation on the original image.

[0162] In this embodiment, the skin smoothing operation on the original image can be a skin smoothing-related operation performed by the user on the client's display interface. The skin smoothing operation may include a recognition operation, which can identify the high-frequency layer image in the original image in response to the recognition operation on the original image. Strong light fusion is performed on the high-frequency layer image, and a preprocessed image of the original image is obtained based on the generated strong light fusion result. Then, the preprocessed image is fused with the original image, and the fused initial result image is rendered on the display interface. After obtaining the initial result image, a smooth step filter can be applied to the strong light fusion result to generate a filtered image. The dark areas in the initial result image are then brightened to generate a secondary brightened image. The initial skin image, the filtered image, and the secondary brightened image are then fused to generate a secondary result image. A skin mask is used to sharpen the secondary result image to generate the target image, which is then displayed on the display interface.

[0163] Figure 9 This is a comparative schematic diagram showing the processing results of images in a video according to an embodiment of the present invention. For example... Figure 9 As shown, where, Figure 9 The rightmost image is the original image, but the skin texture is poor. Figure 9 The leftmost image is the effect of the initial result image of this embodiment, with relatively good skin texture. Figure 9 The rightmost image is slightly better. Figure 9 The image in the middle is the result of the target image obtained from the initial rendering and fusion image. It preserves the skin's details, improves skin texture and portrait clarity, and makes the skin appear clean, clear, and moisturized.

[0164] As described above, this embodiment performs targeted filtering on defects in high-frequency layer images and curve adjustments on low-frequency layer images to achieve skin smoothing while preserving skin details and skin tone uniformity, resulting in clean and clear skin after filtering. This embodiment can detect dark areas in the skin and combine curve adjustments to achieve acne removal and moisturizing effects. This embodiment can also combine skin masks to sharpen the original image and the initial result image separately, compensating for the blurring caused by filtering, making the image clearer, and avoiding the loss of information of the entire image during the skin smoothing process by the optimization tool. This solves the technical problem of poor image clarity after image processing of objects in the video, achieving the technical effect of improving the image clarity of objects in the video after image processing.

[0165] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0166] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0167] Example 3

[0168] According to embodiments of the present invention, a method for implementing the above is also provided. Figure 2 The method for processing images in a video shown is a video image processing device.

[0169] Figure 10 This is a schematic diagram of an image processing apparatus for a video according to an embodiment of the present invention. Figure 10 As shown, the image processing device 100 in the video may include: a first display unit 101, a recognition unit 102, a first processing unit 103, and a first fusion unit 104.

[0170] The first display unit 101 is used to display the original image to be processed, wherein the original image is multimedia data acquired during video playback.

[0171] The recognition unit 102 is used to identify the high-frequency layer image in the original image.

[0172] The first processing unit 103 is used to perform strong light fusion on the high-frequency layer image and obtain a preprocessed image of the original image based on the generated strong light fusion result.

[0173] The first fusion unit 104 is used to fuse the preprocessed image with the original image to obtain an initial result image.

[0174] It should be noted that the first display unit 101, the identification unit 102, the first processing unit 103, and the first fusion unit 104 mentioned above correspond to steps S202 to S208 of Embodiment 1, respectively. The four units and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above units, as part of the device, can run in the computer terminal 10 provided in Embodiment 1.

[0175] According to embodiments of the present invention, a method for implementing the above is also provided. Figure 3 The method for processing images in a video shown is a video image processing device.

[0176] Figure 11 This is a schematic diagram of another video image processing apparatus according to an embodiment of the present invention. Figure 11 As shown, the image processing device 110 in the video may include: a second display unit 111, a second processing unit 112, a second fusion unit 113, a third processing unit 114, and a third fusion unit 115.

[0177] The second display unit 111 is used to display the original image to be processed on the client's display interface, wherein the original image is multimedia data acquired during video playback.

[0178] The second processing unit 112 is used to perform strong light fusion on the original image and obtain a preprocessed image of the original image based on the generated strong light fusion result.

[0179] The second fusion unit 113 is used to fuse the preprocessed image with the original image to obtain an initial result image.

[0180] The third processing unit 114 is used to perform smooth step filtering on the strong light fusion result to generate a filtered image, and to brighten the dark areas in the initial result image to generate a secondary brightened image.

[0181] The third fusion unit 115 is used to fuse the initial result image, the filtered image, and the secondary brightening image to generate a secondary result image, and to use a skin mask on the secondary result image, wherein a skin mask for the facial skin is generated based on the key information of the face in the original image.

[0182] It should be noted that the second display unit 111, the first processing unit 112, the second fusion unit 113, the third processing unit 114, and the third fusion unit 115 mentioned above correspond to steps S302 to S310 of Embodiment 1, respectively. The five units and the corresponding steps implement the same examples and application scenarios, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above units, as part of the device, can run in the computer terminal 10 provided in Embodiment 1.

[0183] According to embodiments of the present invention, a method for implementing the above is also provided. Figure 4 The method for processing images in a video shown is a video image processing device.

[0184] Figure 12 This is a schematic diagram of another video image processing apparatus according to an embodiment of the present invention. Figure 12 As shown, the image processing device 120 in the video may include: a third display unit 121, a first display unit 122 and a fourth fusion unit 123.

[0185] The third display unit 121 is used to display the original image to be processed on the client's display interface, wherein the original image is multimedia data acquired during video playback.

[0186] The first display unit 122 is used to respond to the image processing instructions received by the client and display the image processing results of the image processing instructions on the client's display interface. The image processing instructions are used to perform strong light fusion on the high-frequency layer image of the original image.

[0187] The fourth fusion unit 123 is used to fuse the image processing result with the original image and render the fused initial result image on the display interface.

[0188] It should be noted that the third display unit 121, the first display unit 122, and the fourth fusion unit 123 mentioned above correspond to steps S402 to S406 of Embodiment 1, respectively. The three units and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above units, as part of the device, can run in the computer terminal 10 provided in Embodiment 1.

[0189] According to embodiments of the present invention, a method for implementing the above is also provided. Figure 5 The method for processing images in a video shown is a video image processing device.

[0190] Figure 13 This is a schematic diagram of another video image processing apparatus according to an embodiment of the present invention. Figure 13 As shown, the image processing device 130 in the video may include: a second display unit 131, a determination unit 132, a fourth processing unit 133, a fifth fusion unit 134, and a rendering unit 135.

[0191] The second display unit 131 is used to display the original image to be processed in the video on the display interface.

[0192] The determining unit 132 is used to determine the high-frequency layer image and the low-frequency layer image in the original image in response to the recognition operation on the original image.

[0193] The fourth processing unit 133 is used to perform strong light fusion on the high-frequency layer image and obtain a preprocessed image of the original image based on the generated strong light fusion result.

[0194] The fifth fusion unit 134 is used to fuse the preprocessed image with the original image.

[0195] The rendering unit 135 is used to render the initial result image obtained by fusion in the display interface.

[0196] It should be noted that the second display unit 131, the determining unit 132, the fourth processing unit 133, the fifth fusion unit 134, and the rendering unit 135 mentioned above correspond to steps S502 to S510 of Embodiment 1, respectively. The three units and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above units, as part of the device, can run in the computer terminal 10 provided in Embodiment 1.

[0197] In the video image processing device of this embodiment, a high-frequency layer image is identified from the original image, strong light fusion is performed on the high-frequency layer image, the original image is brightened, and then fused with the original image. This achieves the goal of preserving details and color uniformity while smoothing the skin of the original image, making the picture appear clean and transparent, improving the texture and clarity of the image, and avoiding the loss of information of the entire picture during the skin smoothing process by the optimization tool. This solves the technical problem of poor image clarity after image processing of objects in the video, and achieves the technical effect of improving the image clarity of objects in the video after image processing.

[0198] Example 4

[0199] Embodiments of the present invention can provide a system for processing images in a video. This system may include a computer terminal, which can be any computer terminal device from a group of computer terminals. Optionally, in this embodiment, the computer terminal may be replaced with a mobile terminal or other terminal device.

[0200] Optionally, in this embodiment, the computer terminal may be located in at least one of a plurality of network devices in a computer network.

[0201] In this embodiment, the computer terminal described above can execute the program code for the following steps in the video image processing method of the application: displaying the original image to be processed, wherein the original image is multimedia data collected during video playback; identifying the high-frequency layer image in the original image; performing strong light fusion on the high-frequency layer image, and obtaining a preprocessed image of the original image based on the generated strong light fusion result; fusing the preprocessed image with the original image to obtain an initial result image.

[0202] Optionally, Figure 14 This is a structural block diagram of a computer terminal according to an embodiment of the present invention. Figure 14 As shown, the computer terminal A may include one or more (only one is shown in the figure) processors 142, memory 144, and transmission devices 146.

[0203] The transmission device is used to transmit the original image to be processed; the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the video image processing method and device in this embodiment of the invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the above-mentioned video image processing method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to computer terminal A via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0204] The processor can call the information and application program stored in the memory through the transmission device to perform the following steps: display the original image to be processed, wherein the original image is multimedia data acquired during video playback; identify the high-frequency layer image in the original image; perform strong light fusion on the high-frequency layer image, and obtain a preprocessed image of the original image based on the generated strong light fusion result; fuse the preprocessed image with the original image to obtain an initial result image.

[0205] Optionally, the processor may also execute program code that performs the following steps: brightening the original image to obtain a first-brightened image; generating a pre-processed image based on the intensity fusion result and the first-brightened image.

[0206] Optionally, the processor may also execute program code that performs the following steps: brightening the low-frequency layer image of the original image to obtain a brightened image.

[0207] Optionally, the processor may also execute program code for the following steps: performing Gaussian filtering on the G-channel grayscale image of the original image to obtain the low-frequency layer image of the original image; and calculating the high-contrast layer based on the G-channel grayscale image and the low-frequency layer image to obtain the high-frequency layer image.

[0208] Optionally, the processor may also execute program code that performs strong light fusion on the high-frequency layer image to obtain a filtered result after filtering out defects in the high-frequency layer image.

[0209] Optionally, the processor may also execute program code that adjusts the level of the initial result image based on the received parameters during the process of fusing the preprocessed image with the original image.

[0210] Optionally, the processor may also execute program code for the following steps: after obtaining the initial result image, perform smooth step filtering on the strong light fusion result to generate a filtered image; brighten the dark areas in the initial result image to generate a secondary brightened image; and fuse the initial result image, the filtered image, and the secondary brightened image to generate a secondary result image.

[0211] Optionally, the processor may also execute program code that performs the following steps: after displaying the original image to be processed on the client's display interface, detects key facial information from the original image based on facial features; and generates a skin mask for the facial skin based on the key facial information.

[0212] Optionally, the processor may also execute program code that performs the following steps: after generating the secondary result image, sharpening the secondary result image using a skin mask to generate the target image; and outputting the target image.

[0213] Optionally, the processor may also execute program code that performs the following steps: using a skin mask and different preset adjustment parameters to sharpen the face region in the secondary result image to generate a target image, wherein the background region in the secondary result image remains unchanged, or the sharpening level of the background region is lower than the sharpening level of the face region.

[0214] As an alternative example, the processor can invoke information and applications stored in memory via a transmission device to perform the following steps: displaying the original image to be processed on the client's display interface, wherein the original image is multimedia data acquired during video playback; performing strong light fusion on the original image and obtaining a preprocessed image of the original image based on the generated strong light fusion result; fusing the preprocessed image with the original image to obtain an initial result image; sharpening the skin mask in the initial result image to generate a target image, wherein a skin mask for the facial skin is generated based on key information of the face in the original image.

[0215] As an alternative example, the processor can invoke information and applications stored in memory via a transmission device to perform the following steps: displaying the original image to be processed on the client's display interface, wherein the original image is multimedia data acquired during video playback; responding to the image processing instructions received by the client, displaying the image processing results of the image processing instructions on the client's display interface, wherein the image processing instructions are used to perform strong light fusion on the high-frequency layer image of the original image; fusing the image processing results with the original image, and rendering the initial result image obtained by fusion on the display interface.

[0216] As an alternative example, the processor can invoke information and application programs stored in memory via a transmission device to perform the following steps: displaying the original image to be processed in the video on a display interface; identifying the high-frequency layer image and the low-frequency layer image in the original image in response to a recognition operation on the original image; performing strong light fusion on the high-frequency layer image and obtaining a preprocessed image of the original image based on the generated strong light fusion result; fusing the preprocessed image with the original image; and rendering the initial result image obtained by fusion on the display interface.

[0217] This invention provides a method for processing images in a video. The method involves displaying the original image to be processed, wherein the original image is multimedia data collected during video playback; identifying high-frequency layer images in the original image; performing strong light fusion on the high-frequency layer images; obtaining a pre-processed image of the original image based on the generated strong light fusion result; and fusing the pre-processed image with the original image to obtain an initial result image. This application identifies high-frequency layer images from the original image, performs strong light fusion on the high-frequency layer images, brightens the original image, and then fuses it with the original image. This achieves the goal of preserving details and color uniformity while smoothing the skin of the original image, resulting in a clean and transparent image, improving image texture and clarity. It avoids the loss of information in the entire image during the smoothing process by optimization tools, thus solving the technical problem of poor image clarity after image processing of objects in the video, achieving the technical effect of improving the image clarity of objects in the video after image processing.

[0218] Those skilled in the art will understand that Figure 14 The structure shown is for illustrative purposes only. Computer terminal A can also be a smartphone (such as an Android phone, iOS phone, etc.), tablet computer, mobile internet device (MID), PAD and other terminal devices. Figure 14 This does not limit the structure of the aforementioned computer terminal A. For example, computer terminal A may also include components that are more complex than those described above. Figure 14 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 14 The different configurations shown.

[0219] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0220] Example 5

[0221] Embodiments of the present invention also provide a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium can be used to store the program code executed by the video image processing method provided in Embodiment 1 above.

[0222] Optionally, in this embodiment, the computer-readable storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0223] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: displaying the original image to be processed, wherein the original image is multimedia data acquired during video playback; identifying the high-frequency layer image in the original image; performing strong light fusion on the high-frequency layer image, and obtaining a preprocessed image of the original image based on the generated strong light fusion result; and fusing the preprocessed image with the original image to obtain an initial result image.

[0224] Optionally, the computer-readable storage medium is also configured to store program code for performing the following steps: brightening the original image to obtain a first-brightened image; generating a pre-processed image based on the intensity fusion result and the first-brightened image.

[0225] Optionally, the computer-readable storage medium is also configured to store program code for performing the following steps: brightening the low-frequency layer image of the original image to obtain a first-brightened image.

[0226] Optionally, the computer-readable storage medium is further configured to store program code for performing the following steps: performing Gaussian filtering on the G-channel grayscale image of the original image to obtain a low-frequency layer image of the original image; and calculating a high-contrast layer based on the G-channel grayscale image and the low-frequency layer image to obtain a high-frequency layer image.

[0227] Optionally, the computer-readable storage medium is also configured to store program code for performing the following steps: performing strong light fusion on the high-frequency layer image to obtain a filtered result after filtering out defects in the high-frequency layer image.

[0228] Optionally, the computer-readable storage medium is also configured to store program code for performing the following steps: adjusting the level of the initial result image based on received parameters during the fusion of the preprocessed image and the original image.

[0229] Optionally, the computer-readable storage medium is further configured to store program code for performing the following steps: after obtaining the initial result image, performing a smooth step filter on the strong light fusion result to generate a filtered image; brightening the dark areas in the initial result image to generate a secondary brightened image; and fusing the initial result image, the filtered image, and the secondary brightened image to generate a secondary result image.

[0230] Optionally, the computer-readable storage medium is also configured to store program code for performing the following steps: after displaying the original image to be processed on the client's display interface, detecting key facial information from the original image based on facial features; and generating a skin mask for the facial skin based on the key facial information.

[0231] Optionally, the computer-readable storage medium is also configured to store program code for performing the following steps: after generating the secondary result image, sharpening the secondary result image using a skin mask to generate the target image; and outputting the target image.

[0232] Optionally, the computer-readable storage medium is further configured to store program code for performing the following steps: sharpening the face region in the secondary result image using a skin mask and different preset adjustment parameters to generate a target image, wherein the background region in the secondary result image remains unchanged, or the sharpening level of the background region is lower than the sharpening level of the face region.

[0233] As an optional example, a computer-readable storage medium is configured to store program code for performing the following steps: displaying the original image to be processed on a client's display interface, wherein the original image is multimedia data acquired during video playback; performing intensity fusion on the original image and obtaining a preprocessed image of the original image based on the generated intensity fusion result; fusing the preprocessed image with the original image to obtain an initial result image; and sharpening the secondary result image using a skin mask to generate a target image, wherein a skin mask for the facial skin is generated based on key information of the face in the original image.

[0234] As an optional example, a computer-readable storage medium is configured to store program code for performing the following steps: displaying the original image to be processed on a client's display interface, wherein the original image is multimedia data acquired during video playback; responding to an image processing instruction received by the client, displaying the image processing result of executing the image processing instruction on the original image on the client's display interface, wherein the image processing instruction is used to perform intense light fusion on the high-frequency layer image of the original image; fusing the image processing result with the original image, and rendering the initial result image obtained by fusion on the display interface.

[0235] As an optional example, a computer-readable storage medium is configured to store program code for performing the following steps: displaying the original image to be processed in a video in a display interface; identifying the high-frequency layer image and the low-frequency layer image in the original image in response to a recognition operation on the original image; performing intensity fusion on the high-frequency layer image and obtaining a preprocessed image of the original image based on the generated intensity fusion result; fusing the preprocessed image with the original image; and rendering the initial result image obtained by fusion in the display interface.

[0236] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0237] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0238] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0239] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0240] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0241] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0242] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for processing images in a video, characterized in that, include: Display the original image to be processed, wherein the original image is multimedia data acquired during video playback; Identify the high-frequency layer image in the original image; The high-frequency layer image is subjected to intense light fusion, and a preprocessed image of the original image is obtained based on the generated intense light fusion result; The preprocessed image is fused with the original image to obtain an initial result image; The process of obtaining a preprocessed image of the original image based on the generated strong light fusion result includes: adjusting the curve of the original image; brightening the original image after curve adjustment to obtain a first brightened image; and generating the preprocessed image based on the strong light fusion result and the first brightened image. The process of fusing the preprocessed image with the original image to obtain an initial result image includes: fusing the pixels of the preprocessed image with the pixels of the original image using a skin smoothing strength parameter to obtain the initial result image; The method further includes: performing a smooth step filter on the strong light fusion result to generate a filtered image; generating a secondary brightening image using the calculated pixel values ​​of the dark areas in the initial result image; and fusing the pixels of the initial result image, the filtered image, and the secondary brightening image to generate a secondary result image.

2. The method according to claim 1, characterized in that, The original image after curve adjustment is brightened to obtain a first-brightened image, including: The low-frequency layer image of the original image after curve adjustment is brightened to obtain the first brightened image.

3. The method according to claim 1, characterized in that, Identifying the high-frequency layer image in the original image includes: Gaussian filtering is applied to the G channel grayscale image of the original image to obtain the low-frequency layer image of the original image; Based on the G-channel grayscale image and the low-frequency layer image, a high-contrast layer is calculated to obtain the high-frequency layer image.

4. The method according to claim 1, characterized in that, Performing intensity fusion on the high-frequency layer image includes: The high-frequency layer image is subjected to strong light fusion to obtain a filtered result after filtering out the defects in the high-frequency layer image.

5. The method according to claim 1, characterized in that, During the process of fusing the preprocessed image with the original image, the level of the initial result image is adjusted based on the received parameters.

6. The method according to claim 1, characterized in that, After displaying the original image to be processed, the method further includes: Based on facial features, key facial information is detected from the original image; Based on the key facial information, a skin mask for the facial skin is generated.

7. The method according to claim 6, characterized in that, After generating the secondary result image, the method further includes: The skin mask is used to sharpen the secondary result image to generate the target image; Output the target image.

8. The method according to claim 7, characterized in that, The secondary result image is sharpened using the skin mask to generate the target image, including: The face region in the secondary result image is sharpened using the skin mask and different preset adjustment parameters to generate the target image, wherein the background region in the secondary result image remains unchanged, or the sharpening level of the background region is lower than that of the face region.

9. A method for processing images in a video, characterized in that, include: The original image to be processed is displayed on the client's display interface, wherein the original image is multimedia data acquired during video playback; The original image is subjected to intensity fusion, and a preprocessed image of the original image is obtained based on the generated intensity fusion result; The preprocessed image is fused with the original image to obtain an initial result image; The strong light fusion result is subjected to smooth step filtering to generate a filtered image, and a secondary brightening image is generated by using the calculated pixel values ​​of the dark areas in the initial result image. The pixels of the initial result image, the pixels of the filtered image, and the pixels of the secondary brightening image are fused to generate a secondary result image. A skin mask is then used to sharpen the secondary result image to generate the target image. The skin mask for the facial skin is generated based on the key information of the face in the original image. The process of obtaining a preprocessed image of the original image based on the generated strong light fusion result includes: adjusting the curve of the original image; brightening the original image after curve adjustment to obtain a first brightened image; and generating the preprocessed image based on the strong light fusion result and the first brightened image. The process of fusing the preprocessed image with the original image to obtain an initial result image includes: fusing the pixels of the preprocessed image with the pixels of the original image using a skin smoothing intensity parameter to obtain the initial result image.

10. A method for processing images in a video, characterized in that, include: The original image to be processed is displayed on the client's display interface, wherein the original image is multimedia data acquired during video playback; In response to the image processing command received by the client, the image processing result of executing the image processing command on the original image is displayed on the client's display interface. The image processing command is used to perform strong light fusion on the high-frequency layer image of the original image. The image processing result includes a pre-processed image of the original image, which is generated based on the strong light fusion result and a first brightening image. The strong light fusion result is obtained by performing strong light fusion on the high-frequency layer image, and the first brightening image is obtained by brightening the original image after curve adjustment. The image processing result is fused with the original image, and the fused initial result image is rendered on the display interface; The process of fusing the image processing result with the original image includes: fusing the pixels of the preprocessed image with the pixels of the original image using a skin smoothing intensity parameter to obtain the initial result image; The method further includes: performing a smooth step filter on the strong light fusion result to generate a filtered image; generating a secondary brightening image using the calculated pixel values ​​of the dark areas in the initial result image; and fusing the pixels of the initial result image, the filtered image, and the secondary brightening image to generate a secondary result image.

11. A method for processing images in a video, characterized in that, include: The original image to be processed in the video is displayed in the display interface; In response to the recognition operation on the original image, the high-frequency layer image in the original image is determined; The high-frequency layer image is subjected to intense light fusion, and a preprocessed image of the original image is obtained based on the generated intense light fusion result; The preprocessed image is then fused with the original image. The initial result image obtained by fusion is rendered in the display interface; The process of obtaining a preprocessed image of the original image based on the generated strong light fusion result includes: adjusting the curve of the original image; brightening the original image after curve adjustment to obtain a first brightened image; and generating the preprocessed image based on the strong light fusion result and the first brightened image. The process of fusing the preprocessed image with the original image to obtain an initial result image includes: fusing the pixels of the preprocessed image with the pixels of the original image using a skin smoothing strength parameter to obtain the initial result image; The method further includes: performing a smooth step filter on the strong light fusion result to generate a filtered image; generating a secondary brightening image using the calculated pixel values ​​of the dark areas in the initial result image; and fusing the pixels of the initial result image, the filtered image, and the secondary brightening image to generate a secondary result image.

12. An apparatus for processing images in a video, characterized in that, include: The first display unit is used to display the original image to be processed, wherein the original image is multimedia data collected during video playback; The recognition unit is used to identify the high-frequency layer image in the original image; The first processing unit is used to perform strong light fusion on the high-frequency layer image and obtain a preprocessed image of the original image based on the generated strong light fusion result. The first fusion unit is used to fuse the preprocessed image with the original image to obtain an initial result image; The first processing unit is configured to obtain a preprocessed image of the original image based on the generated strong light fusion result by performing the following steps: adjusting the curve of the original image; brightening the original image after curve adjustment to obtain a first brightened image; and generating the preprocessed image based on the strong light fusion result and the first brightened image. The first fusion unit is configured to fuse the preprocessed image and the original image by performing the following steps to obtain an initial result image: fusing the pixels of the preprocessed image and the pixels of the original image by using a skin smoothing strength parameter to obtain the initial result image; The device is further configured to perform the following steps: perform smooth step filtering on the strong light fusion result to generate a filtered image; generate a secondary brightening image using the calculated pixel values ​​of the dark areas in the initial result image; and fuse the pixels of the initial result image, the pixels of the filtered image, and the pixels of the secondary brightening image to generate a secondary result image.

13. An apparatus for processing images in a video, characterized in that, include: The second display unit is used to display the original image to be processed on the client's display interface, wherein the original image is multimedia data acquired during video playback; The second processing unit is used to perform strong light fusion on the original image and obtain a preprocessed image of the original image based on the generated strong light fusion result. The second fusion unit is used to fuse the preprocessed image with the original image to obtain an initial result image; The third processing unit is used to perform smooth step filtering on the strong light fusion result, generate a filtered image, and generate a secondary brightening image based on the calculated pixel values ​​of the dark areas in the initial result image. The third fusion unit is used to fuse the pixels of the initial result image, the pixels of the filtered image, and the pixels of the secondary brightening image to generate a secondary result image, and to use a skin mask on the secondary result image, wherein the skin mask of the face is generated based on the key information of the face in the original image. The second processing unit is configured to obtain a preprocessed image of the original image based on the generated strong light fusion result by performing the following steps: adjusting the curve of the original image; brightening the original image after curve adjustment to obtain a first brightened image; and generating the preprocessed image based on the strong light fusion result and the first brightened image. The second fusion unit is used to fuse the preprocessed image and the original image by performing the following steps to obtain an initial result image: fusing the pixels of the preprocessed image and the pixels of the original image by using a skin smoothing strength parameter to obtain the initial result image.

14. An apparatus for processing images in a video, characterized in that, include: The third display unit is used to display the original image to be processed on the client's display interface, wherein the original image is multimedia data collected during video playback; The first display unit is configured to respond to an image processing instruction received by the client and display the image processing result of executing the image processing instruction on the original image on the client's display interface. The image processing instruction is used to perform strong light fusion on the high-frequency layer image of the original image. The image processing result includes a preprocessed image of the original image, which is generated based on the strong light fusion result and a first brightening image. The strong light fusion result is obtained by performing strong light fusion on the high-frequency layer image, and the first brightening image is obtained by brightening the original image after curve adjustment. The fourth fusion unit is used to fuse the image processing result with the original image and render the fused initial result image on the display interface; The fourth fusion unit is used to fuse the image processing result with the original image by performing the following steps: fusing the pixels of the preprocessed image with the pixels of the original image using a skin smoothing intensity parameter to obtain the initial result image; The device is further configured to perform the following steps: perform smooth step filtering on the strong light fusion result to generate a filtered image; generate a secondary brightening image using the calculated pixel values ​​of the dark areas in the initial result image; and fuse the pixels of the initial result image, the pixels of the filtered image, and the pixels of the secondary brightening image to generate a secondary result image.

15. An apparatus for processing images in a video, characterized in that, include: The second display unit is used to display the original image to be processed in the video on the display interface; A determining unit is configured to determine the high-frequency layer image and the low-frequency layer image in the original image in response to a recognition operation on the original image; The fourth processing unit is used to perform strong light fusion on the high-frequency layer image and obtain a preprocessed image of the original image based on the generated strong light fusion result; The fifth fusion unit is used to fuse the preprocessed image with the original image; A rendering unit is used to render the initial result image obtained by fusion in the display interface; The fourth processing unit is configured to obtain a preprocessed image of the original image based on the generated strong light fusion result by performing the following steps: curve adjustment of the original image; The original image after curve adjustment is brightened to obtain a first-brightened image; the preprocessed image is generated based on the intensity fusion result and the first-brightened image. The fifth fusion unit is used to fuse the preprocessed image and the original image by performing the following steps to obtain an initial result image: fusing the pixels of the preprocessed image and the pixels of the original image by using a skin smoothing strength parameter to obtain the initial result image; The device is further configured to perform the following steps: perform smooth step filtering on the strong light fusion result to generate a filtered image; generate a secondary brightening image using the calculated pixel values ​​of the dark areas in the initial result image; and fuse the pixels of the initial result image, the pixels of the filtered image, and the pixels of the secondary brightening image to generate a secondary result image.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein when the program is run by a processor, it controls the device in which the computer-readable storage medium resides to perform the method according to any one of claims 1 to 11.

17. A processor, characterized in that, The processor is used to run a program, wherein the program is executed by the processor to perform the method according to any one of claims 1 to 11.

18. A system for processing images in a video, characterized in that, include: processor; A memory, connected to the processor, is used to provide the processor with instructions to perform the following processing steps: displaying the original image to be processed, wherein the original image is multimedia data acquired during video playback; identifying high-frequency and low-frequency images in the original image; filtering the high-frequency images and brightening the low-frequency images to obtain a preprocessed image of the original image; and fusing the preprocessed image with the original image to obtain an initial result image. The process of obtaining a preprocessed image of the original image based on the generated strong light fusion result includes: adjusting the curve of the original image; brightening the low-frequency layer image of the original image after curve adjustment to obtain a first brightened image; and generating the preprocessed image based on the strong light fusion result and the first brightened image. The process of fusing the preprocessed image with the original image to obtain an initial result image includes: fusing the pixels of the preprocessed image with the pixels of the original image using a skin smoothing strength parameter to obtain the initial result image; The memory is also used to provide the processor with instructions to process the following steps: perform smooth step filtering on the strong light fusion result to generate a filtered image; generate a secondary brightening image using the calculated pixel values ​​of the dark areas in the initial result image; and fuse the pixels of the initial result image, the pixels of the filtered image, and the pixels of the secondary brightening image to generate a secondary result image.

Images