Image processing method and device, medium and program product

Abstract

The invention discloses an image processing method and device, a medium and a program product in the technical field of computers. According to the scheme, on the basis of a live broadcast image and a filter effect reference image corresponding to the live broadcast image, K color correction images are obtained based on the live broadcast image, then N color correction images most similar to the filter effect reference image are selected from the K color correction images, and the N color correction images are synthesized to obtain a color correction composite image; adding a filter effect to the live broadcast image by using the color correction composite image to obtain a live broadcast effect image, so that the live broadcast effect image not only contains key information of the original live broadcast image, but also can be close to the filter effect of the filter effect reference image, and the filter effect of the live broadcast image is ensured; and a user does not need to manually add the filter, so that the imaging efficiency of the filter is also improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image processing method, device, medium and program product. Background Art

[0002] Currently, users can manually add various filters during live broadcasts. Most of these filters are obtained by stretching the RGB average value of the original live broadcast image. The image filter effect is not good, and because manual adjustment is required, the adjustment is time-consuming. Summary of the invention

[0003] In view of this, the purpose of this application is to provide an image processing method, device, medium and program product to improve the filter effect and imaging efficiency of live broadcast images. The specific scheme is as follows:

[0004] To achieve the above objectives, on the one hand, the present application provides an image processing method, comprising:

[0005] Acquire a live broadcast image and a filter effect reference image corresponding to the live broadcast image;

[0006] Obtain K color-corrected images based on the live image;

[0007] Selecting N color-corrected images that are most similar to the filter effect reference image from the K color-corrected images, and synthesizing the N color-corrected images to obtain a color-corrected composite image; K is a positive integer, and K is not less than N;

[0008] The color correction composite image is used to add a filter effect to the live broadcast image to obtain a live broadcast effect image.

[0009] Optionally, obtaining a live broadcast image and a filter effect reference image corresponding to the live broadcast image includes:

[0010] Extracting the live broadcast image and the filter effect reference image from a user live broadcast request;

[0011] or

[0012] Extracting the live broadcast image from a user live broadcast request;

[0013] An image that is most similar to the live broadcast image is selected from a preset filter effect image library as the filter effect reference image.

[0014] Optionally, obtaining K color-corrected images based on the live broadcast image includes:

[0015] Based on the LUT, the live image is subjected to K color mapping processes to obtain the K color-corrected images.

[0016] Optionally, selecting N color-corrected images that are most similar to the filter effect reference image from the K color-corrected images includes:

[0017] Calculating the distances between the K color-corrected images and the filter effect reference image respectively to obtain K distance values;

[0018] Sort the K distance values ​​by size to obtain a distance sequence;

[0019] If the distance sequence is arranged in descending order, N distance values ​​are obtained starting from the end of the distance sequence;

[0020] If the distance sequence is arranged in ascending order, N distance values ​​are obtained starting from the head of the distance sequence;

[0021] The N color-corrected images corresponding to the N distance values ​​obtained by interception are used as the N color-corrected images most similar to the filter effect reference image.

[0022] Optionally, synthesizing the N color-corrected images to obtain a color-corrected composite image includes:

[0023] The color average values ​​of the N color-corrected images are calculated to obtain the color-corrected composite image.

[0024] Optionally, after obtaining the live broadcast effect diagram, the method further includes:

[0025] The color difference between the live broadcast effect image and the filter effect reference image is superimposed on the color correction composite image and the live broadcast image to obtain an enhanced live broadcast effect image.

[0026] Optionally, the color difference between the live broadcast effect image and the filter effect reference image is superimposed on the color correction composite image and the live broadcast image to obtain an enhanced live broadcast effect image, including:

[0027] Calculating the color difference between the live broadcast effect image and the filter effect reference image;

[0028] Superimposing the color difference value onto the color correction composite image to obtain a filter intermediate image;

[0029] The filter intermediate image is used to add a filter effect to the live broadcast image to obtain the enhanced live broadcast effect image.

[0030] Optionally, it also includes:

[0031] Display live broadcast effect pictures or enhance live broadcast effect pictures;

[0032] Based on the user's confirmation request for the live broadcast rendering or the enhanced live broadcast rendering, the live broadcast is started; or, based on the user's denial request for the live broadcast rendering or the enhanced live broadcast rendering, the step of obtaining the live broadcast image and the filter effect reference image corresponding to the live broadcast image and other subsequent steps are re-executed to regenerate the live broadcast rendering or the enhanced live broadcast rendering.

[0033] On the other hand, the present application also provides an electronic device, comprising a processor and a memory; wherein the memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the aforementioned image processing method.

[0034] On the other hand, the present application further provides a storage medium, in which computer executable instructions are stored. When the computer executable instructions are loaded and executed by a processor, the aforementioned image processing method is implemented.

[0035] On the other hand, the present application also provides a computer program product, including a computer program / instruction, which implements the aforementioned image processing method when executed by a processor.

[0036] The present application is based on a live broadcast image and a filter effect reference image corresponding to the live broadcast image. K color correction images are first obtained based on the live broadcast image, and then N color correction images that are most similar to the filter effect reference image are selected from the K color correction images, and the N color correction images are synthesized to obtain a color correction composite image; then, the filter effect is added to the live broadcast image using the color correction composite image to obtain a live broadcast effect image, so that the live broadcast effect image not only contains the key information of the original live broadcast image, but also is close to the filter effect of the filter effect reference image, thereby ensuring the filter effect of the live broadcast image; and there is no need for users to manually add filters, which also improves the filter imaging efficiency.

[0037] Correspondingly, the electronic equipment, media and program products provided by this application also have the above-mentioned technical effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0039] Figure 1 A schematic diagram of a physical architecture applicable to the present application provided for the present application;

[0040] Figure 2 A flow chart of an image processing method provided in this application;

[0041] Figure 3 Another image processing method flow chart provided by this application;

[0042] Figure 4 A server structure diagram provided for this application;

[0043] Figure 5 A terminal structure diagram provided for this application. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In addition, in the embodiments of the present application, "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0045] Currently, users can manually add various filters during live broadcasts. Most of these filters are obtained by stretching the RGB average value of the original live broadcast image. The image filter effect is not good, and because manual adjustment is required, the adjustment is time-consuming.

[0046] In view of the above-mentioned problems that currently exist, the present application proposes an image processing solution, which can ensure that the live broadcast effect image not only contains the key information of the original live broadcast image, but also is close to the filter effect of the filter effect reference image, thereby ensuring the filter effect of the live broadcast image; and there is no need for users to manually add filters, which also improves the filter imaging efficiency.

[0047] For ease of understanding, the physical framework to which this application applies is first introduced.

[0048] It should be understood that the image processing method provided in the present application can be applied to a system or program with an image processing function. Specifically, the system or program with an image processing function can be run in a server, a personal computer, or other device.

[0049] like Figure 1 As shown, Figure 1 This is a schematic diagram of the physical architecture applicable to this application. Figure 1In the invention, a system or program with an image processing function can be run on a server, which obtains a live broadcast image and a filter effect reference image corresponding to the live broadcast image from other terminal devices through a network, first obtains K color correction images based on the live broadcast image, then selects N color correction images that are most similar to the filter effect reference image from the K color correction images, and synthesizes the N color correction images to obtain a color correction composite image; then, the color correction composite image is used to add a filter effect to the live broadcast image to obtain a live broadcast effect image.

[0050] Depend on Figure 1 It can be seen that the server can establish communication connections with multiple devices, and the server obtains the live broadcast image and the filter effect reference image corresponding to the live broadcast image from these devices. The server synthesizes the corresponding live broadcast effect image for the live broadcast image.

[0051] Figure 1 A variety of terminal devices are shown in FIG. 1 . In actual scenarios, more or fewer types of terminal devices may be involved in the image processing process. The specific number and type depend on the actual scenario and are not limited here. In addition, Figure 1 One server is shown in the figure, but in an actual scenario, multiple servers may be involved, and the specific number of servers depends on the actual scenario.

[0052] It is understandable that the above-mentioned system and program with image processing function can be used as a kind of cloud service program. The specific operation mode depends on the actual scenario and is not limited here.

[0053] Combined with the above commonalities, please see Figure 2 , Figure 2 This is a flow chart of an image processing method provided in an embodiment of the present application. Figure 2 As shown, the image processing method may include the following steps:

[0054] S201, obtaining a live broadcast image and a filter effect reference image corresponding to the live broadcast image.

[0055] In this embodiment, the live broadcast image can be any frame of video in the live broadcast video after the user turns on the live broadcast. In order to provide users with more suitable live broadcast filters, this embodiment allows the user to submit a filter effect reference image when turning on the live broadcast. If the user does not submit a filter effect reference image when turning on the live broadcast, the back-end server can automatically match the corresponding filter effect reference image according to the user's live broadcast request. Among them, the filter effect reference image is: a frame of image obtained after adding a suitable filter to any frame of video in the user's live broadcast video. For example: after adding a suitable filter to live broadcast image A, the image obtained is the filter effect reference image, and this filter effect reference image is considered to be the best filter effect image for live broadcast image A.

[0056] In one embodiment, obtaining a live broadcast image and a filter effect reference image corresponding to the live broadcast image includes: extracting the live broadcast image and the filter effect reference image from the user's live broadcast request, that is, the user submits the filter effect reference image on his own initiative when starting the live broadcast; or extracting the live broadcast image from the user's live broadcast request; selecting the image most similar to the live broadcast image in the preset filter effect image library as the filter effect reference image, that is, the user does not submit the filter effect reference image when starting the live broadcast, and the back-end server automatically matches the corresponding filter effect reference image according to the user's live broadcast request; the preset filter effect image library pre-stores a plurality of reference images of the best filter effects. Exemplarily, when the user starts the live broadcast, each frame of the live video can be used as a live broadcast image, and the steps provided in this embodiment can be executed to add filter effects to each frame of the live video, and the video frames with the added filter effects can be displayed in real time and in sequence, so that the filter addition and smooth playback of the live video can be completed.

[0057] S202: Obtain K color-corrected images based on the live broadcast image.

[0058] In order to improve the filter effect, this embodiment first performs multiple color corrections on the live image, for example, performing K color corrections on the live image, where K is a positive integer and can be in the interval [3,10]. Exemplarily, color correction can be implemented based on LUT technology. LUT (Look-Up-Table) can convert input color values ​​into output color values ​​to achieve multiple functions such as color correction and stylized color adjustment. LUT can remap the color information in the image to achieve functions such as color correction and stylized color adjustment. It can be seen that LUT is used for color correction and color grading, and can map the input color value to another color value. Therefore, in one embodiment, K color-corrected images are obtained based on the live image, including: based on LUT, performing K color mapping processing on the live image to obtain K color-corrected images. For example: K LUT tables are preset, and K color mapping processing is performed on the same live image with reference to the K LUT tables, so that K color-corrected images can be obtained.

[0059] S203 , selecting N color-corrected images that are most similar to the filter effect reference image from the K color-corrected images, and synthesizing the N color-corrected images to obtain a color-corrected composite image.

[0060] It should be noted that before selecting the N color-corrected images that are most similar to the filter effect reference image from the K color-corrected images, it is first necessary to calculate the similarity between the K color-corrected images and the filter effect reference image. Exemplarily, the similarity can be calculated by using histogram matching, image similarity calculation based on feature points, and image matrix decomposition, and the specific implementation can be referred to the relevant technology.

[0061] Exemplarily, histogram matching: evaluate the similarity of images by calculating the histograms of two images and comparing the similarity between them; for example: calculate the normalized correlation coefficients (such as Bhattacharyya distance, histogram intersection distance, etc.) between the grayscale histograms of K color-corrected images and the grayscale histograms of the filter effect reference image, and determine the degree of similarity based on the normalized correlation coefficient. Image similarity calculation based on feature points: characterize the features of the image by detecting the corner points in the image, which are usually the locations with large changes in the image. For example: use the SIFT (Scale-Invariant Feature Transform) algorithm to detect the key points in the image and describe its local features, and determine the similarity between images based on these key points and local features. Image matrix decomposition: extract the features of the image by matrix decomposition to calculate the similarity. Exemplarily, the image matrix is ​​subjected to singular value decomposition to obtain singular value eigenvectors, which have good stability and noise resistance. Therefore, the similarity between images can be calculated based on these singular value eigenvectors; and the singular value decomposition provides an overall representation of the image, which has algebraic and geometric invariance to orthogonal transformation, rotation, displacement, mirror mapping, etc.

[0062] In one embodiment, selecting N color-corrected images that are most similar to a filter effect reference image from K color-corrected images includes: respectively calculating the distances between the K color-corrected images and the filter effect reference image to obtain K distance values; sorting the K distance values ​​by size to obtain a distance sequence; if the distance sequence is arranged in descending order, then starting from the end of the distance sequence, N distance values ​​are obtained; if the distance sequence is arranged in ascending order, then starting from the beginning of the distance sequence, N distance values ​​are obtained; and the N color-corrected images corresponding to the N distance values ​​obtained are used as the N color-corrected images that are most similar to the filter effect reference image. Wherein, K is not less than N.

[0063] Referring to the above, after determining the N color-corrected images that are most similar to the filter effect reference image, the N color-corrected images are synthesized. Exemplarily, an image fusion method may be used to synthesize the N most similar color-corrected images.

[0064] In this embodiment, the image fusion methods may include: pixel-level image fusion method: directly process each pixel of the image and retain all pixel information, such as: average method, maximum method, minimum method, median method, etc. Feature-level image fusion method: mainly use the edge, texture, color and other features of the image for fusion. Such as: fusion based on wavelet transform, fusion based on Laplace pyramid, etc. Model-based image fusion methods include: fusion methods based on Gaussian mixture models, fusion methods using convolutional neural networks or generative adversarial networks. In one embodiment, synthesizing N color-corrected images to obtain a color-corrected composite image includes: calculating the color average of the N color-corrected images to obtain a color-corrected composite image. It can be seen that the color average of each pixel of the N color-corrected images that are most similar to the filter effect reference image is calculated to obtain a color-corrected composite image.

[0065] S204: Add a filter effect to the live broadcast image using the color correction composite image to obtain a live broadcast effect image.

[0066] It should be noted that adding a filter effect to a live broadcast image using a color correction composite image means realizing pixel superposition and fusion of the color correction composite image and the live broadcast image, which can be specifically realized by referring to relevant image fusion methods.

[0067] This embodiment selects N color correction images that are closest to the reference image from K color correction images, and synthesizes a color correction composite image based on these N color correction images. A filter effect is added to the live broadcast image based on the color correction composite image to obtain a preliminary live broadcast effect image, thereby improving the filter addition effect of the live broadcast. Furthermore, after obtaining the live broadcast effect image, a live broadcast effect image with a better filter effect can be obtained based on the color difference, thereby further improving the filter addition effect of the live broadcast. Therefore, in one embodiment, after obtaining the live broadcast effect image, it also includes: superimposing the color difference between the live broadcast effect image and the filter effect reference image to the color correction composite image and the live broadcast image to obtain an enhanced live broadcast effect image. The enhanced live broadcast effect image can also be implemented with reference to the relevant image fusion method.

[0068] The color difference between the live broadcast effect image and the filter effect reference image is superimposed on the color correction composite image and the live broadcast image to obtain an enhanced live broadcast effect image, including: calculating the color difference between the live broadcast effect image and the filter effect reference image; superimposing the color difference on the color correction composite image to obtain a filter intermediate image; using the filter intermediate image to add a filter effect to the live broadcast image to obtain an enhanced live broadcast effect image. Exemplarily, the RGB difference between the live broadcast effect image LUTb(x) and the filter effect reference image w can be calculated: LUTb(x)-w; then this RGB difference is added to the color correction composite image LUTb to achieve image color compensation, then the final enhanced live broadcast effect image d=LUTfinal(x)=LUTb+LUTb(x)-w.

[0069] In order to satisfy the user with the live filter effect, the present embodiment allows the user to manually confirm and update the live effect picture or the enhanced live effect picture. In one embodiment, after obtaining the live effect picture or the enhanced live effect picture, it also includes: displaying the live effect picture or the enhanced live effect picture; starting the live broadcast according to the user's confirmation request for the live effect picture or the enhanced live effect picture; or, according to the user's denial request for the live effect picture or the enhanced live effect picture, re-execute the step of obtaining the live image and the filter effect reference picture corresponding to the live image and other subsequent steps to regenerate the live effect picture or the enhanced live effect picture. That is: if the user is not satisfied with the live effect picture or the enhanced live effect picture, the live effect picture or the enhanced live effect picture can be denied, and then the scheme provided by the present embodiment can be re-executed to update the live effect picture or the enhanced live effect picture until the user confirms the live effect picture or the enhanced live effect picture.

[0070] It can be seen that this embodiment is based on the live broadcast image and the filter effect reference image corresponding to the live broadcast image. K color correction images are first obtained based on the live broadcast image, and then N color correction images that are most similar to the filter effect reference image are selected from the K color correction images, and the N color correction images are synthesized to obtain a color correction composite image; then the color correction composite image is used to add a filter effect to the live broadcast image to obtain a live broadcast effect image. Furthermore, the color difference between the obtained live broadcast effect image and the filter effect reference image can be superimposed on the color correction composite image and the live broadcast image, and finally an enhanced live broadcast effect image is obtained, so that the live broadcast effect image contains both the key information of the original live broadcast image and the filter effect of the filter effect reference image, thereby ensuring the filter effect of the live broadcast image; and there is no need for the user to manually add filters, which also improves the filter imaging efficiency.

[0071] See also Figure 3After inputting the original live broadcast picture, first add a filter to the original live broadcast picture (i.e., live broadcast image) through the online live broadcast filter, and then use KNN (K-nearest neighbor algorithm) to output the N closest to the reference live broadcast picture (i.e., filter effect reference picture). These N images with filters are compared and learned with the reference live broadcast picture to correct and output the filter effect picture. Among them, the reference live broadcast picture is the filter effect reference picture, which represents the filter effect expected by the user. The output filter effect picture is output after the user is satisfied. If the user is not satisfied, then the user can provide a new reference live broadcast picture and repeat the above steps to make the filter effect picture more optimized.

[0072] The exemplary implementation steps can refer to the following process: for the original live picture, use LUT technology to obtain 1, 2, ..., K filter effect pictures, which are expressed by the formula: yK=LUTK(x), yK represents the effect picture corresponding to the filter LUTK, x represents the original live picture, and K is the mark of the filter LUTK, that is, the mark of the Kth LUT table. Then compare the K filter effect pictures with the reference picture w: calculate the distance zK=distance(yK,w) between the effect picture corresponding to the filter LUTK and the reference picture, specifically, the Manhattan distance, Euclidean distance, etc. can be calculated. After comparison, select N color correction images corresponding to smaller distances from the K distance values: LUTz1, LUTz2, ..., LUTzN, and calculate the average value of the color correction images LUT1, LUT2, ..., LUTN corresponding to these N smaller distances, which is expressed by the formula: LUTb=minzn(LUT1+LUT2+...LUTN) / N. The next step is to use LUTb to filter the original live broadcast image x to obtain the live broadcast effect image LUTb(x), and then calculate the RGB difference between LUTb(x) and the reference image w: LUTb(x)-w; add this value to LUTb to achieve image color compensation, which is expressed by the formula: the final enhanced live broadcast effect image d=LUTfinal(x)=LUTb+LUTb(x)-w.

[0073] In this example, the reference image may be different for different original live broadcast images, but it is best that the original live broadcast image and the reference image are of the same person, but with different backgrounds, styles and tones. The reference image is specified by the live broadcast user himself or automatically selected by the live broadcast platform based on the user's facial features.

[0074] It can be seen that in this embodiment, the original live broadcast image first generates online K LUTs filter effect images through the prior LUT technology, and then compares with the reference image to obtain N closest LUTs filters. Then, the N LUTs filter effect images are compared and studied with the reference image, and the final enhanced live broadcast effect image d is corrected, so that the final enhanced live broadcast effect image contains the key information of the original live broadcast image and can be close to the filter effect of the filter effect reference image, ensuring the filter effect of the live broadcast image; and there is no need for users to manually add filters, which also improves the filter imaging efficiency.

[0075] An image processing device provided in an embodiment of the present application is introduced below. The image processing device described below and the related implementation steps of the above embodiment can be referenced to each other.

[0076] An image processing device provided in an embodiment of the present application includes:

[0077] An acquisition module, used to acquire a live broadcast image and a filter effect reference image corresponding to the live broadcast image;

[0078] A preprocessing module, used for obtaining K color-corrected images based on the live broadcast image;

[0079] A selection module is used to select N color-corrected images that are most similar to the filter effect reference image from the K color-corrected images, and synthesize the N color-corrected images to obtain a color-corrected composite image; K is a positive integer, and K is not less than N;

[0080] An adding module is used to add a filter effect to the live broadcast image using the color correction composite image to obtain a live broadcast effect image.

[0081] In one implementation, the acquisition module is specifically used to:

[0082] Extracting the live broadcast image and the filter effect reference image from a user live broadcast request;

[0083] or

[0084] Extracting the live broadcast image from a user live broadcast request;

[0085] An image that is most similar to the live broadcast image is selected from a preset filter effect image library as the filter effect reference image.

[0086] In one embodiment, the preprocessing module is specifically used for:

[0087] Based on the LUT, the live image is subjected to K color mapping processes to obtain the K color-corrected images.

[0088] In one embodiment, the selection module is specifically used to:

[0089] Calculating the distances between the K color-corrected images and the filter effect reference image respectively to obtain K distance values;

[0090] Sort the K distance values ​​by size to obtain a distance sequence;

[0091] If the distance sequence is arranged in descending order, N distance values ​​are obtained starting from the end of the distance sequence;

[0092] If the distance sequence is arranged in ascending order, N distance values ​​are obtained starting from the head of the distance sequence;

[0093] The N color-corrected images corresponding to the N distance values ​​obtained by interception are used as the N color-corrected images most similar to the filter effect reference image.

[0094] In one embodiment, the selection module is specifically used to:

[0095] The color average values ​​of the N color-corrected images are calculated to obtain the color-corrected composite image.

[0096] In one embodiment, it further includes:

[0097] The correction module is used to obtain the live broadcast effect image, and then superimpose the color difference between the live broadcast effect image and the filter effect reference image on the color correction composite image and the live broadcast image to obtain an enhanced live broadcast effect image.

[0098] In one embodiment, the correction module is specifically used to:

[0099] Calculate the color difference between the live broadcast effect image and the filter effect reference image;

[0100] The color difference is superimposed on the color correction composite image to obtain the filter intermediate image;

[0101] The filter middle image is used to add a filter effect to the live broadcast image to obtain an enhanced live broadcast effect image.

[0102] In one embodiment, it further includes:

[0103] A display module is used to display a live broadcast effect picture or an enhanced live broadcast effect picture; start live broadcast according to a user's confirmation request for the live broadcast effect picture or the enhanced live broadcast effect picture; or, according to a user's denial request for the live broadcast effect picture or the enhanced live broadcast effect picture, re-execute the steps of obtaining a live broadcast image and a filter effect reference picture corresponding to the live broadcast image and other subsequent steps to regenerate the live broadcast effect picture or the enhanced live broadcast effect picture.

[0104] It can be seen that the image processing device provided in this embodiment can make the final enhanced live broadcast effect image contain both the key information of the original live broadcast image and the filter effect close to the filter effect reference image, thereby ensuring the filter effect of the live broadcast image; and there is no need for users to manually add filters, which also improves the filter imaging efficiency.

[0105] An electronic device provided in an embodiment of the present application is introduced below. The electronic device described below and related implementation steps of the above embodiment can be referenced to each other.

[0106] The present application discloses an electronic device, including:

[0107] Memory, used to store computer programs;

[0108] A processor is used to execute the computer program to implement the method disclosed in any of the above embodiments.

[0109] In this embodiment, when the processor executes the computer program stored in the memory, the following steps can be specifically implemented: obtaining a live broadcast image and a filter effect reference image corresponding to the live broadcast image; obtaining K color correction images based on the live broadcast image; selecting N color correction images that are most similar to the filter effect reference image from the K color correction images, and synthesizing the N color correction images to obtain a color correction composite image; K is not less than N; and using the color correction composite image to add a filter effect to the live broadcast image to obtain a live broadcast effect image.

[0110] In this embodiment, when the processor executes the computer program stored in the memory, the following steps can be specifically implemented: extracting the live broadcast image and the filter effect reference image from the user's live broadcast request; or extracting the live broadcast image from the user's live broadcast request; selecting the image most similar to the live broadcast image in the preset filter effect image library as the filter effect reference image.

[0111] In this embodiment, when the processor executes the computer program stored in the memory, the following steps may be specifically implemented: based on the LUT, color mapping processing is performed K times on the live image to obtain the K color-corrected images.

[0112] In this embodiment, when the processor executes the computer program stored in the memory, the following steps may be specifically implemented: respectively calculating the distances between the K color-corrected images and the filter effect reference image to obtain K distance values; sorting the K distance values ​​according to size to obtain a distance sequence; if the distance sequence is arranged in descending order, then starting from the end of the distance sequence, N distance values ​​are obtained by intercepting; if the distance sequence is arranged in ascending order, then starting from the beginning of the distance sequence, N distance values ​​are obtained by intercepting; and the N color-corrected images corresponding to the intercepted N distance values ​​are used as the N color-corrected images that are most similar to the filter effect reference image.

[0113] In this embodiment, when the processor executes the computer program stored in the memory, the following steps may be specifically implemented: calculating the color average values ​​of the N color-corrected images to obtain the color-corrected composite image.

[0114] In this embodiment, when the processor executes the computer program stored in the memory, the following steps can be specifically implemented: the color difference between the live broadcast effect image and the filter effect reference image is superimposed on the color correction composite image and the live broadcast image to obtain an enhanced live broadcast effect image.

[0115] In this embodiment, when the processor executes the computer program stored in the memory, the following steps can be specifically implemented: calculating the color difference between the live broadcast effect image and the filter effect reference image; superimposing the color difference on the color correction composite image to obtain a filter intermediate image; using the filter intermediate image to add a filter effect to the live broadcast image to obtain an enhanced live broadcast effect image.

[0116] In this embodiment, when the processor executes the computer program stored in the memory, the following steps can be specifically implemented: displaying a live broadcast effect picture or an enhanced live broadcast effect picture; starting the live broadcast according to the user's confirmation request for the live broadcast effect picture or the enhanced live broadcast effect picture; or, according to the user's denial request for the live broadcast effect picture or the enhanced live broadcast effect picture, re-executing the step of obtaining the live broadcast image and the filter effect reference picture corresponding to the live broadcast image and other subsequent steps to regenerate the live broadcast effect picture or the enhanced live broadcast effect picture.

[0117] Furthermore, the present application also provides an electronic device. The electronic device can be Figure 4 The server 50 shown may also be Figure 5 Terminal 60 is shown. Figure 4 and Figure 5 All of them are structural diagrams of electronic devices according to an exemplary embodiment, and the contents in the diagrams cannot be regarded as any limitation on the scope of use of the present application.

[0118] Figure 4A schematic diagram of the structure of a server provided in an embodiment of the present application. The server 50 may specifically include: at least one processor 51, at least one memory 52, a power supply 53, a communication interface 54, an input / output interface 55, and a communication bus 56. The memory 52 is used to store a computer program, which is loaded and executed by the processor 51 to implement the relevant steps in the image processing disclosed in any of the aforementioned embodiments.

[0119] In this embodiment, the power supply 53 is used to provide working voltage for each hardware device on the server 50; the communication interface 54 can create a data transmission channel between the server 50 and external devices, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 55 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0120] In addition, the memory 52, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon include an operating system 521, a computer program 522 and data 523, etc. The storage method can be temporary storage or permanent storage.

[0121] The operating system 521 is used to manage and control the hardware devices and computer programs 522 on the server 50, so as to realize the operation and processing of the data 523 in the memory 52 by the processor 51, and it can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the image processing method disclosed in any of the above embodiments, the computer program 522 can also further include a computer program that can be used to complete other specific tasks. In addition to including data such as application update information, the data 523 can also include data such as application developer information.

[0122] Figure 5 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present application. The terminal 60 may specifically include but is not limited to a smart phone, a tablet computer, a laptop computer or a desktop computer.

[0123] Generally, the terminal 60 in this embodiment includes: a processor 61 and a memory 62 .

[0124] Among them, the processor 61 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 61 can be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 61 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 61 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 61 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0125] The memory 62 may include one or more computer-readable storage media, which may be non-transitory. The memory 62 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 62 is at least used to store the following computer program 621, wherein the computer program, after being loaded and executed by the processor 61, can implement the relevant steps in the image processing method performed by the terminal side disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 62 may also include an operating system 622 and data 623, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 622 may include Windows, Unix, Linux, etc. Data 623 may include, but is not limited to, update information of the application.

[0126] In some embodiments, the terminal 60 may further include a display screen 63 , an input / output interface 64 , a communication interface 65 , a sensor 66 , a power supply 67 , and a communication bus 68 .

[0127] Those skilled in the art will understand that Figure 5 The structure shown in the figure does not constitute a limitation on the terminal 60, and the terminal 60 may include more or less components than those shown in the figure.

[0128] A storage medium provided in an embodiment of the present application is introduced below. The storage medium described below and the related implementation steps of the above embodiment can be referenced to each other.

[0129] Furthermore, the embodiment of the present application also discloses a storage medium, in which computer executable instructions are stored, and when the computer executable instructions are loaded and executed by a processor, the image processing method disclosed in any of the above embodiments is implemented. For the specific steps of the method, reference can be made to the corresponding contents disclosed in the above embodiments, and no further description will be given here.

[0130] A computer program product provided in an embodiment of the present application is introduced below. The computer program product described below can be cross-referenced with other embodiments described in this document.

[0131] A computer program product includes a computer program / instruction, which implements the image processing method disclosed in the above embodiment when the computer program / instruction is executed by a processor.

[0132] It should be pointed out that the above is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

[0133] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0134] Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An image processing method, characterized in that: include: Acquire a live broadcast image and a filter effect reference image corresponding to the live broadcast image; Obtain K color-corrected images based on the live image; Selecting N color-corrected images that are most similar to the filter effect reference image from the K color-corrected images, and synthesizing the N color-corrected images to obtain a color-corrected composite image; K is a positive integer, and K is not less than N; The color correction composite image is used to add a filter effect to the live broadcast image to obtain a live broadcast effect image.

2. The method according to claim 1, characterized in that Acquiring a live broadcast image and a filter effect reference image corresponding to the live broadcast image, including: Extracting the live broadcast image and the filter effect reference image from a user live broadcast request; or Extracting the live broadcast image from a user live broadcast request; An image that is most similar to the live broadcast image is selected from a preset filter effect image library as the filter effect reference image.

3. The method according to claim 1, characterized in that Obtaining K color-corrected images based on the live broadcast image, including: Based on the LUT, the live image is subjected to K color mapping processes to obtain the K color-corrected images.

4. The method according to claim 1, characterized in that: Selecting N color-corrected images that are most similar to the filter effect reference image from the K color-corrected images includes: Calculating the distances between the K color-corrected images and the filter effect reference image respectively to obtain K distance values; Sort the K distance values ​​by size to obtain a distance sequence; If the distance sequence is arranged in descending order, N distance values ​​are obtained starting from the end of the distance sequence; If the distance sequence is arranged in ascending order, N distance values ​​are obtained starting from the head of the distance sequence; The N color-corrected images corresponding to the N distance values ​​obtained by interception are used as the N color-corrected images most similar to the filter effect reference image.

5. The method according to claim 1, characterized in that The N color-corrected images are synthesized to obtain a color-corrected composite image, including: The color average values ​​of the N color-corrected images are calculated to obtain the color-corrected composite image.

6. The method according to claim 1, characterized in that After getting the live broadcast effect picture, it also includes: The color difference between the live broadcast effect image and the filter effect reference image is superimposed on the color correction composite image and the live broadcast image to obtain an enhanced live broadcast effect image.

7. The method according to claim 6, characterized in that The method further comprises: superimposing a color difference between the live broadcast effect image and the filter effect reference image onto the color correction composite image and the live broadcast image to obtain an enhanced live broadcast effect image, including: Calculating the color difference between the live broadcast effect image and the filter effect reference image; Superimposing the color difference value onto the color correction composite image to obtain a filter intermediate image; The filter intermediate image is used to add a filter effect to the live broadcast image to obtain the enhanced live broadcast effect image.

8. The method according to any one of claims 1 to 7, characterized in that: Also includes: Display live broadcast effect pictures or enhance live broadcast effect pictures; In response to a user's confirmation request for the live broadcast rendering or the enhanced live broadcast rendering, starting the live broadcast; Or, based on the user's denial request for the live broadcast effect image or the enhanced live broadcast effect image, the step of obtaining the live broadcast image and the filter effect reference image corresponding to the live broadcast image and other subsequent steps are re-executed to regenerate the live broadcast effect image or the enhanced live broadcast effect image.

9. An electronic device, characterized in that: The electronic device comprises a processor and a memory; wherein the memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the method according to any one of claims 1 to 8.

10. A storage medium, characterized in that: The storage medium stores computer executable instructions, and when the computer executable instructions are loaded and executed by the processor, the method according to any one of claims 1 to 8 is implemented.

11. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images