Display chip, image processing method and electronic equipment

Abstract

The invention discloses a display chip, an image processing method and electronic equipment, and belongs to the technical field of displays. The display chip is configured to obtain a first image of a first resolution; determining first brightness information and first color information based on the first image; processing the first brightness information through a super-resolution model to obtain second brightness information; interpolating the first color information to obtain second color information; a second image of a second resolution is determined based on the second brightness information and the second color information, the second resolution being greater than the first resolution. According to the invention, the resolution of the image is improved through the display chip, the quality of the high-resolution image is ensured, and the application scene is greatly enriched.

Description

Technical Field

[0001] Embodiments of the present application relate to the technical field of displays, and in particular, to a display chip, an image processing method, and an electronic device. Background Art

[0002] In the technical field of displays, the resolution refers to the number of pixel points existing in the horizontal and vertical directions on the screen, which can reflect the resolution ability of the display for image details and determines the fineness of the image displayed by the display. With the improvement of display technology, a low-resolution image can be converted into a high-resolution image. This technology of increasing the resolution of the source image is also called Super Resolution (SR).

[0003] In related technologies, a low-resolution image can be converted into a high-resolution image through a super-resolution model. Among them, the super-resolution model is a neural network model based on deep learning. Since each pixel point in the low-resolution image corresponds to brightness information and color information, the super-resolution model contains more parameters. Therefore, in the image conversion process, more computing resources and storage resources are required, and the time required for the image conversion process is relatively long, which affects the application of the image processing method on mobile devices. Summary of the Invention

[0004] The present application provides a display chip, an image processing method, and an electronic device, which can convert a low-resolution image into a high-resolution image through the display chip and improve the application on mobile devices. The technical solution includes the following content.

[0005] On the one hand, a display chip is provided. The chip is applied to a display screen and is configured to:

[0006] Obtain a first image with a first resolution;

[0007] Determine first brightness information and first color information based on the first image;

[0008] Process the first brightness information through a super-resolution model to obtain second brightness information;

[0009] Interpolate the first color information to obtain second color information;

[0010] Determine a second image with a second resolution based on the second brightness information and the second color information, where the second resolution is greater than the first resolution.

[0011] On the other hand, an image processing method is provided. The method includes:

[0012] Obtain a first image with a first resolution;

[0013] Determine first luminance information and first color information based on the first image;

[0014] Process the first luminance information through a super-resolution model to obtain second luminance information;

[0015] Interpolate the first color information to obtain second color information;

[0016] Determine a second image with a second resolution based on the second luminance information and the second color information, where the second resolution is greater than the first resolution.

[0017] On the other hand, an electronic device is provided, and the electronic device includes the display chip described above.

[0018] The technical solution provided by this application at least brings the following beneficial effects:

[0019] In the technical solution provided by this application, since the human eye is very sensitive to luminance information and relatively insensitive to color information, and compared with interpolating an image, processing an image through a model can better improve the quality of the image. Therefore, by processing the first luminance information of a low-resolution image through a super-resolution model to obtain second luminance information, interpolating the first color information of the low-resolution image to obtain second color information, and determining a second image based on the second luminance information and the second color information, the quality of the high-resolution image can be ensured.

[0020] Since the luminance information and the color information are processed separately, it is such that: compared with processing the first luminance information and the first color information through a super-resolution model, only processing the first luminance information through a super-resolution model can reduce the amount of data to be processed, thereby saving computing resources and storage resources. Compared with processing the first color information through a super-resolution model, interpolating the first color information can save more computing resources and storage resources. Since the solution of this application occupies less computing resources and storage resources, this solution can be deployed in a chip, and the chip can be deployed in mobile and fixed terminals, greatly enriching the application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of this application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of this application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0022] Figure 1 It is a schematic diagram of a display chip provided by an embodiment of this application;

[0023] Figure 2 is a flowchart of an image processing method provided by an embodiment of the present application;

[0024] Figure 3 is a flowchart for processing brightness information provided by an embodiment of the present application;

[0025] Figure 4 is a schematic diagram of a super-resolution model provided by an embodiment of the present application;

[0026] Figure 5 is a schematic diagram of a non-linear mapping unit provided by an embodiment of the present application;

[0027] Figure 6 is a schematic diagram of an interpolation method provided by an embodiment of the present application;

[0028] Figure 7 is a schematic diagram of an image processing method provided by an embodiment of the present application;

[0029] Figure 8 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0030] To make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the accompanying drawings.

[0031] It should be noted that the terms "first", "second", etc. in the present application are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such terms can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0032] An embodiment of the present application provides a display chip. As Figure 1 shown, the display chip 101 is applied to the display screen 10. The embodiment of the present application does not limit the type, size, etc. of the display screen 10. Exemplarily, the display screen 10 can be a CRT (Cathode Ray Tube Display), an LCD (Liquid Crystal Display), an LED (Light Emitting Diode Display), a plasma display screen, or the like. The embodiment of the present application also does not limit the structure and function of the display chip 101. Among them, the display chip 101 is configured to execute asFigure 2 The image processing method shown, the image processing method includes the following steps.

[0033] Step 201, obtain a first image with a first resolution.

[0034] In the embodiments of the present application, the image content of the first image and its acquisition method are not limited. Exemplarily, the first image is an image input by the user, or can be an image acquired by an image acquisition device such as a camera or a mobile phone. The resolution of the first image is the first resolution, and the first resolution can be any resolution. In the embodiments of the present application, the resolution refers to the number of pixel points existing in the horizontal and vertical directions of the image, and its size is generally represented by a product. For example, the resolution 1920×1080 means that there are 1920 pixel points in the horizontal direction of the image and 1080 pixel points in the vertical direction of the image. Since the resolution is closely related to the number of pixel points, the resolution can reflect the fineness of the image, and the finer the image, the higher the resolution.

[0035] Step 202, determine first brightness information and first color information based on the first image.

[0036] The display chip 10 can extract first brightness information for characterizing its brightness and first color information for characterizing its color from the first image. The extraction method is not limited here. For example, the first image can be format-converted to obtain the converted first image. Each pixel point in the converted first image corresponds to a brightness channel and a color channel. The first brightness information includes the values of each pixel point in the brightness channel of the converted first image, and the first color information includes the values of each pixel point in the color channel of the converted first image.

[0037] In the embodiments of the present application, the format of the first image before format conversion and the format of the first image after format conversion are not limited. Exemplarily, the first image before format conversion can correspond to RGB format or BGGR format, etc., and the first image after format conversion can correspond to YUV format or YCbCr format, etc. There are differences in the conversion methods of different formats. Taking the conversion from RGB format to YCbCr format as an example, optionally, the conversion is performed according to the following formula (1).

[0038]

[0039] Wherein, represents the values of each pixel point in the Y channel, Cb channel, and Cr channel of the first image after format conversion, where the Y channel is the brightness channel, and the Cb channel and Cr channel are both color channels. Characterize the values of each pixel point in the first image before format conversion in the R channel, G channel, and B channel, where the R channel is the red channel, the G channel is the green channel, and the B channel is the blue channel, and these three channels all belong to the color channels. Characterize the weight matrix, Characterize the bias matrix.

[0040] It should be noted that the values of the elements in the above weight matrix and bias matrix need to conform to various industry standards. Taking the industry standard BT 709 for high-definition television as an example, assuming that the image signal bit width is 8 bits (bit), then the above formula (1) can specifically be the following formula (2).

[0041]

[0042] Step 203, process the first luminance information through the super-resolution model to obtain the second luminance information.

[0043] The super-resolution model is a model obtained after training a neural network model based on deep learning technology. Generally, the electronic device can obtain the luminance information of the low-resolution image and the luminance information of the high-resolution image, input the luminance information of the low-resolution image into the neural network model, output the luminance information through the neural network model, determine the loss value based on the luminance information output by the model and the luminance information of the high-resolution image, and adjust the parameters of the neural network model based on the loss value. After adjusting the parameters of the neural network model multiple times, the adjusted neural network model is the super-resolution model. Then, input the first luminance information into the super-resolution model, and output the second luminance information through the super-resolution model.

[0044] The embodiments of the present application do not limit the model structure of the super-resolution model. In actual application, the super-resolution model includes at least one network layer such as a convolutional layer, a normalization layer, an activation layer, an attention layer, and a linear layer. It can be understood that different network structures of the super-resolution model result in different ways of processing luminance information. The following shows a possible implementation.

[0045] In this example, when the display chip 101 executes step 203, it is configured to execute steps S1 to S2 (not shown in the figure).

[0046] Step S1, extract the features of the first luminance information to obtain shallow features and deep features. The shallow features characterize the detailed content of the first luminance information, and the deep features characterize the global content of the first luminance information.

[0047] Such as Figure 3As shown in the figure, the super-resolution model includes a deep feature extraction module 301. The first luminance information 01 is input into the deep feature extraction module 301. The deep feature extraction module 301 performs deep feature extraction on the first luminance information 01 to obtain and output deep features. Deep features refer to features extracted through a multi-layer neural network, where the number of layers of the neural network is greater than a threshold value. This threshold value can be a randomly determined value or a value set according to artificial experience. Deep features have a large receptive field and rich semantic information, and can better describe the global content of the first luminance information. Among them, the receptive field refers to the size of the area in the input image mapped by a pixel point in the output feature. The larger the receptive field, the larger the area in the input image corresponding to a pixel point in the output feature, which also means that the output feature pays more attention to the global content; on the contrary, the smaller the receptive field, the smaller the area in the input image corresponding to a pixel point in the output feature, which also means that the output feature pays more attention to the local content. Since the first luminance information reflects the luminance of the first image, the global content of the first luminance information can refer to the overall brightness and darkness of the first image.

[0048] In practical applications, the deep feature extraction module 301 can have different structures. Different structures of the deep feature extraction module 301 result in different ways of extracting deep features. The following shows a possible implementation.

[0049] In an exemplary embodiment, the super-resolution model includes a first convolution module, a non-linear mapping module, and a second convolution module. Specifically, the super-resolution model includes a deep feature extraction module 301, as Figure 3 shown. The deep feature extraction module 301 includes a first convolution module, a non-linear mapping module, and a second convolution module. When the display chip 101 executes the step of "extracting the features of the first luminance information to obtain deep features" in step S1, it is configured to execute steps S11 to S13 (not shown in the figure).

[0050] Step S11: Perform convolution processing on the first luminance information to obtain a convolution processing result.

[0051] That is to say, the first luminance information is input into the first convolution module. After the first convolution module performs convolution processing on the first luminance information, it obtains and outputs a convolution processing result. The convolution processing result is feature data used to characterize the first luminance information. Among them, convolution processing is a mathematical operation. The first convolution module slides the convolution kernel on the input data (such as the first luminance information) to calculate the corresponding part of the convolution kernel and the input data, thereby realizing the convolution operation.

[0052] The first convolution module includes at least one convolutional layer, and each convolutional layer is used to perform a convolution operation on the input data. If the first convolution module includes at least two convolutional layers, any two convolutional layers can be connected in series or in parallel. In addition, besides the convolutional layer, the first convolution module can also include other types of network layers such as a splicing layer, an activation layer, a linear layer, etc. Thus, it can be seen that in practical applications, the structure of the first convolution module can be set flexibly.

[0053] In a possible implementation manner, when the display chip 101 executes step S11, it is configured to: perform convolution processing on the first luminance information at different scales to obtain at least two first processing results; splice at least two first processing results to obtain a first splicing result; and perform convolution processing on the first splicing result to obtain a convolution processing result.

[0054] In this example, the first convolution module includes at least two convolutional layers connected in parallel, a splicing layer, and a convolutional layer. Among them, the first luminance information is input into each of the convolutional layers connected in parallel, and each convolutional layer performs convolution processing on the first luminance information to obtain the first processing results output by each convolutional layer. Any two convolutional layers connected in parallel correspond to different convolutional kernels. Since the convolutional kernels of the convolutional layers are different, the scales of the convolution processing performed by the convolutional layers on the first luminance information also vary. The splicing layer is connected in series after at least two convolutional layers connected in parallel and is used to splice each first processing result, that is, stack each first processing result in the channel dimension to obtain a first splicing result and increase the depth of the first splicing result. A convolutional layer is connected in series after the splicing layer, and the convolutional layer performs convolution processing on the first splicing result at any scale to obtain a convolution processing result.

[0055] Hereinafter, the Mth (M is a positive integer) convolutional layer can be denoted as convolutional layer M. The convolutional kernel of convolutional layer M is (Kr, Kc), where Kr corresponds to the row dimension and Kc corresponds to the column dimension.

[0056] As Figure 4 shown, the first convolution module 401 includes convolutional layer 1, convolutional layer 2, a splicing layer, and convolutional layer 3. The first luminance information is input into convolutional layer 1. Since the convolutional kernel of convolutional layer 1 is (3, 3), convolutional layer 1 is used to perform 3×3 convolution processing on the first luminance information to obtain a first processing result. The first luminance information is input into convolutional layer 2. Since the convolutional kernel of convolutional layer 2 is (5, 5), convolutional layer 2 is used to perform 5×5 convolution processing on the first luminance information to obtain another first processing result. The two first processing results are input into the splicing layer, and the splicing layer splices the two first processing results to obtain a first splicing result. Then, the first splicing result is input into convolutional layer 3. Since the convolutional kernel of convolutional layer 3 is (1, 1), convolutional layer 3 is used to perform 1×1 convolution processing on the first splicing result to obtain a convolution processing result.

[0057] By performing convolution processing on the first luminance information at different scales, features with different levels of detail are extracted from the first luminance information, enabling each first processing result to represent the information of the first luminance information at different scales and improving the comprehensiveness of the information. By splicing each first processing result and performing convolution processing on the spliced result, the fusion of each first processing result is achieved, making the convolution processing result have a strong representation ability and being beneficial to improving the quality of the second image.

[0058] Step S12: Perform a non-linear mapping on the convolution processing result to obtain a mapping result.

[0059] That is to say, input the convolution processing result into the non-linear mapping module, and the non-linear mapping module performs a non-linear mapping on the convolution processing result to obtain and output the mapping result. Non-linear mapping is a mapping that does not satisfy the linear condition, which mathematically manifests as not satisfying the superposition principle, that is, a small change in the input will cause a large change in the output.

[0060] The non-linear mapping module includes at least one non-linear mapping unit, and each non-linear mapping unit is used to perform a non-linear mapping on the input data. If the non-linear mapping module includes at least two non-linear mapping units, these non-linear mapping units are connected in series, and any two non-linear mapping units may have the same or different structures. In addition, in addition to the non-linear mapping units, the non-linear mapping module may also include other types of network layers such as a splicing layer, an activation layer, and a linear layer. It can be seen that in practical applications, the structure of the non-linear mapping module can be flexibly set. For example, in Figure 4 , the non-linear mapping module 402 includes non-linear mapping units 0 to non-linear mapping unit N, where N is a positive integer. For example, N is 3.

[0061] For ease of description, in the following, taking the non-linear mapping module including one non-linear mapping unit as an example, the implementation principle of step S12 is described. Optionally, when the display chip 101 executes step S12, it is configured to execute steps S121 to S123 (not shown in the figure).

[0062] Step S121: Based on the convolution processing result, determine the channel features of at least two channels.

[0063] In this example, the non-linear mapping unit includes a first processing unit. As Figure 5 shown, the non-linear mapping unit includes a first processing unit 501. The first processing unit processes the convolution processing result to obtain a first feature map. The first feature map includes the channel features of at least two channels (Channel), and each type of channel feature is a feature of the first luminance information. The embodiments of the present application do not limit the structure and processing method of the first processing unit.

[0064] Exemplarily, when the display chip 101 executes step S121, it is configured to: determine the second processing result of each channel based on the convolution processing result; determine the third processing result of each channel based on the second processing result of each channel; for any channel, add the second processing result and the third processing result of any channel to obtain the channel feature of any channel.

[0065] In this example, the first processing unit includes a first processing subunit. The convolution processing result is input into the first processing subunit, and the second processing result of each channel is determined by the first processing subunit. In practical applications, the structure of the first processing subunit can be flexibly set.

[0066] Optionally, as Figure 5 shown, the first processing unit 501 includes a first processing subunit 502, and the first processing subunit 502 includes a depth-wise separable convolution layer (Depth-Wise Separable Convolution, DW Conv) 01, a convolution layer 02, and a PReLU (Parametric Rectified Linear Unit) layer.

[0067] Among them, the depth-wise separable convolution layer 01 is used to perform depth convolution (Depth-Wise Convolution) and point-wise convolution (Point-Wise Convolution) on the convolution processing result. Depth convolution is to perform convolution on each channel of the convolution processing result, and the scale of the convolution is not limited here. As Figure 5 shown, since the convolution kernel of the depth-wise separable convolution layer 01 is (3, 3), the depth convolution is to perform 3×3 convolution on each channel of the convolution processing result. Point-wise convolution is to perform a linear combination between channels on the convolution processing result using a 1×1 convolution kernel. After performing depth convolution and point-wise convolution on the convolution processing result, output data is obtained, and the output data corresponds to at least two channels. Since the number of parameters of depth convolution is proportional to the number of channels, and the number of parameters of point-wise convolution is relatively small, the number of parameters of the depth-wise separable convolution layer 01 is small, which can save storage resources and computing resources.

[0068] The convolution layer 02 is used to perform convolution processing of any scale on the output data (corresponding to at least two channels) of the depth-wise separable convolution layer 01 to fuse the information of all or part of the channels and improve the representation ability of the convolution result. As Figure 5As shown, the convolution kernel of the convolution layer 02 is (1, 1), so the convolution layer 02 performs a 1×1 convolution on the output data of the depthwise separable convolution layer 01. It should be noted that the convolution layer 02 includes at least two convolution kernels, and each convolution kernel is used to perform convolution processing on the output data of the depthwise separable convolution layer 01 to obtain the processing result of one channel. The output data of the convolution layer 02 includes the processing results of each channel.

[0069] The PReLU layer is used to perform a non-linear mapping on the output data of the convolution layer 02 (i.e., the processing results of each channel) to obtain the output data, and the output data includes the second processing results of each channel. For example, if the output data of the convolution layer 02 includes the processing results of 24 channels, then the output data of the PReLU layer includes the second processing results of 24 channels.

[0070] The first processing unit further includes a second processing subunit. The second processing results of each channel are input into the second processing subunit, and the third processing results of each channel are determined by the second processing subunit. In actual applications, the structure of the second processing subunit can also be flexibly set.

[0071] Optionally, as Figure 5 shown, the first processing unit 501 further includes a second processing subunit 503. The second processing subunit 503 includes a convolution layer 03, a convolution layer 04, and a convolution layer 05. Among them, the convolution layer 03 is used to perform convolution processing of any scale on the output data of the PReLU layer (i.e., the second processing results of each channel). For example, if the convolution kernel of the convolution layer 03 is (1, 1), then the convolution layer 03 performs a 1×1 convolution on the output data of the PReLU layer. Similarly, the convolution layer 04 is used to perform convolution processing of any scale on the output data of the convolution layer 03. For example, if the convolution kernel of the convolution layer 04 is (1, 3), then the convolution layer 04 performs a 1×3 convolution on the output data of the convolution layer 03. The convolution layer 05 is used to perform convolution processing of any scale on the output data of the convolution layer 04. For example, if the convolution kernel of the convolution layer 05 is (1, 1), then the convolution layer 05 performs a 1×1 convolution on the output data of the convolution layer 04. Through 1×1 or 1×3 convolutions, one-dimensional convolution of the data is realized, which is beneficial to saving computing resources and storage resources and facilitating deployment in the chip.

[0072] It should be noted that the convolutional layer 05 includes at least two convolutional kernels. The processing result of each convolutional kernel is the third processing result of one channel, that is, the output data of the convolutional layer 05 includes the third processing results of each channel. In addition, the number of convolutional kernels included in the convolutional layer 05 is the same as the number of convolutional kernels included in the convolutional layer 02. For example, if the convolutional layer 02 includes 24 convolutional kernels, such that the output data of the convolutional layer 02 includes the processing results of 24 channels, then the convolutional layer 05 also includes 24 convolutional kernels, such that the output data of the convolutional layer 05 includes the third processing results of 24 channels.

[0073] As Figure 5 shown, the first processing unit 501 is further configured to, for any one channel, add (i.e., splice) the second processing result of the channel (i.e., the output data of the first processing subunit 502) and the third processing result of the channel (i.e., the output data of the second processing subunit 503), so as to stack the second processing result and the third processing result to obtain the channel feature of the channel.

[0074] The second processing results of each channel are determined through the convolutional processing results, the third processing results of each channel are determined through the second processing results of each channel, and the second processing result and the third processing result of each channel are added, realizing the short skip connection of the second processing result and the third processing result, reducing the phenomenon of information loss caused by processing, enabling the channel feature obtained by addition to represent sufficient details, and thus improving the quality of the second image. In addition, the short skip connection has the effect of accelerating the convergence speed of the model during training.

[0075] Step S122, for any one channel, based on the channel feature of any one channel, determine the weight information of any one channel, and the weight information of any one channel represents the importance degree of the features of each pixel point in the channel feature of any one channel to the quality of the second image.

[0076] In this example, the non-linear mapping unit further includes a second processing unit. As Figure 5 shown, the non-linear mapping unit further includes a second processing unit 504, and the second processing unit 504 is connected in series after the first processing unit 501. The second processing unit processes the channel feature of each channel to obtain the weight information of the channel. The channel feature of the channel includes the features of each pixel point in the first image, the weight information of the channel includes the weights of each pixel point, and the weight of any one pixel point represents the importance degree of the feature of the pixel point to the quality of the second image. Generally, the more important the feature of the pixel point is to the quality of the second image, the higher the weight of the pixel point. The embodiments of the present application do not limit the structure and processing method of the second processing unit. Different structures of the second processing unit result in different determination methods of the weight information of the channel.

[0077] Optionally, when the display chip 101 executes step S122, it is configured to: determine the first weight of each pixel point in any channel based on the channel feature of any channel, where the first weight of a pixel point in any channel represents the importance of the feature of a pixel point in the channel feature of any channel to the quality of the second image; obtain the second weight of any channel, where the second weight of any channel represents the importance of the channel feature of any channel to the quality of the second image; determine the weight information of any channel based on the first weight of each pixel point in any channel and the second weight of any channel.

[0078] As Figure 5 shown, the second processing unit 504 includes a convolutional layer 06. For any channel, the channel feature of the channel includes the features of at least two pixel points. Based on this, the channel feature of the channel is input into the convolutional layer 06, and the convolutional layer 06 performs convolutional processing on the channel feature of the channel to obtain the first weight of each pixel point in the channel. Assume that the channel feature of the channel includes the features of N×M pixel points, where N and M are both positive integers. Then: by performing convolutional processing on the channel feature of the channel through the convolutional layer 06, N×M first weights of pixel points can be obtained. The convolutional scale of the convolutional layer 06 is not limited here. Exemplarily, as Figure 5 shown, the convolutional kernel of the convolutional layer 06 is (1, 1), and by performing 1×1 convolutional processing on the channel feature of the channel through the convolutional layer 06, the first weight of each pixel point in the channel is obtained. The first weight of any pixel point in the channel represents the importance of the feature of this pixel point in the channel feature of the channel to the quality of the second image. Generally, the more important the feature of the pixel point is to the quality of the second image, the higher the first weight of the pixel point.

[0079] The second processing unit further includes the second weight of any channel. The second weight of the channel is a value learned during the process of training the super-resolution model. As Figure 5 shown, the parameter "P" represents the second weight of the channel and is a learnable parameter in the second processing unit 504. Generally, each channel corresponds to a second weight, and different channels may correspond to the same or different second weights. The more important the channel feature of the channel is to the quality of the second image, the higher the second weight of the channel.

[0080] Next, as Figure 5As shown, the first weight of each pixel point in any channel is multiplied by the second weight of the channel by the second processing unit 504 to obtain the multiplication result of this pixel point in the channel. The weight information of the channel includes the multiplication results of each pixel point in the channel. By determining the first weights of each pixel point in the channel, the importance degree of the pixel point features in the channel features to the image quality is determined, so that the model pays more attention to the pixel points in the channel that are beneficial to improving the image quality, and the noise is removed. By determining the second weight of the channel, the importance degree of the channel features to the image quality is determined, so that the model pays more attention to the channels that are beneficial to improving the image quality. By the first weights of each pixel point in the channel and the second weight of the channel, the weight information of the channel is determined, and the total importance degree of the pixel point features in the channel features to the image quality is determined, improving the accuracy of the weight information and making the quality of the second image higher.

[0081] Step S123, determining a mapping result based on the channel features and weight information of each channel.

[0082] In this example, the non-linear mapping unit further includes a third processing unit. As Figure 5 shown, the non-linear mapping unit further includes a third processing unit 505, and the third processing unit 505 is connected in series after the first processing unit 501 and the second processing unit 504. The channel features of each channel and the weight information of each channel can be input into the third processing unit, and the mapping result is output through the third processing unit. The structure of the third processing unit is not limited in the embodiments of the present application. In actual application, when the structure of the third processing unit is different, the determination method of the mapping result also varies. For example, the third processing unit includes a multiplier and a splicing layer. The channel features of any channel are multiplied by the weight information of the channel by the multiplier, and the multiplication results of each channel are spliced by the splicing layer to obtain the mapping result.

[0083] By determining the channel features and weight information of each channel and determining the mapping result based on the channel features and weight information of each channel, the mapping result contains information that can improve the image quality and the noise is removed, thereby improving the quality of the second image.

[0084] Optionally, when the display chip 101 executes step S123, it is configured to: for any channel, multiply the channel features of any channel by the weight information to obtain the multiplication result of any channel, add the convolution processing result and the multiplication result of any channel to obtain the addition result of any channel; determine the mapping result based on the addition results of each channel.

[0085] In this example, as Figure 5As shown in the figure, the third processing unit 505 includes a multiplier, an adder, and a splicing layer (not shown in the figure). First, the channel features of any channel (i.e., the output data of the first processing unit 501) and the weight information of this channel (i.e., the output data of the second processing unit 504) are multiplied by the multiplier to obtain the multiplication result of this channel. Then, the multiplication result of the channel and the convolution processing result (i.e., the input data of the first processing unit 501) are added by the adder to obtain the addition result of this channel. After that, the addition results of each channel are spliced through the splicing layer to obtain the mapping result.

[0086] By multiplying the channel features of the channel and the weight information, the information beneficial to improving the image quality is extracted from the channel features, thereby improving the quality of the second image. By adding the convolution processing result and the multiplication result of the channel, the long skip connection of the convolution processing result and the multiplication result of the channel is realized, reducing the phenomenon of information loss caused by processing, so that the addition result can represent sufficient details, thereby improving the quality of the second image. In addition, the long skip connection has the effect of accelerating the convergence speed of the model during training.

[0087] Step S13, perform convolution processing on the mapping result to obtain deep features.

[0088] That is, the mapping result is input into the second convolution module. After the second convolution module performs convolution processing on the mapping result, deep features are obtained and output. Among them, the second convolution module includes at least one convolution layer, and each convolution layer is used to perform convolution operations on the input data. If the second convolution module includes at least two convolution layers, any two convolution layers can be connected in series or in parallel. In addition, in addition to the convolution layer, the second convolution module can also include other types of network layers such as a splicing layer, an activation layer, and a linear layer. It can be seen that in actual applications, the structure of the second convolution module can be set flexibly.

[0089] Optionally, as Figure 4 shown, the second convolution module 403 includes a convolution layer 4 and a convolution layer 5. The convolution layer 4 is used to perform convolution processing on the mapping result. The scale of the convolution processing is not limited here. For example, if the convolution kernel of the convolution layer 4 is (1, 1), the convolution layer 4 can perform 1×1 convolution processing on the mapping result. By performing convolution processing on the mapping result through the convolution layer 4, the number of channels is compressed and the data volume is reduced. The convolution layer 5 is used to perform convolution processing on the output data of the convolution layer 4. The scale of the convolution processing is not limited here. For example, if the convolution kernel of the convolution layer 5 is (3, 3), the convolution layer 5 can perform 3×3 convolution processing on the mapping result. By performing convolution processing on the output data of the convolution layer 4 through the convolution layer 5, the features of interest are extracted from the output data of the convolution layer 4, the noise is removed, and the representation ability of the deep features is improved, thereby improving the quality of the second image.

[0090] The method for determining deep features is described above. Next, the content of shallow features is introduced. In this example, as Figure 3 shown, the super-resolution model further includes a shallow feature extraction module 302. The first luminance information 01 is input into the shallow feature extraction module 302, and the shallow feature extraction module 302 performs shallow feature extraction on the first luminance information 01 to obtain and output shallow features. Shallow features refer to features extracted through at least one layer of neural network, where the number of layers of the neural network is less than a threshold. Shallow features have a smaller receptive field, are closer to the input data, and contain more pixel point information. For example, shallow features contain fine-grained information such as color, texture, edges, and corners.

[0091] In practical applications, the shallow feature extraction module 302 can have different structures. Different structures of the shallow feature extraction module 302 result in different ways of extracting shallow features. A possible implementation is shown below.

[0092] As Figure 4 shown, the shallow feature extraction module 302 includes a convolutional layer 6. The convolutional layer 6 is used to perform convolutional processing on the first luminance information. The scale of the convolutional processing is not limited here. For example, the convolution kernel of the convolutional layer 6 is (5, 5), and the convolutional layer 6 can perform 5×5 convolutional processing on the first luminance information to obtain shallow features.

[0093] Step S2: Determine the second luminance information based on the shallow features and the deep features.

[0094] As Figure 3 shown, the super-resolution model includes an upsampling module 303. The addition result obtained by adding the shallow features (i.e., the output data of the shallow feature extraction module 302) and the deep features (i.e., the output data of the deep feature extraction module 301) can be input into the upsampling module 303, and the upsampling module 303 determines and outputs the first luminance information 02. In practical applications, the upsampling module 303 can have different structures. Different structures of the upsampling module 303 result in different ways of determining the second luminance information. A possible implementation is shown below.

[0095] As Figure 4 shown, the upsampling module 303 includes a convolutional layer 7 and a permutation module. The convolutional layer 7 is used to perform convolutional processing on the addition result obtained by adding the shallow features and the deep features. The scale of the convolutional processing is not limited here. For example, the convolution kernel of the convolutional layer 7 is (3, 3), and the convolutional layer 7 can perform 3×3 convolutional processing on the addition result. The permutation module is used to permute the output data of the convolutional layer 7 to obtain and output the second luminance information. Among them, the magnification of the permutation module is N, where N is a positive integer greater than 1, and the output data of the convolutional layer 7 includes N2 The feature maps of one channel, and the feature map of each channel includes the feature data of each pixel point in the first image. For each pixel point, the feature data of this pixel point in N 2 channels are rearranged in N rows and N columns to obtain the feature data of four pixel points, thereby achieving N-fold magnification in both the horizontal and vertical directions. That is to say, by arranging the output data of the convolutional layer 7, the image is magnified and the resolution of the image is improved. In practical applications, N can be flexibly set according to the actual application scenario. For example, N can be 2 or 4. N being 2 is equivalent to magnifying 2 times in both the horizontal and vertical directions, and N being 4 is equivalent to magnifying 4 times in both the horizontal and vertical directions.

[0096] By extracting the shallow features and deep features of the first luminance information and determining the second luminance information based on the shallow features and deep features, the second luminance information contains both the global content and the detailed content of the image, improving the accuracy of the second luminance information, and thus the quality of the second image can be improved.

[0097] Step 204, interpolate the first color information to obtain the second color information.

[0098] The first color information can be interpolated by any interpolation method to magnify the resolution of the first color information and obtain the second color information. For example, the interpolation methods include bicubic interpolation method, bilinear interpolation method, nearest neighbor interpolation method, area interpolation method, Lanczos interpolation method, etc. The computational resources and storage resources required by the interpolation method are less than those of the artificial intelligence-based image processing method, which is more conducive to being applied to chips and mobile terminals.

[0099] In a possible implementation manner, the first color information includes the color values of each first pixel point in the first image, and the second color information includes the color values of each first pixel point and the color values of at least two second pixel points. When the display chip 101 executes step 204, it is configured to: for any second pixel point, determine the adjacent pixel points located around any second pixel point from each first pixel point, and interpolate the color value of any second pixel point based on the color values of the adjacent pixel points; determine the second color information based on the color values of each first pixel point and the color values of each second pixel point.

[0100] In this example, based on the color values of each first pixel in the first color information, the color values of at least two second pixels can be interpolated. When interpolating, for any second pixel, the adjacent pixels located around the second pixel are determined from each first pixel, and the number of adjacent pixels is at least two. For example, any adjacent pixel can be located to the left, right, above, below, upper left, lower left, upper right, or lower right of the second pixel. Then, the color value of the second pixel is interpolated based on the color values of each adjacent pixel.

[0101] Optionally, the first image includes first pixels arranged in M rows and N columns. Taking the first pixel at the 0th row and 0th column as the origin, a coordinate system is established as follows Figure 6The rectangular coordinate system shown. For the second pixel point 601, four adjacent pixel points can be determined from the first pixel points in M rows and N columns. For ease of description, the four adjacent pixel points are sequentially represented by labels 602-1 to 602-4. First, based on the color value of the first pixel point 602-1 and the color value of the first pixel point 602-2, the color value of the central pixel point 603-1 is interpolated. The interpolation formula is as follows: color_interp(r-, c) = color(y, x) + (color(y, x + 1) - color(y, x)) × dx. Where, color_interp(r-, c) represents the color value of the central pixel point 603-1, color(y, x) represents the color value of the first pixel point 602-1, color(y, x + 1) represents the color value of the first pixel point 602-2, and dx represents the distance between the central pixel point 603-1 and the first pixel point 602-1. Then, based on the color value of the first pixel point 602-3 and the color value of the first pixel point 602-4, the color value of the central pixel point 603-2 is interpolated. The interpolation formula is as follows: color_interp(r+, c) = color(y + 1, x) + (color(y + 1, x + 1) - color(y + 1, x)) × dx. Where, color_interp(r+, c) represents the color value of the central pixel point 603-2, color(y + 1, x) represents the color value of the first pixel point 602-3, color(y + 1, x + 1) represents the color value of the first pixel point 602-4, and dx represents the distance between the central pixel point 603-2 and the first pixel point 602-3. After that, based on the color value of the central pixel point 603-1 and the color value of the central pixel point 603-2, the color value of the second pixel point 601 is interpolated. The interpolation formula is as follows: color_interp(r, c) = color_interp(r-, c) + (color_interp(r+, c) - color_interp(r-, c)) × dy. Where, color_interp(r, c) represents the color value of the second pixel point 601, and dy represents the distance between the second pixel point 601 and the central pixel point 603-1.

[0102] In the above manner, based on the color values of each first pixel point, the color values of at least two second pixel points can be interpolated. The color values of these first pixel points and the second pixel points form the second color information. By interpolating the color value of the second pixel point through the color values of the adjacent pixel points around the second pixel point, the natural transition of colors is achieved, making the picture of the second image look more natural and vivid, which is beneficial to improving the quality of the second image.

[0103] In practical applications, the color channels of the first image can be at least two. For each color channel, the second color information of the color channel can be determined based on the first color information of the color channel in the above-mentioned manner. For example, if the color channels include the Cb channel and the Cr channel, the second color information of the Cb channel can be determined based on the first color information of the Cb channel, and the second color information of the Cr channel can be determined based on the first color information of the Cr channel. Among them, the number of pixel points corresponding to the second color information is the same as the number of pixel points corresponding to the second luminance information.

[0104] Step 205: Determine a second image with a second resolution based on the second luminance information and the second color information, where the second resolution is greater than the first resolution.

[0105] After determining the second luminance information and the second color information, a high-resolution image can be determined, and this high-resolution image can be the second image. For example, each pixel point in the second image corresponds to a luminance channel and a color channel. The second luminance information includes the values of each pixel point in the second image in the luminance channel, and the second color information includes the values of each pixel point in the second image in the color channel. Alternatively, the high-resolution image can be format-converted to obtain the second image so that the format of the second image is consistent with that of the first image. It can be understood that the format conversion here is the inverse transformation of the format conversion performed on the first image. For example, if the first image is format-converted according to formula (1) mentioned above, the high-resolution image can be converted according to formula (3) shown below.

[0106]

[0107] Among them, represents the values of each pixel point in the high-resolution image in the Y channel, Cb channel, and Cr channel, where the Y channel is the luminance channel, and the Cb channel and the Cr channel are both color channels. represents the values of each pixel point in the second image in the R channel, G channel, and B channel, where the R channel is the red channel, the G channel is the green channel, and the B channel is the blue channel, and these three channels all belong to the color channels. represents the weight matrix, represents the bias matrix.

[0108] It should be noted that the values of each element in the above weight matrix and bias matrix need to comply with various industry standards. Taking the industry standard BT 709 of the high-definition television industry as an example, assuming that the bit width of the image signal is 8 bits (bit), formula (3) above can be specifically formula (4) shown below.

[0109]

[0110] In the above manner, super-resolution of the first image is achieved to obtain a second image, and the second resolution of the second image is greater than the first resolution of the first image. Here, the specific value of the second resolution is not limited.

[0111] Optionally, as Figure 7 shown, first, a first conversion process (i.e., format conversion) is performed on the first image 701 to convert the RGB image format into the YCbCr image format. Among them, the data of the first image 701 in the Y channel is the first luminance information, and the first luminance information can be processed by a super-resolution network to obtain the second luminance information. The first color information includes the data of the first image 701 in the Cb channel and the data of the first image 701 in the Cr channel, and the first color information can be interpolated to obtain the second color information. Then, a second conversion process (i.e., format conversion) is performed on the second luminance information and the second color information to convert the YCbCr image format into the RGB image format, obtaining the second image 702.

[0112] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.), and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards in the relevant region. For example, the first image involved in this application is obtained under full authorization.

[0113] In the above display chip, since the human eye is very sensitive to luminance information and relatively insensitive to color information, and compared with interpolating an image, processing an image through a model can better improve the quality of the image. Therefore, by processing the first luminance information of a low-resolution image through a super-resolution model to obtain the second luminance information, interpolating the first color information of the low-resolution image to obtain the second color information, and determining the second image based on the second luminance information and the second color information, the quality of the high-resolution image can be ensured.

[0114] Since the luminance information and the color information are processed separately, it is such that: compared with processing the first luminance information and the first color information through a super-resolution model, only processing the first luminance information through a super-resolution model can reduce the amount of data to be processed, thereby saving computing resources and storage resources. Compared with processing the first color information through a super-resolution model, interpolating the first color information can save more computing resources and storage resources. Since the solution of this application occupies less computing resources and storage resources, this solution can be deployed in a chip, and the chip can be deployed on mobile and fixed terminals, greatly enriching the application scenarios.

[0115] An embodiment of the present application also provides an image processing method, which is executed by a display chip. As Figure 2 shown, the method includes the following steps.

[0116] Step 201, obtain a first image with a first resolution.

[0117] Step 202, determine first brightness information and first color information based on the first image.

[0118] Step 203, process the first brightness information through a super-resolution model to obtain second brightness information.

[0119] Step 204, interpolate the first color information to obtain second color information.

[0120] Step 205, determine a second image with a second resolution based on the second brightness information and the second color information, where the second resolution is greater than the first resolution.

[0121] In a possible implementation, processing the first brightness information to obtain the second brightness information includes:

[0122] Extract features of the first brightness information to obtain shallow features and deep features, where the shallow features represent the detailed content of the first brightness information, and the deep features represent the global content of the first brightness information;

[0123] Determine the second brightness information based on the shallow features and the deep features.

[0124] In a possible implementation, extracting the deep features from the first brightness information includes:

[0125] Perform convolution processing on the first brightness information to obtain a convolution processing result;

[0126] Perform a non-linear mapping on the convolution processing result to obtain a mapping result;

[0127] Perform convolution processing on the mapping result to obtain deep features.

[0128] In a possible implementation, performing convolution processing on the first brightness information to obtain a convolution processing result includes:

[0129] Perform convolution processing on the first brightness information at different scales to obtain at least two first processing results;

[0130] Concatenate at least two first processing results to obtain a first concatenation result;

[0131] Perform convolution processing on the first concatenation result to obtain a convolution processing result.

[0132] In a possible implementation, performing a non-linear mapping on the convolution processing result to obtain a mapping result, including:

[0133] Based on the convolution processing result, determining channel features of at least two channels;

[0134] For any one channel, based on the channel feature of any one channel, determining the weight information of any one channel, where the weight information of any one channel characterizes the importance degree of the features of each pixel point in the channel feature of any one channel to the quality of the second image;

[0135] Based on the channel features and weight information of each channel, determining the mapping result.

[0136] In a possible implementation, based on the channel feature of any one channel, determining the weight information of any one channel, including:

[0137] Based on the channel feature of any one channel, determining the first weight of each pixel point in any one channel, where the first weight of a pixel point in any one channel characterizes the importance degree of the feature of a pixel point in the channel feature of any one channel to the quality of the second image;

[0138] Obtaining the second weight of any one channel, where the second weight of any one channel characterizes the importance degree of the channel feature of any one channel to the quality of the second image;

[0139] Based on the first weight of each pixel point in any one channel and the second weight of any one channel, determining the weight information of any one channel.

[0140] In a possible implementation, based on the convolution processing result, determining channel features of at least two channels, including:

[0141] Based on the convolution processing result, determining the second processing result of each channel;

[0142] Based on the second processing result of each channel, determining the third processing result of each channel;

[0143] For any one channel, adding the second processing result and the third processing result of any one channel to obtain the channel feature of any one channel.

[0144] In a possible implementation, based on the channel features and weight information of each channel, determining the mapping result, including:

[0145] For any one channel, multiplying the channel feature and the weight information of any one channel to obtain the multiplication result of any one channel, and adding the convolution processing result and the multiplication result of any one channel to obtain the addition result of any one channel;

[0146] Determine the mapping result based on the addition result of each channel.

[0147] In a possible implementation, the first color information includes the color values of each first pixel point in the first image, and the second color information includes the color values of each first pixel point and the color values of at least two second pixel points; interpolating the first color information to obtain the second color information includes:

[0148] For any second pixel point, determine the adjacent pixel points located around any second pixel point from each first pixel point, and interpolate the color value of any second pixel point based on the color values of the adjacent pixel points;

[0149] Determine the second color information based on the color values of each first pixel point and the color values of each second pixel point.

[0150] In the above method, since the human eye is very sensitive to brightness information and relatively insensitive to color information, and compared with interpolating an image, processing an image through a model can better improve the quality of the image. Therefore, processing the first brightness information of the low-resolution image through a super-resolution model to obtain the second brightness information, interpolating the first color information of the low-resolution image to obtain the second color information, and determining the second image based on the second brightness information and the second color information can ensure the quality of the high-resolution image.

[0151] Since the brightness information and the color information are processed separately, it is such that: compared with processing the first brightness information and the first color information through a super-resolution model, only processing the first brightness information through a super-resolution model can reduce the amount of data to be processed, thereby saving computing resources and storage resources. Compared with processing the first color information through a super-resolution model, interpolating the first color information can save more computing resources and storage resources. Since the solution of the present application occupies less computing resources and storage resources, the solution can be deployed in a chip, and the chip can be deployed in mobile and fixed terminals, greatly enriching the application scenarios.

[0152] It should be understood that the above method embodiments and the embodiments of the display chip belong to the same concept, and the specific implementation process is detailed in the embodiments of the display chip, which will not be elaborated here.

[0153] In an exemplary embodiment, as Figure 8 shown, there is also provided an electronic device 800, and the electronic device 800 includes a display chip 101, and the display chip 101 is configured to execute an image processing method related to Figure 2 this.

[0154] It should be understood that the "plurality" mentioned herein refers to two or more. "And / or" describes the association relationship of associated objects and indicates that three relationships may exist. For example, A and / or B may represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0155] The serial numbers of the embodiments of the present application above are only for description and do not represent the superiority or inferiority of the embodiments.

[0156] The above are only exemplary embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the principle of the present application shall be included in the protection scope of the present application.

Claims

1. A display chip, characterized in that: The chip is applied to a display screen, and the chip is configured as follows: Acquire a first image with a first resolution; determining first brightness information and first color information based on the first image; Processing the first brightness information through a super-resolution model to obtain second brightness information; interpolating the first color information to obtain second color information; A second image with a second resolution is determined based on the second brightness information and the second color information, where the second resolution is greater than the first resolution.

2. The chip according to claim 1, characterized in that: When the chip processes the first brightness information to obtain the second brightness information, the chip is configured as follows: Extracting features of the first brightness information to obtain shallow features and deep features, wherein the shallow features represent detailed content of the first brightness information, and the deep features represent global content of the first brightness information; Based on the shallow features and the deep features, second brightness information is determined.

3. The chip according to claim 2, characterized in that: When the chip extracts the features of the first brightness information and obtains the deep features, the chip is configured as follows: Performing convolution processing on the first brightness information to obtain a convolution processing result; Performing nonlinear mapping on the convolution processing result to obtain a mapping result; The mapping result is subjected to convolution processing to obtain deep features.

4. The chip according to claim 3, characterized in that: When the chip performs convolution processing on the first brightness information to obtain a convolution processing result, the chip is configured as follows: Performing convolution processing of different scales on the first brightness information to obtain at least two first processing results; splicing the at least two first processing results to obtain a first splicing result; Perform convolution processing on the first splicing result to obtain a convolution processing result.

5. The chip according to claim 3, characterized in that: When the chip performs nonlinear mapping on the convolution processing result to obtain the mapping result, the chip is configured as follows: Based on the convolution processing result, determining channel characteristics of at least two channels; For any channel, based on the channel characteristics of the any channel, determine the weight information of the any channel, wherein the weight information of the any channel represents the importance of the characteristics of each pixel point in the channel characteristics of the any channel to the quality of the second image; Based on the channel characteristics and weight information of each channel, the mapping result is determined.

6. The chip according to claim 5, characterized in that: When determining the weight information of any one of the channels based on the channel characteristics of any one of the channels, the chip is configured as follows: Based on the channel characteristics of any one of the channels, determining a first weight of each pixel in the any one of the channels, wherein the first weight of a pixel in the any one of the channels represents the importance of a feature of the pixel in the channel characteristics of the any one of the channels to the quality of the second image; Acquire a second weight of any one of the channels, where the second weight of any one of the channels represents the importance of a channel feature of any one of the channels to the quality of the second image; Based on the first weight of each pixel in any one of the channels and the second weight of any one of the channels, weight information of any one of the channels is determined.

7. The chip according to any one of claims 1 to 6, characterized in that: The first color information includes the color value of each first pixel in the first image, and the second color information includes the color value of each first pixel and the color value of at least two second pixels; when the chip interpolates the first color information to obtain the second color information, it is configured as follows: For any second pixel point, determine neighboring pixel points located around the second pixel point from the first pixel points, and obtain a color value of the second pixel point by interpolation based on the color values ​​of the neighboring pixel points; The second color information is determined based on the color values ​​of the first pixels and the color values ​​of the second pixels.

8. An image processing method, characterized in that: The method comprises: Acquire a first image with a first resolution; determining first brightness information and first color information based on the first image; Processing the first brightness information through a super-resolution model to obtain second brightness information; interpolating the first color information to obtain second color information; A second image with a second resolution is determined based on the second brightness information and the second color information, where the second resolution is greater than the first resolution.

9. The method according to claim 8, characterized in that The processing of the first brightness information to obtain second brightness information includes: Extracting features of the first brightness information to obtain shallow features and deep features, wherein the shallow features represent detailed content of the first brightness information, and the deep features represent global content of the first brightness information; Based on the shallow features and the deep features, second brightness information is determined.

10. The method according to claim 9, characterized in that Extracting features of the first brightness information to obtain deep features includes: Performing convolution processing on the first brightness information to obtain a convolution processing result; Performing nonlinear mapping on the convolution processing result to obtain a mapping result; The mapping result is subjected to convolution processing to obtain deep features.

11. The method according to claim 10, characterized in that The performing convolution processing on the first brightness information to obtain a convolution processing result includes: Performing convolution processing of different scales on the first brightness information to obtain at least two first processing results; splicing the at least two first processing results to obtain a first splicing result; Perform convolution processing on the first splicing result to obtain a convolution processing result.

12. The method according to claim 10, characterized in that The performing nonlinear mapping on the convolution processing result to obtain a mapping result includes: Based on the convolution processing result, determining channel characteristics of at least two channels; For any channel, based on the channel characteristics of the any channel, determine the weight information of the any channel, wherein the weight information of the any channel represents the importance of the characteristics of each pixel point in the channel characteristics of the any channel to the quality of the second image; Based on the channel characteristics and weight information of each channel, the mapping result is determined.

13. The method according to claim 12, characterized in that The determining the weight information of any one of the channels based on the channel characteristics of any one of the channels comprises: Based on the channel characteristics of any one of the channels, determining a first weight of each pixel in the any one of the channels, wherein the first weight of a pixel in the any one of the channels represents the importance of a feature of the pixel in the channel characteristics of the any one of the channels to the quality of the second image; Acquire a second weight of any one of the channels, where the second weight of any one of the channels represents the importance of a channel feature of any one of the channels to the quality of the second image; Based on the first weight of each pixel in any one of the channels and the second weight of any one of the channels, weight information of any one of the channels is determined.

14. The method according to any one of claims 8 to 13, characterized in that The first color information includes the color value of each first pixel in the first image, and the second color information includes the color value of each first pixel and the color value of at least two second pixels; and interpolating the first color information to obtain the second color information includes: For any second pixel point, determine neighboring pixel points located around the second pixel point from the first pixel points, and obtain a color value of the second pixel point by interpolation based on the color values ​​of the neighboring pixel points; The second color information is determined based on the color values ​​of the first pixels and the color values ​​of the second pixels.

15. An electronic device, characterized in that: The electronic device comprises the chip as described in any one of claims 1 to 7.

Images