Image processing method and device

Abstract

The invention discloses an image processing method and device. The method comprises the steps of obtaining a to-be-processed first image; obtaining a target sharpening value, wherein the target sharpening value is used for representing a target sharpening degree of sharpening processing on the image; adjusting a weight coefficient in the image processing model according to the target sharpening value; the first image is processed based on the image processing model, a generated second image is obtained, and the sharpening degree of the second image is different from that of the first image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method and device. Background Art

[0002] Image sharpening is an image processing technique that improves the visual clarity of an image by enhancing high-frequency details (such as edges or textures) in the image.

[0003] Currently, the degree of image sharpening applied to images using image processing models is fixed. However, different users have different requirements for image sharpening. Even the same user may have different requirements for image sharpening in different image processing scenarios. Therefore, how to reasonably control image sharpening based on different sharpening requirements is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] In one aspect, the present application provides an image processing method, comprising:

[0005] obtaining a first image to be processed;

[0006] Obtaining a target sharpening value, wherein the target sharpening value is used to represent a target sharpening degree of sharpening processing performed on the image;

[0007] Adjusting the weight coefficients in the image processing model according to the target sharpening value;

[0008] The first image is processed based on the image processing model to obtain a generated second image, where a sharpening degree of the second image is different from a sharpening degree of the first image.

[0009] In a possible implementation, adjusting the weight coefficient in the image processing model according to the target sharpening value includes:

[0010] Determining a target weight coefficient corresponding to the last convolution operation in the image processing model according to the target sharpening value;

[0011] The weight coefficient corresponding to the last layer of convolution operation in the image processing model is set as the target weight coefficient.

[0012] In another possible implementation, determining, according to the target sharpening value, a target weight coefficient corresponding to a last convolution operation in an image processing model includes:

[0013] Based on the target sharpening value and in combination with a conversion function between the configured sharpening value and the weight coefficient, a target weight coefficient corresponding to the last layer of convolution operation in the image processing model is determined.

[0014] In another possible implementation, determining the target weight coefficient corresponding to the last convolution operation in the image processing model based on the target sharpening value in combination with a conversion function between the configured sharpening value and the weight coefficient includes:

[0015] Calling and running a conversion model through a target processor, wherein the conversion model is configured with a conversion function between a sharpening value and a weight coefficient, and the target processor is a processor running the image processing model;

[0016] Based on the target sharpening value, a target weight coefficient corresponding to the last convolution operation in the image processing model is determined using a conversion function in the conversion model.

[0017] In another possible implementation, the processing effect of the target weight coefficient determined based on the conversion function on the image data is equivalent to the sharpening effect produced by processing the image data through the first equivalent filter and the cascade filter respectively and superimposing the image processing results, and the cascade filter is a cascade of the first equivalent filter and the second equivalent filter.

[0018] In another possible implementation, the conversion function is a function that uses the sharpening value as a weighting coefficient and performs a weighted operation on the first weighting coefficient and the second weighting coefficient;

[0019] Wherein, the first weight coefficient is equivalent to the first filter coefficient of the first equivalent filter;

[0020] The second weight coefficient is equivalent to the second filter coefficient of the second equivalent filter.

[0021] In another possible implementation, the conversion function is a function that uses a sharpening value as a first weighting coefficient of the cascade filter, uses a difference between 1 and the sharpening value as a second weighting coefficient of the first equivalent filter, and performs a weighted operation on the first filter coefficient of the first equivalent filter and the third filter coefficient of the cascade filter.

[0022] The third filter coefficient of the cascade filter is a tensor product of the first filter coefficient of the first equivalent filter and the second filter coefficient of the second equivalent filter.

[0023] In another possible implementation, the second equivalent filter is a filter used for performing image sharpening processing based on a gradient sharpening algorithm;

[0024] The first equivalent filter is equivalent to the last convolution layer in the image processing model.

[0025] In another possible implementation, the method further includes: calling and running the image processing model through a target processor;

[0026] Setting the weight coefficient corresponding to the last convolution operation in the image processing model as the target weight coefficient includes:

[0027] The target weight coefficient is stored in a designated cache area in the target processor, and the target storage address of the weight coefficient corresponding to the last layer of convolution operation in the image processing model is set to the address of the target weight coefficient in the designated cache area.

[0028] In another aspect, the present application further provides an image processing device, comprising:

[0029] an image obtaining unit, configured to obtain a first image to be processed;

[0030] A sharpening value obtaining unit, configured to obtain a target sharpening value, wherein the target sharpening value is used to represent a target sharpening degree for performing sharpening processing on an image;

[0031] A coefficient adjustment unit, configured to adjust a weight coefficient in an image processing model according to the target sharpening value;

[0032] An image processing unit is configured to process the first image based on the image processing model to obtain a generated second image, wherein a sharpening degree of the second image is different from a sharpening degree of the first image. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0034] Figure 1 A flowchart of the image processing method provided in this application;

[0035] Figure 2 A schematic diagram of another flow chart of the image processing method provided by this application;

[0036] Figure 3 A schematic diagram of a model architecture for adjusting image sharpness in this application;

[0037] Figure 4 An example diagram of a convolutional layer of a neural network performing convolution processing on the pixels of an image;

[0038] Figure 5 A schematic diagram of another flow chart of the image processing method provided by this application;

[0039] Figure 6FIG. 1 shows an example of an implementation framework for adjusting weight coefficients of the last layer of convolution operations of an image processing model in the present application;

[0040] Figure 7 FIG. 2 shows an example of a component structure of an image processing apparatus provided in the present application;

[0041] Figure 8 FIG. 3 shows an example of a component structure of an electronic device provided in the present application. DETAILED DESCRIPTION

[0042] The embodiments of the present application are described below with reference to the accompanying drawings. The terms used in the implementation part of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application. It is known to those skilled in the art that, with the development of technology and the appearance of new scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0043] The terms "first", "second", and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, and this is only a way of distinguishing the objects with the same attributes in the description of the embodiments of the present application. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, so that the processes, methods, systems, products or devices containing a series of units do not have to be limited to those units, but can include other units not clearly listed or inherent to these processes, methods, products or devices.

[0044] As Figure 1 FIG. 1 shows an example of an implementation flowchart of an image processing method provided in the present application. The method of the present embodiment can be applied to an electronic device, which can be any electronic device with image processing capability, such as a personal computer or a server, etc., without limitation.

[0045] The method of the present embodiment can include:

[0046] S101, obtaining a first image to be processed.

[0047] The first image can be an image that needs to be sharpened.

[0048] S102, obtaining a target sharpening value.

[0049] For example, obtaining a target sharpening value input by a user, or obtaining a configured target sharpening value, etc.

[0050] The target sharpening value is used to represent the target sharpening degree of the image sharpening process.

[0051] Sharpening an image refers to enhancing the edge and contour details in the image by processing the image. Based on this, the sharpening degree of the image reflects the contrast of the edge and details, the contour definition, and the visual perception intensity of the texture in the image.

[0052] S103, adjust the weight coefficient in the image processing model according to the target sharpening value.

[0053] The image processing model can be used at least for sharpening the image.

[0054] Further, the image processing model can also be used for enhancing the image before sharpening the image. On this basis, the image processing model is a model with image enhancement processing capability and image sharpening capability. The image enhancement processing includes but is not limited to image denoising and image resolution enhancement, and the enhancement processing required by the image processing model will be different according to the application scene of image processing, which is not limited.

[0055] Unlike the traditional method of enhancing and sharpening the image by two models respectively, the image processing model in the present application can be regarded as integrating the image sharpening processing function into the model with image enhancement processing function. The model type of the image processing model can have multiple possibilities, which is not limited.

[0056] For example, in the traditional image sharpening processing, a neural network with convolution layer such as Convolutional Neural Network (CNN) or Recurrent Neural Network (RNN) is used to enhance the image, and a filter is connected after the neural network to sharpen the enhanced image. On this basis, the image processing model of the present application can be a model obtained by configuring the weight coefficient of the last convolution operation in the neural network as an adjustable weight coefficient. Correspondingly, after determining the weight coefficient corresponding to the image processing model, the processing effect of processing the image based on the weight coefficient can be equivalent to the processing effect of processing the image by the convolution layer in the last layer of the neural network and the filter connected to the convolution layer.

[0057] For example, in a traditional image processing scenario, when an image is enhanced by an artificial intelligence model such as a large language model, the weight coefficient of the artificial intelligence model can be set as the weight coefficient determined based on the target sharpening value, so that the artificial intelligence model can be used as the image processing model. Based on this, after the weight coefficient of the artificial intelligence model is determined based on the target sharpening value, the image is processed by the artificial intelligence model based on the weight coefficient, which can achieve the purpose of enhancing the image and sharpening the image. For example, based on the weight coefficient determined based on the target sharpening value, the image is processed by the artificial intelligence model, which can achieve the effect of using a super-resolution model to super-resolve the image and sharpening the super-resolved image.

[0058] Unlike the weight coefficient of a conventional image processing model, which is fixed, in this application, the weight coefficient in the image processing model is adjustable. The weight coefficient in the image processing model can be dynamically adjusted based on different target sharpening values, so that the sharpening degree of the image processing model after the weight coefficient adjustment can match the target sharpening degree corresponding to the target sharpening value.

[0059] S104, processing the first image based on the image processing model to obtain a generated second image.

[0060] The sharpening degree of the second image is different from the sharpening degree of the first image.

[0061] It can be understood that since the weight coefficient in the image processing model is adjusted based on the target sharpening value, the first image is processed by the image processing model after the weight coefficient adjustment, which can make the sharpening degree of the processed second image reach the target sharpening degree corresponding to the target sharpening value.

[0062] From the above, it can be seen that after obtaining the first image, the weight coefficient in the image processing model can be adjusted according to the desired target sharpening value. Since the target sharpening value is used to represent the target sharpening degree of the image sharpening process, based on the target sharpening value, the weight parameter of the image processing model can be adjusted, so that the image processing model after the weight parameter adjustment can sharpen the first image based on the target sharpening degree, so that the sharpening degree of the generated second image meets the sharpening degree requirement, thereby realizing reasonable control of the sharpening process of the image according to different requirements of the sharpening degree, and improving the flexibility of the image sharpening process.

[0063] In this application, the specific implementation of adjusting the weight coefficient of the image processing model can have multiple possibilities. When the sharpening algorithm used by the image processing model for sharpening processing is different, the specific implementation of adjusting the weight coefficient of the image processing model will also be different, which is not limited.

[0064] In one possible implementation, when the last layer of the image processing model is a convolution layer, the processing of the image by the last layer of the image processing model can affect the final sharpening effect of the image. Based on this, the present application sets the weight coefficient of the convolution operation of the last layer of the image processing model as a dynamically adjustable weight coefficient. On this basis, the present application can determine the target weight coefficient corresponding to the last layer of convolution operation in the image processing model according to the target sharpening value. Accordingly, the weight coefficient corresponding to the last layer of convolution operation in the image processing model can be set to the target weight coefficient.

[0065] There are also multiple possibilities for determining the target weight coefficient according to the target sharpening value.

[0066] For example, weight coefficients corresponding to different sharpening values ​​may be pre-configured, and accordingly, a target weight coefficient corresponding to a target sharpening value may be determined.

[0067] In a possible implementation, the present application may also pre-configure a conversion function for determining a weight coefficient based on a sharpening value, and use the conversion function to determine the target weight value. Figure 2 Provide explanation.

[0068] like Figure 2 , shows another flow chart of the image processing method provided by the present application. The method of this embodiment may include:

[0069] S201: Obtain a first image to be processed.

[0070] S202: Obtain a target sharpening value.

[0071] The target sharpening value is used to represent the target sharpening degree of the image sharpening process.

[0072] S203: Based on the target sharpening value and in combination with a conversion function between the configured sharpening value and the weight coefficient, determine the target weight coefficient corresponding to the last layer of convolution operation in the image processing model.

[0073] Among them, the conversion function is a function used to determine the weight coefficient corresponding to the last layer of convolution operation in the image processing model, and the sharpening value can be converted into a weight coefficient through the conversion function. Based on this, the target weight coefficient corresponding to the target sharpening value can be determined using the conversion function.

[0074] In the present application, there are many possible specific forms of the conversion function, which can be set according to actual needs.

[0075] In one possible implementation, the processing effect of the target weight coefficient determined based on the transfer function on the image data is equivalent to the sharpening effect produced by processing the image data through the first equivalent filter and the cascade filter, respectively, and superimposing the image processing results. The cascade filter is a cascade of the first equivalent filter and the second equivalent filter.

[0076] The superimposing of the image processing results refers to superimposing the first processing result of the image data by the first equivalent filter and the second processing result of the image data by the cascade filter.

[0077] When superimposing the first processing result and the second processing result, since the proportions of the first processing result and the second processing result are different, the superposition methods of the first processing result and the second processing result are different, so the sharpness of the superimposed image can be different. In this application, the sharpening values ​​are different, and the target weight coefficients determined based on the conversion function are also different. The processing effects of different target weight coefficients on the image are equivalent to superimposing the first processing result and the second processing result in different forms. However, for a certain determined target weight coefficient, the processing effect on the image data based on the target weight coefficient is equivalent to the superposition form of the first processing result and the second processing result, which is also determined, and the sharpening effect produced by superimposing the first processing result and the second processing result is fixed.

[0078] For example, the second equivalent filter is a filter for sharpening an image, while the first equivalent filter may be a filter for enhancing or otherwise processing the image before the second equivalent filter performs the image sharpening. The first processing result obtained by applying the first equivalent filter to the image data is effectively an unsharpened image, while the second processing result obtained by applying the cascaded filters to the image data is a sharpened image. Based on this, the first and second processing results can be superimposed to varying degrees to produce a superimposed image with varying degrees of sharpness.

[0079] It is understandable that there are many algorithms for sharpening images. Considering that in the gradient-based sharpening method, it involves extracting edge information through an edge detection convolution kernel, and extracting edge information through an edge detection convolution kernel can be achieved through a filter. Normally, in order to improve the image sharpening effect, the image is generally first convolved through a neural network to perform image enhancement and other processing. In the gradient-based sharpening method, the convolution in the neural network can also be equivalent to a filter. Based on this, in one possible implementation method, the first equivalent filter can be equivalent to the last convolution layer in the image processing model in this application, and the second equivalent filter is a filter for image sharpening based on the gradient sharpening algorithm.

[0080] On this basis, the present application constructs a conversion function, and makes the processing effect of the target weight coefficient determined based on the conversion function on the image data equivalent to the sharpening effect produced by processing the image data through the first equivalent filter and the cascade filter respectively and superimposing the image processing results. It can be achieved by only changing the weight coefficient of the last layer of convolution operation in the image processing model, thereby achieving the effect of processing the image data through the first equivalent filter and the cascade filter respectively and superimposing the image processing results.

[0081] In the present application, the specific form of the conversion function may have many possibilities, which can be set according to actual needs and is not limited to this.

[0082] For example, in one possible case of a conversion function, the conversion function may be a function that uses a sharpening value as a weighting coefficient and performs a weighted operation on a first weighting coefficient and a second weighting coefficient. The first weighting coefficient is equivalent to the first filter coefficient of any of the first equivalent filters mentioned above, and the second weighting coefficient is equivalent to the second filter coefficient of any of the second equivalent filters mentioned above.

[0083] It can be understood that, under the premise of the cascade connection of the first and second filters, using the sharpening value as the weighting coefficient, the effect achieved by performing a weighted operation on the first filter coefficient of the first equivalent filter and the second filter coefficient of the second equivalent filter is the effect of performing a weighted operation on the image output after processing by the first filter and the image output after processing by the first and second filters. On this basis, when the sharpening value used as the weighting coefficient is different, the degree of sharpening of the image obtained by performing the weighted operation on the image output by the first filter and the image output by the second filter will also be different.

[0084] Furthermore, the conversion function may be a function that uses the sharpening value as the first weighting coefficient of the cascade filter, uses the difference between 1 and the sharpening value as the second weighting coefficient of the first equivalent filter, and performs a weighted operation on the first filter coefficient of the first equivalent filter and the third filter coefficient of the cascade filter. The third filter coefficient of the cascade filter is the dot product of the first filter coefficient of the first equivalent filter and the second filter coefficient of the second equivalent filter.

[0085] S204: Set the weight coefficient corresponding to the last convolution operation in the image processing model as the target weight coefficient.

[0086] S205: Process the first image based on the image processing model to obtain a generated second image.

[0087] The sharpening degree of the second image is different from the sharpening degree of the first image.

[0088] To facilitate understanding of the benefits and significance of using the conversion function to determine the target weight coefficient based on the target sharpening value in the embodiment, the following uses the application scenario of sharpening an image using a gradient-based sharpening algorithm as an example to deduce an implementation process of obtaining the conversion function of the present application.

[0089] In the application scenario of image sharpening based on gradient sharpening algorithm, the image can be enhanced and preprocessed by using the convolution layer of the neural network before being input into the gradient-based image sharpening filter. On this basis, in order to dynamically adjust the sharpness of the image and reduce calculations, the convolution layer of the neural network can also be equivalent to a filter. Then the model architecture that can adjust the sharpness of the image can be as follows: Figure 3 shown.

[0090] Depend on Figure 3 It can be seen that:

[0091] During the image sharpening process, the image needs to be processed by a neural network first. The last layer in the neural network is a convolutional layer. After the convolutional layer in the neural network performs a convolution operation on the image, the image processed by the convolutional layer will be input into a second filter, which is used to sharpen the image.

[0092] Among them, the convolution layer of the neural network can perform convolution processing on the pixels in the original image based on the convolution kernel to obtain the target pixel value corresponding to the pixel, such as Figure 4 An example diagram showing how the convolutional layer of a neural network performs convolution processing on the pixels of an image. Figure 4 In the figure, the convolution kernel 401 is taken as an example of a 3 by 3 matrix. On this basis, for the pixel point 402 with a pixel value of 1 in the original image, after the convolution kernel is used to convolve the pixel point 402, the pixel value of the new pixel point 403 obtained is 8. The specific process is not repeated here.

[0093] exist Figure 3 In order to simplify the calculation process, the convolution layer at the last layer in the neural network is equivalent to a filter. For the sake of distinction, the filter equivalent to the convolution layer at the last layer in the neural network is called the first filter.

[0094] It is understandable that after the image is processed by the neural network, the first image output by the convolutional layer located at the last layer of the neural network (i.e., equivalent to the first filter) is an unsharpened image, while the second image output by the second filter is a sharpened image. Based on this, if the first image output by the convolutional layer located at the last layer of the neural network (i.e., equivalent to the first filter) is superimposed with the second image output by the second filter, and the superposition ratio of the second image and the second image during the superposition process is controlled, the sharpness of the superimposed image can be adjusted.

[0095] Since the second image output by the second filter is an image obtained by sequentially processing the original image through the first filter and the second filter, the second image output by the second filter is obtained by processing a cascade filter obtained by cascading the first filter and the second filter.

[0096] In order to simplify the calculation, the present application can design a target filter, the structure of which is as follows: Figure 3 As shown in the figure. Based on this, the output of the target filter is equivalent to the superposition of the first image output by the first filter and the second image output by the cascade filter. Moreover, the target filter can control the corresponding superposition ratio of the first image and the second image to obtain images with different degrees of sharpening. The constructed target filter is actually a model that can achieve different degrees of sharpening of the image.

[0097] In which, in order to make the length of the first filter different from the length of the second filter in the cascade, the filter coefficients of the first filter and the shorter filter in the cascade filter can be padded with zeros at the end to make the lengths of the filter coefficients of the first filter and the cascade filter the same, so as to facilitate the subsequent addition operation of the filter coefficients of the first filter and the filter coefficients of the cascade filter.

[0098] On this basis, it is assumed that the input image is represented as X, the filter coefficient of the first filter is represented as H1, and the filter coefficient of the second filter is represented as H2.

[0099] Then, by Figure 3 It can be seen that the first image Y1 output by the first filter can be expressed as the following formula 1:

[0100]

[0101] in, Represents a tensor product.

[0102] The second image Y2 obtained by processing the image through the cascade filter of the first filter and the second filter can be expressed as the following formula 2:

[0103]

[0104] Since the first image Y1 is an unsharpened image, and the second image Y2 is a sharpened image, as mentioned above, by superimposing these two images to different degrees, we can obtain images with different degrees of sharpness. Based on this, assuming that the superposition ratio corresponding to the second image output by the cascade filter is a, and the superposition ratio corresponding to the first image output by the first filter equivalent to the convolution layer is 1-a, then the final superimposed image Y can be expressed as the following formula 3:

[0105] Y=(1-a)*Y1+a*Y2 (Formula 3);

[0106] Then, by substituting Formula 1 and Formula 2 into Formula 3, we can deduce as follows:

[0107]

[0108] Among them, according to the derivation It can be seen that when the value of a is different, the sharpness of the image result Y obtained after the image X is processed will also be different. It is the equivalent weight coefficient of the target filter, which is equivalent to the weight coefficient used to sharpen the image X, and the weight coefficient can change with the value of a. Therefore, It is the conversion function used to determine the weight coefficient. It can be seen that the conversion function: The weight coefficients for performing different degrees of sharpening processing on the image can be determined according to different values ​​of a.

[0109] Among them, the value of a can affect the degree of image sharpness. a can be used as the sharpening value in this application, and 1-a represents the degree of image sharpness.

[0110] In particular, in order to enable the sharpening value subsequently input by the user to directly represent the degree of sharpening, the present application may also set b=1-a, and since the first filter above is the first equivalent filter equivalent to constructing the transfer function, and the second filter is the second equivalent filter, therefore, in the present application, the transfer function can be expressed as the following formula 4:

[0111]

[0112] Where b is the sharpening value, H1 is the first weight coefficient, i.e., the first filter coefficient of the first equivalent filter, and H2 is the second weight coefficient, i.e., the second filter coefficient of the second equivalent filter. As can be seen from the above conversion function, after obtaining the target sharpening value, this target sharpening value is used as the sharpening value b in the conversion function shown in Formula 1, and the target weight coefficient can be determined using this conversion function.

[0113] Furthermore, from the above conversion function and its derivation process, it can be seen that if the weight coefficient of the convolution function of the last layer of the neural network is replaced with the weight coefficient determined based on the conversion function, then the convolution function of the last layer of the neural network is actually equivalent to the target filter mentioned above. Therefore, processing the image through the convolution function of the last layer of the neural network is equivalent to processing the image using the target filter. On this basis, after obtaining the sharpening value b, the target weight coefficient corresponding to the degree of sharpening represented by the sharpening value can be obtained based on the conversion function, and after replacing the weight coefficient of the convolution function of the last layer of the neural network with the target weight coefficient, then based on the last layer of the convolution function, the image is sharpened and an image with the corresponding degree of sharpening can be obtained.

[0114] Of course, the above is explained using a neural network as an example. If the image processing model is replaced with other network models (such as the artificial intelligence model mentioned above), as long as the weight coefficient of the last layer of convolution operation of the image processing model is the target weight coefficient determined based on the conversion function, the image can be sharpened accordingly based on the sharpening requirements using the image processing model.

[0115] It can be understood that after obtaining the sharpening value, the present application only needs to calculate the target weight coefficient corresponding to the last layer of convolution function in the image processing model based on the conversion function, so that the degree of sharpening of the image by the image processing model can match the degree of sharpening represented by the sharpening value without the need for other complex data processing, with less calculation amount and low calculation time, so that the image processing model can meet different image sharpening processing requirements with less data calculation amount.

[0116] In one possible implementation, in order to improve the efficiency of determining the target weight coefficient, in this application, a conversion function between the above-mentioned sharpening value and the weight coefficient can be configured in the conversion model. On this basis, this application can call and run the conversion model through the target processor, and based on the target sharpening value, use the conversion function in the conversion model to determine the target weight corresponding to the last layer of convolution operation in the image processing model.

[0117] The target processor is a processor that runs the image processing model. Based on this, when the target processor runs the image processing model, it determines the target weight coefficient corresponding to the last layer of convolution operation of the image processing model by calling and running the conversion model, and can directly set the calculated target weight coefficient as the weight coefficient corresponding to the last layer of convolution operation of the image processing model, thereby avoiding the situation where the target weight coefficient is calculated by other processors and the target weight coefficient is stored in an external memory outside the target processor (such as a double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDR), etc.). Naturally, there is no need to obtain the calculated target weight coefficient from the external memory, thereby improving the efficiency of configuring the target weight coefficient for the image processing model, and thereby improving the efficiency of processing the first image based on the image processing model.

[0118] The target processor for running the image processing model can be of many types, and this application does not impose any restrictions on this. For example, the target processor can be a neural network processor (NPU), a central processing unit (CPU), or a graphics processing unit (GPU), etc., without any specific restrictions.

[0119] In the above embodiments of the present application, in order to more efficiently set the weight coefficient corresponding to the last layer of convolution operation in the image processing model as the target weight coefficient, when calling and running the image processing model through the target processor, the present application can also store the target weight coefficient in a designated cache area in the target processor, and set the storage address of the weight coefficient corresponding to the last layer of convolution operation in the image processing model to the address of the target weight coefficient in the designated cache area.

[0120] It can be understood that, compared with storing different models for achieving different degrees of sharpening, the amount of data of the target weight coefficient is relatively small, such as approximately 1.1KB. The storage space required to store only the target weight coefficient is relatively small, so that the target weight coefficient can be cached in the cache space in the target processor.

[0121] Moreover, after the target weight coefficient is stored in the designated cache area in the target processor used to run the image processing model, the storage address of the weight coefficient corresponding to the last layer of convolution operation in the image processing model is set to the address of the target weight coefficient in the designated cache area. Then, when the target processor is running the image processing model, the target weight coefficient can be directly obtained from its designated cache area without having to obtain the target weight coefficient from the cache memory outside the target processor, thereby improving the efficiency of loading the target weight coefficient for the image processing model.

[0122] In order to facilitate understanding of the implementation of the image processing method of the present application, a specific implementation method is used as an example for explanation below.

[0123] like Figure 5 , shows another flow chart of the image processing method provided by the present application. The method of this embodiment may include:

[0124] S501: Obtain a first image to be processed.

[0125] S502: Obtain a target sharpening value.

[0126] The target sharpening value is used to represent the target sharpening degree of the image sharpening process.

[0127] S503: Load and run the image processing model through the target processor.

[0128] S504: Call and run the conversion model through the target processor, and determine the target weight coefficient corresponding to the last layer of convolution operation in the image processing model based on the target sharpening value using the conversion function configured in the conversion model.

[0129] The conversion function may be referred to in the related introduction of any of the previous embodiments, and will not be described again here.

[0130] It is understandable that the order of step S504 and step S503 can be interchanged, or steps S503 and S504 can be performed simultaneously, without specific limitation.

[0131] S505, stores the target weight coefficient in the designated cache area in the target processor, and sets the target storage address of the weight coefficient corresponding to the last layer of convolution operation in the image processing model to the address of the target weight coefficient in the designated cache area, so that the target processor loads the target weight coefficient as the weight coefficient corresponding to the last layer of convolution operation in the image processing model.

[0132] For example, updating the target storage address of the weight coefficient corresponding to the last layer of convolution operation in the image processing model to the address of the target weight coefficient in the specified cache area can be updating the starting address and length of each parameter in the last layer of convolution operation in the image processing model to the starting address and length corresponding to the corresponding parameter in the specified storage area.

[0133] For ease of understanding, combined Figure 6 For the above steps S504 to S505, please refer to Figure 5 , which shows a schematic diagram of the implementation principle framework of the present application for configuring the target weight coefficient for the convolution layer located at the last layer in the image processing model.

[0134] Depend on Figure 5 It can be seen that when the target processor loads the image processing model, the target storage address corresponding to the weight coefficient in the convolution layer of the last layer of the image processing model can be determined. On this basis, the target processor will calculate the target weight coefficient based on the obtained target sharpening value b using the conversion function. Among them, the target weight coefficient calculated by the conversion function is equivalent to the weight coefficient corresponding to the target filter mentioned above, that is, the conversion function can be expressed as: After the target processor stores the target weight coefficient in the designated cache area of ​​the target processor, the target storage address of the last convolution layer in the image processing model is updated to the address corresponding to the target weight coefficient, so that the target processor can run the last convolution layer in the image processing model based on the target weight coefficient and complete the corresponding image sharpening processing.

[0135] S506: Process the first image based on the image processing model to obtain a generated second image.

[0136] The sharpening degree of the second image is different from the sharpening degree of the first image.

[0137] Corresponding to the image processing method provided by this application, this application also provides an image processing device. Figure 7 , shows a schematic diagram of the composition structure of the image processing device provided by the present application. The device of this embodiment may include:

[0138] An image obtaining unit 701 is configured to obtain a first image to be processed;

[0139] A sharpening value obtaining unit 702 is used to obtain a target sharpening value, where the target sharpening value is used to represent a target sharpening degree for performing sharpening processing on an image;

[0140] A coefficient adjustment unit 703, configured to adjust the weight coefficients in the image processing model according to the target sharpening value;

[0141] The image processing unit 704 is configured to process the first image based on the image processing model to obtain a generated second image, where the sharpening degree of the second image is different from that of the first image.

[0142] In a possible implementation, the system adjustment unit includes:

[0143] A coefficient determination subunit, configured to determine a target weight coefficient corresponding to the last convolution operation in the image processing model according to the target sharpening value;

[0144] The coefficient setting subunit is used to set the weight coefficient corresponding to the last layer of convolution operation in the image processing model to the target weight coefficient.

[0145] In yet another possible implementation, the coefficient determination subunit includes:

[0146] The coefficient conversion subunit is used to determine the target weight coefficient corresponding to the last layer of convolution operation in the image processing model based on the target sharpening value and the conversion function between the configured sharpening value and the weight coefficient.

[0147] In one possible implementation, the processing effect of the target weight coefficient determined based on the conversion function on the image data is equivalent to the sharpening effect produced by processing the image data through a first equivalent filter and a cascade filter respectively and superimposing the image processing results, and the cascade filter is a cascade of the first equivalent filter and the second equivalent filter.

[0148] In yet another possible implementation, the coefficient conversion subunit includes:

[0149] a model running subunit, configured to call and run a conversion model through a target processor, wherein the conversion model is configured with a conversion function between a sharpening value and a weight coefficient, and the target processor is a processor running the image processing model;

[0150] The model conversion subunit is used to determine the target weight coefficient corresponding to the last layer of convolution operation in the image processing model based on the target sharpening value using the conversion function in the conversion model.

[0151] In yet another possible implementation, the image processing apparatus further includes:

[0152] A model calling unit, used for calling and running the image processing model through a target processor;

[0153] The coefficient determines the subunits, including:

[0154] The coefficient changing subunit is used to store the target weight coefficient in the designated cache area in the target processor, and set the target storage address of the weight coefficient corresponding to the last layer of convolution operation in the image processing model to the address of the target weight coefficient in the designated cache area.

[0155] An electronic device is also provided in an embodiment of the present application. Figure 8 , which shows a schematic diagram of the composition structure of the electronic device, the electronic device at least includes a processor 801 and a memory 802;

[0156] The processor 801 is configured to execute the image processing method described in any one of the above embodiments;

[0157] The memory 802 is used to store programs required by the processor to perform operations.

[0158] It is understandable that the electronic device may further include a display unit 803 and an input unit 804 .

[0159] Of course, the electronic device may also have Figure 8 There is no limitation to more or fewer components.

[0160] A computer program product is also provided in an embodiment of the present application, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any image processing method provided in the embodiment of the present application.

[0161] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any image processing method provided in the embodiment of the present application.

[0162] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0163] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits or special circuits, etc. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a number of instructions to enable a computer device (which can be a personal computer, training equipment, or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0164] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0165] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a training device or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website, a computer, a training device or a data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

Claims

1. An image processing method, comprising: obtaining a first image to be processed; Obtaining a target sharpening value, wherein the target sharpening value is used to represent a target sharpening degree of sharpening processing performed on the image; Adjusting the weight coefficients in the image processing model according to the target sharpening value; The first image is processed based on the image processing model to obtain a generated second image, where a sharpening degree of the second image is different from a sharpening degree of the first image.

2. The image processing method according to claim 1, wherein adjusting the weight coefficients in the image processing model according to the target sharpening value comprises: Determining a target weight coefficient corresponding to the last convolution operation in the image processing model according to the target sharpening value; The weight coefficient corresponding to the last layer of convolution operation in the image processing model is set as the target weight coefficient.

3. The image processing method according to claim 2, wherein determining the target weight coefficient corresponding to the last convolution operation in the image processing model according to the target sharpening value comprises: Based on the target sharpening value and in combination with a conversion function between the configured sharpening value and the weight coefficient, a target weight coefficient corresponding to the last layer of convolution operation in the image processing model is determined.

4. The image processing method according to claim 3, wherein determining the target weight coefficient corresponding to the last convolution operation in the image processing model based on the target sharpening value and in combination with a configured conversion function between the sharpening value and the weight coefficient comprises: Calling and running a conversion model through a target processor, wherein the conversion model is configured with a conversion function between a sharpening value and a weight coefficient, and the target processor is a processor running the image processing model; Based on the target sharpening value, a target weight coefficient corresponding to the last convolution operation in the image processing model is determined using a conversion function in the conversion model.

5. The image processing method according to claim 3 or 4, wherein the processing effect of the target weight coefficient determined based on the conversion function on the image data is equivalent to the sharpening effect produced by processing the image data through a first equivalent filter and a cascade filter respectively and superimposing the image processing results, and the cascade filter is a cascade of the first equivalent filter and the second equivalent filter.

6. The image processing method according to claim 5, wherein the conversion function is a function that uses the sharpening value as a weighting coefficient and performs a weighted operation on the first weighting coefficient and the second weighting coefficient; in, The first weight coefficient is equivalent to the first filter coefficient of the first equivalent filter; The second weight coefficient is equivalent to the second filter coefficient of the second equivalent filter.

7. The image processing method according to claim 6, wherein the conversion function is a function that uses a sharpening value as a first weighting coefficient of the cascade filter, uses a difference between 1 and the sharpening value as a second weighting coefficient of the first equivalent filter, and performs a weighted operation on the first filter coefficient of the first equivalent filter and the third filter coefficient of the cascade filter; in, The third filter coefficient of the cascade filter is a tensor product of the first filter coefficient of the first equivalent filter and the second filter coefficient of the second equivalent filter.

8. The image processing method according to claim 5, wherein the second equivalent filter is a filter for performing image sharpening processing based on a gradient sharpening algorithm; The first equivalent filter is equivalent to the last convolution layer in the image processing model.

9. The image processing method according to claim 2, further comprising: Call and run the image processing model through the target processor; Setting the weight coefficient corresponding to the last convolution operation in the image processing model as the target weight coefficient includes: The target weight coefficient is stored in a designated cache area in the target processor, and the target storage address of the weight coefficient corresponding to the last layer of convolution operation in the image processing model is set to the address of the target weight coefficient in the designated cache area.

10. An image processing device, comprising: an image obtaining unit, configured to obtain a first image to be processed; A sharpening value obtaining unit, configured to obtain a target sharpening value, wherein the target sharpening value is used to represent a target sharpening degree for performing sharpening processing on an image; A coefficient adjustment unit, configured to adjust a weight coefficient in an image processing model according to the target sharpening value; An image processing unit is configured to process the first image based on the image processing model to obtain a generated second image, wherein a sharpening degree of the second image is different from a sharpening degree of the first image.

Images