An image processing method, device, storage medium and terminal device

Abstract

The application discloses an image processing method and device, a storage medium and a terminal device. The method comprises the following steps: acquiring an image block corresponding to each pixel point in a to-be-processed image; determining filter weight coefficients of each filter in a plurality of preset filters based on the image block; and determining an output pixel point corresponding to the pixel point based on each filter, the filter weight coefficients of each filter and the image block, so as to obtain an output image corresponding to the to-be-processed image. When the to-be-processed image is processed, the image block of each pixel point is taken as a processing item, the target filter of the image block is determined based on each filter and each filter weight coefficient, and the output pixel point corresponding to the pixel point is determined based on the target filter. In this way, the target filter of the pixel point is determined based on a plurality of filters, so that the filtering effect corresponding to the pixel point can be ensured, the calculation amount of the filtering process can be reduced by taking the image block as the processing item, and the memory requirement in the filtering process is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an image processing method and device, a storage medium and a terminal device. BACKGROUND

[0002] With the continuous development of deep learning technology, deep learning methods are widely used in the field of image enhancement. The image enhancement effect achieved by deep learning methods far exceeds that achieved by traditional methods (for example, linear interpolation and bilinear interpolation). However, the current deep learning-based image processing model for image enhancement requires a large amount of computation in the processing process and thus occupies a high memory, making it difficult to apply to mobile terminals. SUMMARY

[0003] The technical problem to be solved by the present application is to provide an image processing method, device, storage medium and terminal device to overcome the shortcomings of the prior art.

[0004] To solve the above technical problems, the first aspect of the present application provides an image processing method, which comprises:

[0005] For each pixel point in the image to be processed, an image block corresponding to the pixel point is obtained; based on the image block, filter weight coefficients corresponding to each filter in a plurality of preset filters are determined; based on each filter in the plurality of filters, the filter weight coefficients corresponding to each filter, and the image block, an output pixel point corresponding to the pixel point is determined.

[0006] According to the output pixel points respectively corresponding to each pixel point in the image to be processed, an output image corresponding to the image to be processed is determined.

[0007] The image processing method, wherein the filter weight coefficients corresponding to each filter in the plurality of preset filters are determined based on the image block are:

[0008] The image block is input into a trained image processing model, and the filter weight coefficients corresponding to each filter in the plurality of preset filters are determined by the image processing model.

[0009] The image processing method, wherein the output pixel point corresponding to the pixel point is determined based on each filter in the plurality of filters, the filter weight coefficients corresponding to each filter, and the image block specifically comprises:

[0010] Each filter is used to perform convolution operation on the image block respectively to obtain a candidate pixel point corresponding to each filter;

[0011] The candidate pixel point corresponding to each filter is weighted based on the filter weight coefficient corresponding to each filter to obtain an output pixel point corresponding to the pixel point.

[0012] The image processing method, wherein at least a first filter and a second filter exist in the plurality of filters, the filter type of the first filter is different from the filter type of the second filter.

[0013] The image processing method, wherein the filter kernel size corresponding to each filter in the plurality of filters is the same.

[0014] The image processing method, wherein the image size of the image block is equal to the filter kernel size of one filter in the plurality of filters.

[0015] The image processing method, wherein the obtaining of the image block corresponding to the pixel point comprises:

[0016] For each pixel point in the image to be processed, the filter kernel size corresponding to the plurality of filters is obtained.

[0017] An image region is determined based on the filter kernel size corresponding to the plurality of filters and the image to be processed, and the image region is taken as the image block corresponding to the pixel point, wherein the pixel point is the center point of the image region.

[0018] The image processing method, wherein the pixel points in the image to be processed are obtained one by one in the row direction or the column direction.

[0019] The second aspect of the embodiment of the present application provides an image processing device, and the image processing device specifically comprises:

[0020] An obtaining module is configured to obtain an image block corresponding to each pixel point in an image to be processed; determine filter weight coefficients corresponding to each filter in a plurality of preset filters based on the image block; and determine an output pixel point corresponding to the pixel point based on each filter in the plurality of filters, the filter weight coefficients corresponding to each filter, and the image block.

[0021] A determining module is configured to determine an output image corresponding to the image to be processed based on the output pixel points corresponding to each pixel point in the image to be processed respectively.

[0022] The third aspect of the embodiment of the present application provides a computer readable storage medium, and the computer readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the image processing method according to any one of the above.

[0023] The fourth aspect of the embodiments of the present application provides a terminal device, which comprises a processor, a memory and a communication bus; the memory stores a computer readable program which can be executed by the processor;

[0024] The communication bus realizes the connection communication between the processor and the memory;

[0025] The processor realizes the steps in the image processing method according to any one of the above when executing the computer readable program.

[0026] Advantages: compared with the prior art, the present application provides an image processing method, device, storage medium and terminal device, the method comprises: for each pixel point in the image to be processed, obtaining the image block corresponding to the pixel point; based on the image block, determining the filter weight coefficient corresponding to each filter in the predetermined filter; based on each filter in the filter, the filter weight coefficient corresponding to each filter and the image block, determining the output pixel point corresponding to the pixel point. When the image to be processed is processed, the image block corresponding to each pixel point is taken as a processing item, and the target filter corresponding to the image block is determined by each filter and the filter weight coefficient corresponding to each filter, and the output pixel point corresponding to the pixel point is determined by the target filter. In this way, the target filter corresponding to the pixel point is determined by the filter, which can ensure the filtering effect of the pixel point, and the calculation amount of the filtering process can be reduced by taking the image block as the processing item, thereby reducing the memory requirement in the filtering process. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the premise of not departing from the essence of the present application.

[0028] Figure 1 The flowchart of the image processing method provided by the present application.

[0029] Figure 2 The principle flowchart of the existing image super-resolution model.

[0030] Figure 3 The schematic diagram of the selection method of the pixel in the image processing method provided by the present application.

[0031] Figure 4 The principle flowchart of the image processing model in the image processing method provided by the present application.

[0032] Figure 5A structural schematic diagram of an image processing device provided in the present application is shown in FIG. 1.

[0033] Figure 6 A structural schematic diagram of a terminal device provided in the present application is shown in FIG. 2. DETAILED DESCRIPTION

[0034] The present application provides an image processing method, device, storage medium and terminal device. To make the purpose, technical solutions and effects of the present application clearer and more explicit, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0035] Those skilled in the art can understand that the singular forms "a", "an" and "the" used herein include plural forms unless specifically stated otherwise. It should be further understood that the use of the phrase "comprises" in the specification of the present application means that the features, integers, steps, operations, elements and / or components exist, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we say that an element is "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or there can be intermediate elements. In addition, "connected" or "coupled" used herein can include wireless connection or wireless coupling. The phrase "and / or" used herein includes all or any single unit and all combinations of the associated listed items.

[0036] Those skilled in the art can understand that unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as that generally understood by those skilled in the art to which the present application belongs. It should also be understood that terms such as those defined in a general dictionary should be understood to have meanings consistent with those in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as such.

[0037] The inventors have found that with the continuous development of deep learning technology, deep learning methods are widely used in the field of image enhancement technology. The image enhancement effect achieved by deep learning methods far exceeds that achieved by traditional methods (e.g., linear interpolation and bilinear interpolation). However, the image processing model for image enhancement determined based on deep learning currently requires a large amount of computation in the processing process and thus needs to occupy a very high memory, making it difficult to apply to mobile terminals. For example, as shown in FIG. 1, the image processing model for image enhancement determined based on deep learning needs to occupy a very high memory, which makes it difficult to apply to mobile terminals. Figure 2As shown, for image processing models in super-resolution tasks, the input is typically an H*W*3 image. After processing through multiple CNN layers, the output resolution becomes (H*C)*(W*C)*3. Super-resolution image processing models generally require 20 or more network layers. The computational cost of the neural network is O(W*H*K*K*CIN*COUT*N), where K*K is the kernel size, CIN is the number of input channels, COUT is the number of output channels, and N is the number of network layers. Therefore, for a 4K resolution input image, the computational power required by the image processing model is far greater than 15 TOPS. Furthermore, as displays evolve to 8K / 120Hz, the computational power required by the image processing model will increase significantly. This makes it impossible for image processing models to be used on terminal devices for online processing, and it also requires a large amount of memory during the processing process.

[0038] To address the aforementioned issues, in this embodiment, for each pixel in the image to be processed, an image block corresponding to that pixel is obtained; based on the image block, the filter weight coefficients corresponding to each of a preset set of filters are determined; and based on each of the filters, their respective filter weight coefficients, and the image block, the output pixel corresponding to that pixel is determined. When processing the image to be processed, this application treats the image block corresponding to each pixel as a processing item, determines the target filter corresponding to that image block using each filter and its respective filter weight coefficients, and then determines the output pixel corresponding to that pixel using the target filter. This method of determining the target filter for each pixel using multiple filters ensures the filtering effect for that pixel. Furthermore, treating the image block as a processing item reduces the computational load of the filtering process, thereby reducing memory requirements.

[0039] The application content will be further explained below with reference to the accompanying drawings and the description of the embodiments.

[0040] This embodiment provides an image processing method, which can be executed by an image processing device. The device can be implemented in software and applied to terminal devices such as smartphones, tablets, or personal digital assistants. See also... Figure 1 The image processing method provided in this embodiment specifically includes:

[0041] S10. For each pixel in the image to be processed, obtain the image block corresponding to the pixel; based on the image block, determine the filter weight coefficients corresponding to each of the preset filters; based on each of the preset filters, the filter weight coefficients corresponding to each filter, and the image block, determine the output pixel corresponding to the pixel.

[0042] Specifically, the images to be processed can be obtained through an imaging system (e.g., a camera, camcorder, under-display camera, etc.), or they can be images captured by other external devices (e.g., digital cameras) and stored on the terminal device, or they can be images sent to the terminal device via the cloud. In this embodiment, the images to be processed are images captured by an imaging system, which can be configured on the terminal device itself or on other devices. For example, the image to be processed could be a landscape image captured by a mobile phone equipped with an imaging system, or it could be an image of a kitten captured by a digital camera and sent to the terminal device.

[0043] The image block includes the pixel, and the image block is selected from the image to be processed. It can be understood that an image block is an image region in the image to be processed, and this image region contains the pixel. In one implementation of this embodiment, the image size of the image blocks corresponding to each pixel in the image to be processed is the same, for example, 3*3, 5*5, 7*7, etc. For example, if the image to be processed includes pixel a and pixel b, pixel a corresponds to image region A in the image to be processed, and pixel b corresponds to image region B in the image to be processed, then image region A contains pixel a, image region B contains pixel b, and the image size of image region A is equal to the image size of image region B.

[0044] In one implementation of this embodiment, obtaining the image block corresponding to each pixel in the image to be processed specifically includes:

[0045] The pixels in the image to be processed are obtained one by one according to the row direction or column direction;

[0046] For each obtained pixel, the corresponding image block is obtained, wherein the image block includes the pixel.

[0047] Specifically, the pixels in the image to be processed are arranged in an array, where the row direction refers to the row direction of the array formed by the pixels, and the column direction refers to the column direction of the array formed by the pixels. For example, if the row direction of the array formed by the pixels is the width direction of the image to be processed, then the row direction refers to the width direction of the image size to be processed, and the column direction refers to the height direction of the image size to be processed. Furthermore, as... Figure 3As shown, when acquiring pixels in the image to be processed row by row, the first pixel in the first column of the first row is selected, then the pixel in the second column of the first row is selected, and so on, until the pixel in the last column of the first row is selected; the same method is used to select pixels in the second row, and so on, until the last row is selected. Similarly, when acquiring pixels in the image to be processed column by column, the selection process is the same as when acquiring pixels in the image to be processed row by row, except that the selection starts from the pixel in the first row of the first column.

[0048] For example: Suppose the array of pixels in the image to be processed includes a first pixel a11, a second pixel a12, a third pixel a21, and a second pixel a22, where i represents the row number and j represents the column number, and the values ​​of i and j are in the range {1,2}. Then, when the pixels in the image to be processed are obtained one by one in the row direction, the order of obtaining the first pixel a11, the second pixel a12, the third pixel a21, and the second pixel a22 is: first pixel a11, second pixel a12, third pixel a21, and second pixel a22. When the pixels in the image to be processed are obtained one by one in the column direction, the order of obtaining the first pixel a11, the second pixel a12, the third pixel a21, and the second pixel a22 is: first pixel a11, the third pixel a21, the second pixel a12, and second pixel a22.

[0049] In one implementation of this embodiment, the plurality of filters are pre-configured for filtering image patches. Each filter in the plurality of filters has the same kernel size, for example, the kernel size of each filter is 3*3, 5*5, or 7*7. At least a first filter and a second filter exist among the plurality of filters. The filter type of the first filter is different from that of the second filter. The filter type reflects the function of the filter; for example, the filter type may include Gaussian filters, edge filters, and low-pass filters, etc. For example, the plurality of filters includes 50 filters, of which 20 are Gaussian filters, 20 are edge filters, and 10 are low-pass filters. This embodiment, by pre-setting a plurality of filters containing several filter types, allows for different processing of image patches (e.g., image denoising, edge extraction, etc.), and the filtering weight coefficients of each filter in the plurality of filters differ when performing different processing on the image patch.

[0050] In one implementation of this embodiment, in order to ensure that all filters can filter the image block corresponding to a pixel, the image block corresponding to each pixel has the same image size, and is equal to the filter kernel size. Based on this, obtaining the image block corresponding to the pixel specifically includes:

[0051] For each pixel in the image to be processed, obtain the filter kernel size corresponding to several filters;

[0052] An image region is determined based on the filter kernel size and the image to be processed, and this image region is used as the image block corresponding to the pixel, wherein the pixel is the center point of the image region.

[0053] Specifically, the filter kernel size can be the filter kernel size of any one of the filters, since the filter kernel sizes of all filters are equal. Therefore, when obtaining the filter kernel sizes corresponding to the filters, one filter can be selected from the filters, and its filter kernel size can be used as the corresponding filter kernel size. Furthermore, the region size of the image area is equal to the filter kernel size, and the pixel is the center point of the image area. It can be understood that for each pixel, with the pixel as the center and the filter kernel size as the neighborhood size, the neighborhood region corresponding to the pixel is selected in the image to be processed, and the selected neighborhood region is used as the determined image region. It is worth noting that before selecting an image block for each pixel in the image to be processed, pixel padding can be performed on the image to be processed so that each pixel in the image to be processed can have a neighborhood region selected. Pixel padding refers to filling the periphery of the image to be processed with several pixels with a pixel value of 0.

[0054] The filter weight coefficient reflects the importance of the filter; a larger filter weight coefficient indicates a higher level of importance, and vice versa. In one implementation of this embodiment, the filter weight coefficient ranges from 0 to 1, and the importance of the filter increases as the filter weight coefficient increases from 0 to 1. For example, a filter with a filter weight coefficient of 0.7 is more important than a filter with a filter weight coefficient of 0.1.

[0055] In one implementation of this embodiment, the filter weight coefficients are determined by a trained image processing model. Accordingly, based on the image patch, the filter weight coefficients corresponding to each of the preset plurality of filters are determined as follows:

[0056] The image patch is input into a trained image processing model, and the image processing model determines the filter weight coefficients corresponding to each filter among a preset number of filters.

[0057] Specifically, the image processing model is trained to determine the filter weight coefficients corresponding to each filter. The input to the image processing model is an image patch, and the output is the filter weight coefficients corresponding to each filter. It can be understood that when an image patch is input into the image processing model, the model learns the image patch and outputs a set of filter weight coefficients corresponding to that image patch. These filter weight coefficients include several filter weight coefficients, and each set corresponds one-to-one with a specific filter, thus obtaining the filter weight coefficients corresponding to each filter. Furthermore, for pixels a and b in the image to be processed, the set of filter weight coefficients corresponding to pixel a can be different from the set of filter weight coefficients corresponding to pixel b. That is, the set of filter weight coefficients corresponding to pixel a contains one filter weight coefficient a, and the set of filter weight coefficients corresponding to pixel b contains one filter weight coefficient b. Filter weight coefficients a and b are different, and the filter corresponding to filter weight coefficient a is the same as the filter corresponding to filter weight coefficient b.

[0058] In one implementation of this embodiment, such as Figure 4 As shown, the image processing model may include an extraction module, a fusion module, and a convolution module. The extraction module is used to obtain a set of filter weight coefficients corresponding to an image patch, which includes the filter weight coefficients corresponding to each filter. The fusion module is used to determine the target filter corresponding to the image patch based on the set of filter weight coefficients and several filters. The convolution module uses the target filter as the convolution kernel and performs convolution operations on the image patch to obtain the output pixel corresponding to that pixel. In this embodiment, as... Figure 4 As shown, the extraction module includes a feature extraction unit, a pooling unit, a first fully connected unit, an activation unit, a second fully connected unit, and a classification unit. These units are cascaded sequentially. The input to the feature extraction unit is an image patch, and the output of the classification module is the set of filter weight coefficients corresponding to that image patch. The set of filter weight coefficients includes the filter weight coefficients corresponding to each filter. The activation unit can be configured with a ReLU activation function, and the classification unit can be configured with a softmax function.

[0059] In one implementation of this embodiment, the training process of the image processing model may include:

[0060] The training images in the training image set are input into the preset network model, and the training filter weight coefficient set corresponding to the training images is determined by the preset network model.

[0061] Based on the training filter weight coefficient set and several preset filters, the predicted image corresponding to the training image is determined.

[0062] The image processing model is obtained by training a preset network model based on the predicted image and the corresponding real image.

[0063] Specifically, the training image set includes several training image groups, each of which includes a training image and a corresponding real image. The real image serves as the training standard image for that training image, used to evaluate whether the predicted image output by the preset network model meets the requirements. The model structure of the preset network model is the same as that of the image processing model; for details, please refer to the description of the image processing model structure, which will not be repeated here. The difference between the preset network model and the image processing model is that the model parameters of the preset network model are the initial model parameters, while the model parameters of the image processing model are the model parameters after training with the training image set.

[0064] The training image weight coefficient set includes several training image weight coefficients, each corresponding one-to-one with several filters. Each training filter weight coefficient is determined by a pre-defined network model based on the training images and reflects the importance of its corresponding filter in processing the training images. The target filter is obtained by weighting the filter weight coefficients corresponding to each filter. For example, assuming the filters are denoted as WK1, WK2, ..., WKn, where n is the number of filters; and the filter weight coefficient corresponding to WK1 is denoted as a1, the filter weight coefficient corresponding to WK2 is denoted as a2, ..., the filter weight coefficient corresponding to WKn is denoted as an, then the predicted image can be represented as:

[0065]

[0066] Where, x j y represents the pixels in the training image. j denoted as the pixel in the predicted image, n is the number of filters, WKi represents the i-th filter, and ai represents the filter weight coefficient of the i-th filter.

[0067] Furthermore, after obtaining the predicted image, the loss value corresponding to the training image is determined based on the real image corresponding to the predicted image and the training image, and the preset network model is trained based on the loss value to obtain the image processing model.

[0068] In one implementation of this embodiment, determining the output pixel corresponding to the pixel based on each filter, the filter weight coefficients corresponding to each filter, and the image block specifically includes:

[0069] Each filter is used to perform convolution operation on the image block to obtain the candidate pixel points corresponding to each filter;

[0070] Based on the filtering weight coefficients of each filter, the candidate pixels corresponding to each filter are weighted to obtain the output pixel corresponding to that pixel.

[0071] Specifically, performing convolution operations on the image block using each filter means performing a convolution operation on the image block based on each filter. The convolution operation refers to multiplying the filter with the image block to obtain the output pixel corresponding to that pixel. The convolution operation can be expressed as:

[0072] y i =x*WKi

[0073] Among them, y i Let x represent the candidate pixel output by the i-th filter, x represent the image patch, and WKi represent the i-th filter.

[0074] Furthermore, after obtaining the candidate pixels corresponding to each filter, the candidate pixels are weighted according to the filter weight coefficients corresponding to each filter to obtain the output pixels. The formula for calculating the output pixels can be:

[0075]

[0076] Where x represents an image patch, y represents an output pixel, n is the number of filters, WKi represents the i-th filter, and ai represents the filter weight coefficient of the i-th filter.

[0077] S30. Determine the output image corresponding to the image to be processed based on the output pixel points corresponding to each pixel point in the image to be processed.

[0078] Specifically, the output image is the image after the image to be processed has been processed by the image processing method provided in this embodiment. The output image includes the output pixels corresponding to each pixel of the image to be processed. These output pixels are obtained by processing the image block based on each filter and its corresponding filter weight coefficients.

[0079] In summary, this embodiment provides an image processing method. The method includes acquiring image blocks corresponding to each pixel in an image to be processed; determining the filter weight coefficients of each filter among a preset set of filters based on the image blocks; and determining the output pixel corresponding to the pixel based on each filter, the filter weight coefficients of each filter, and the image blocks, so as to obtain the output image corresponding to the image to be processed. When processing the image to be processed, this application treats the image block of each pixel as a processing item, determines the target filter for the image block using each filter and its filter weight coefficients, and determines the output pixel corresponding to the pixel using the target filter. This method of determining the target filter for the pixel using several filters ensures the filtering effect for that pixel. Furthermore, treating the image block as a processing item reduces the computational load and memory requirements of the filtering process.

[0080] Based on the above image processing method, this embodiment provides an image processing apparatus, such as... Figure 5 As shown, the image processing device specifically includes:

[0081] The acquisition module 100 is used to acquire the image block corresponding to each pixel in the image to be processed; determine the filter weight coefficients corresponding to each filter in a preset plurality of filters based on the image block; and determine the output pixel corresponding to the pixel based on each filter in the plurality of filters, the filter weight coefficients corresponding to each filter, and the image block.

[0082] The determining module 200 is used to determine the output image corresponding to the image to be processed based on the output pixel points corresponding to each pixel point in the image to be processed.

[0083] Furthermore, it is worth noting that the working process of the image processing device provided in this embodiment is the same as that of the image processing method described above, and will not be repeated here. For details, please refer to the working process of the image processing method described above.

[0084] Based on the above image processing method, this embodiment provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps in the image processing method described in the above embodiment.

[0085] Based on the above image processing method, this application also provides a terminal device, such as... Figure 6As shown, it includes at least one processor 20; a display screen 21; and a memory 22, and may also include a communications interface 23 and a bus 24. The processor 20, display screen 21, memory 22, and communications interface 23 can communicate with each other via the bus 24. The display screen 21 is configured to display a preset user guide interface in the initial setup mode. The communications interface 23 can transmit information. The processor 20 can invoke logical instructions in the memory 22 to execute the methods described in the above embodiments.

[0086] Furthermore, the logical instructions in the aforementioned memory 22 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0087] The memory 22, as a computer-readable storage medium, can be configured to store software programs, computer-executable programs, such as program instructions or modules corresponding to the methods in the embodiments of this disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions, or modules stored in the memory 22, thereby implementing the methods in the above embodiments.

[0088] The memory 22 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 22 may include high-speed random access memory (RAM) and non-volatile memory. Examples include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, as well as transient storage media.

[0089] Furthermore, the specific process of loading and executing multiple instruction processors in the aforementioned storage medium and terminal device has been described in detail in the above method, and will not be repeated here.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. An image processing method, characterized by, The method comprises: For each pixel point in the image to be processed, an image block corresponding to the pixel point is obtained; for each pixel point in the image to be processed, the filter kernel size corresponding to a plurality of filters is obtained; based on the filter kernel size corresponding to the plurality of filters and the image to be processed, an image region is determined, and the image region is taken as the image block corresponding to the pixel point, wherein the pixel point is the center point of the image region; Based on the image block, the filter weight coefficients corresponding to each filter in the plurality of filters are determined; the image block is input into a trained image processing model, and the filter weight coefficients corresponding to each filter in the plurality of filters are determined by the image processing model; the image processing model comprises an extraction module, a fusion module and a convolution module; the extraction module comprises a feature extraction unit, a pooling unit, a first full connection unit, an activation unit, a second full connection unit and a classification unit; the feature extraction unit, the pooling unit, the first full connection unit, the activation unit, the second full connection unit and the classification unit are sequentially cascaded, the input of the feature extraction unit is the image block, and the output of the classification unit is the filter weight coefficient set corresponding to the image block; Based on the plurality of filters, the filter weight coefficients corresponding to each filter and the image block, the output pixel point corresponding to the pixel point is determined; Each filter is used to perform convolution operation on the image block to obtain the candidate pixel point corresponding to each filter; for each filter, the convolution operation of the filter on the image block is performed, that is, the filter and the image block are convolved to obtain the candidate pixel point corresponding to the pixel point; Based on the filter weight coefficients corresponding to each filter, the candidate pixel point corresponding to each filter is weighted to obtain the output pixel point corresponding to the pixel point; According to the output pixel point corresponding to each pixel point in the image to be processed, the output image corresponding to the image to be processed is determined.

2. The image processing method of claim 1, wherein, At least a first filter and a second filter exist in the plurality of filters, and the filter type of the first filter is different from the filter type of the second filter.

3. The image processing method of claim 1, wherein, The filter kernel size corresponding to each filter in the plurality of filters is the same.

4. The image processing method of claim 3, wherein, The image size of the image block is equal to the filter kernel size of a filter in the plurality of filters.

5. The image processing method of claim 1, wherein, The pixel points in the image to be processed are obtained one by one in the row direction or the column direction.

6. An image processing apparatus characterized by comprising: The image processing device for performing the image processing method of any one of claims 1-5 specifically comprises: The acquisition module is used to obtain the image block corresponding to each pixel point in the image to be processed; based on the image block, the filter weight coefficients corresponding to each filter in the plurality of filters are determined; based on the plurality of filters, the filter weight coefficients corresponding to each filter and the image block, the output pixel point corresponding to the pixel point is determined; The determination module is used to determine the output image corresponding to the image to be processed according to the output pixel point corresponding to each pixel point in the image to be processed.

7. A computer readable storage medium characterized in that, The computer readable storage medium stores one or more programs executable by one or more processors to implement the steps of the image processing method of any one of claims 1-5.

8. A terminal device, comprising: Comprise: a processor, a memory and a communication bus; the memory stores a computer readable program executable by the processor; The communication bus realizes the connection communication between the processor and the memory; The processor implements the steps of the image processing method of any one of claims 1-5 when executing the computer readable program.

Images