Image processing method and device, computer equipment and storage medium

Abstract

The invention provides an image processing method and device, computer equipment and a storage medium. A to-be-processed initial image is processed to determine target processing result information of the initial image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, apparatus, computer equipment and storage medium. Background Art

[0002] Image super-resolution refers to an image processing technology that restores a high-resolution image from a low-resolution image or image sequence. It has been widely used in users' daily lives.

[0003] At present, the more common image super-resolution technology is to process low-resolution images based on trained image super-resolution networks to obtain high-resolution images, but the performance of the high-resolution images finally obtained in related technologies is not ideal. Summary of the Invention

[0004] The present application provides an image processing method, apparatus, computer device and storage medium.

[0005] In a first aspect, the present application provides an image processing method, comprising:

[0006] Obtain the initial image to be processed;

[0007] Performing semantic recognition on the initial image to obtain first type information in the initial image;

[0008] The initial image is processed according to the first type information to obtain target image information corresponding to the initial image.

[0009] In a second aspect, the present application provides an image processing device, comprising:

[0010] An acquisition module, used for acquiring an initial image to be processed;

[0011] a recognition module, configured to perform semantic recognition on the initial image to obtain first type information in the initial image;

[0012] A processing module is used to process the initial image according to the first type of information to obtain target image information corresponding to the initial image.

[0013] In a third aspect, the present application further provides a computer device, comprising:

[0014] one or more processors;

[0015] Memory; and

[0016] One or more application programs, wherein the one or more application programs are stored in the memory and configured so that the processor executes any one of the above-mentioned image processing methods.

[0017] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is loaded by a processor to execute any of the above-mentioned image processing methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 A schematic diagram of the steps of an image processing method provided in an embodiment of the present application;

[0020] Figure 2 A schematic flow chart of steps for processing an initial image based on first type information provided in an embodiment of the present application;

[0021] Figure 3 A schematic flow chart of steps for processing a region mask image based on first type information provided in an embodiment of the present application;

[0022] Figure 4 A schematic diagram of a process flow for training a super-resolution network model provided in an embodiment of the present application;

[0023] Figure 5 A schematic flow chart of a step of extracting texture feature information to perform super-resolution processing on a region mask image provided by an embodiment of the present application;

[0024] Figure 6 A schematic diagram of another step flow for image processing provided in an embodiment of the present application;

[0025] Figure 7 A schematic diagram of an image processing effect provided in an embodiment of the present application;

[0026] Figure 8 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0027] Figure 9 A schematic structural diagram of a computer device provided in an embodiment of the application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0029] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the described features. In the description of this application, "plurality" means two or more, unless otherwise specifically specified.

[0030] In the description of this application, the word "for example" is used to mean "used as an example, illustration or illustration". Any embodiment described in this application as "for example" is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any person skilled in the art to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in this application.

[0031] To facilitate understanding of the image processing method provided in the embodiment of the present application, the application scenario of the image processing method provided in the embodiment of the present application is first described. Specifically, the image processing method provided in the embodiment of the present application is mainly used for super-resolution processing of images, that is, the process of restoring high-resolution images from low-resolution images. At present, there are many related technologies for realizing image super-resolution, such as image super-resolution based on interpolation, image super-resolution based on reconstruction, and image super-resolution based on deep learning. Among them, image super-resolution based on deep learning refers to the use of a large number of image samples to train an image super-resolution network to realize image super-resolution.

[0032] However, considering that in actual application, an image super-resolution network is used to perform indiscriminate computational processing on an entire image, it is difficult to meet the different requirements of different subjects in the image for super-resolution details, which will result in the performance of different subjects in the final image being less than ideal. In order to solve the above problems, the present application provides an image processing method, device, computer equipment and storage medium, which performs corresponding super-resolution processing on the image by identifying different subjects in the image, thereby ensuring the performance of different subjects in the final super-resolution image. Specifically, the image processing method is usually set in the image processing device in the form of a computer program, and the image processing device is usually set in the form of a processor (such as a graphics processor GPU or a central processing unit CPU) in a computer device (such as a local client or a remote server). The image processing device in the computer device executes the computer program corresponding to the image processing method, thereby executing the image processing method provided in the embodiment of the present application.

[0033] like Figure 1 As shown, Figure 1 A schematic flow chart of the steps of an image processing method provided in an embodiment of the present application, specifically including steps S110 to S130:

[0034] S110: Acquire an initial image to be processed.

[0035] In the embodiment of the present application, combined with the application scenarios provided above, it can be seen that the initial image to be processed usually refers to an image that needs to be super-resolution processed. For example, it can be an image taken by a user in daily life, and of course it can also be a scene image obtained during the filming process of a movie. The embodiment of the present application does not limit the source of the initial image. Any image that needs to be super-resolution processed can be regarded as the initial image to be processed in this application.

[0036] S120: Perform semantic recognition on the initial image to obtain first type information in the initial image.

[0037] In the embodiment of the present application, by performing semantic recognition on the image, different semantic information can be extracted from the initial image, thereby determining the first type of information in the image. Specifically, there are many ways to implement semantic recognition of the image to determine the first type of information. For example, a common method is to complete the semantic recognition processing of the initial image based on the pre-trained image semantic information decoupler Seg(·). In addition to obtaining a string of 1×n arrays C for representing different first types of information in the initial image, i (array c i The n values ​​in the image correspond to n different types of subjects, and the mask information m used to characterize the regional position of each subject in the initial image can also be obtained synchronously. iSpecifically, as a feasible implementation scheme, the mask information can also be used to improve the image super-resolution processing effect in the subsequent processing of the initial image. The specific implementation scheme can refer to the subsequent step S130 and its explanation.

[0038] Of course, as another feasible embodiment of the present application, in order to further improve the processing effect of the image super-resolution processing, the image can be subjected to preliminary filtering processing before the semantic recognition of the initial image to filter out the noise in the image. After the super-resolution processing of the image is completed, the obtained super-resolution image can also be further filtered to obtain a more refined super-resolution image. The specific implementation scheme can be referred to in the subsequent Figure 6 and its explanatory contents.

[0039] S130: Process the initial image according to the first type information to obtain target image information corresponding to the initial image.

[0040] In an embodiment of the present application, after obtaining the first type of information in the initial image as mentioned above, the first type of information is used to perform differentiated super-resolution processing on the initial image, so as to ensure the performance of different subjects in the target image information finally obtained. For example, as one of the most common feasible implementation schemes, based on different first type information, the image super-resolution network obtained by pre-training can be used to process the initial image to obtain the final target image information.

[0041] Of course, in actual application, when the initial image contains multiple pieces of first-type information, as a feasible embodiment of the present application, in the process of processing the initial image using the first-type information, additional mask information m representing the regional position of each subject in the initial image will be used. i Determine the region mask image corresponding to each subject, and then further perform super-resolution processing on the different region mask images to ensure the super-resolution performance of each subject in the final target image information. The specific implementation scheme can be found in the following Figure 2 and its explanatory contents.

[0042] like Figure 2 As shown, Figure 2 A schematic flow chart of a process for processing an initial image based on first type information provided in an embodiment of the present application, specifically comprising steps S210 to S230:

[0043] S210: Obtain a region mask image corresponding to each first type of information in the initial image.

[0044] In the embodiment of the present application, the mask information m representing the regional position of each subject in the initial image is obtained. i Afterwards, by passing the mask information m i By multiplying the initial image I, we can get the region mask image m corresponding to each first type of information. i ·I.

[0045] S220 , performing super-resolution processing on each region mask image corresponding to each of the first type information according to the first type information, to obtain a region super-resolution image corresponding to the region mask image.

[0046] In the embodiment of the present application, the region mask image m corresponding to each first type of information is obtained i After that, the different first type information is used to perform super-resolution processing on the region mask images corresponding to each first type information, so that the region super-resolution image corresponding to each subject can be obtained, that is, the region super-resolution image where N SR Represents the array c i and region mask image m i · I processing, for example, the more common N SR It can represent the super-resolution network mentioned above. Of course, considering that in actual application, it is necessary to perform corresponding super-resolution processing on the regional mask image based on the first type of information, therefore, as a feasible embodiment of the present application, the super-resolution network provided in the embodiment of the present application is a multimodal semantic super-resolution network model, which includes a deep neural network model for realizing image super-resolution and a linear controller based on semantics to realize differential processing. At this time, the specific implementation scheme for processing the corresponding regional mask image based on the first type of information can be referred to below. Figure 3 and its explanatory contents.

[0047] like Figure 3 As shown, Figure 3 A schematic flow chart of the steps for processing a region mask image based on the first type of information provided in an embodiment of the present application is described in detail as follows.

[0048] In order to improve the effect of super-resolution processing of images by the super-resolution network model, in the embodiment of the present application, a linear controller related to the first type of information is additionally added to the relevant super-resolution network model to realize differential processing of the regional mask images corresponding to different first type of information. Specifically, the process includes steps S310 to S320:

[0049] S310: Input the region mask image corresponding to the first type of information into the deep neural network layer in the super-resolution network model for processing to obtain region mask feature information corresponding to each of the first type of information.

[0050] In the embodiment of the present application, the mask images m of each region are i I is input to the deep neural network layer in the super-resolution network model. The deep neural network layer can extract the image feature information in the region mask image to obtain the corresponding region mask feature information for subsequent super-resolution process.

[0051] S320 , inputting the region mask feature information corresponding to each of the first type information into the corresponding target linear controller for processing, to obtain a region super-resolved image corresponding to the region mask image.

[0052] In an embodiment of the present application, in order to achieve differentiated processing of regional mask images corresponding to different first-type information, in the process of processing regional mask feature information using the linear controller of the super-resolution network model, the mask feature information corresponding to different types of information will be synchronously input into the corresponding target linear controller with different parameters for processing, thereby obtaining the final regional super-resolution image of each regional mask image after super-resolution processing.

[0053] Specifically, in order to process the region mask image, the super-resolution network model usually needs to be trained in advance using a large number of image samples. Therefore, as a feasible embodiment of the present application, a flowchart of the steps of constructing image samples to train the super-resolution network model is provided. Specifically, Figure 4 As shown, Figure 4 A flowchart of a step for training a super-resolution network model provided in an embodiment of the present application, specifically including steps S410 to S450:

[0054] S410 , obtaining a high-resolution sample image, and performing downsampling processing on the high-resolution sample image to obtain a low-resolution sample image.

[0055] In order to train a super-resolution network model, that is, to train a model that can process low-resolution images into high-resolution images, it is usually necessary to rely on sample image pairs consisting of a high-resolution image and a low-resolution image. Therefore, in the embodiments of the present application, a high-resolution sample image is usually first obtained, and then down-sampled to obtain a corresponding low-resolution sample image as the aforementioned sample image pair.

[0056] S420 , performing semantic recognition on the low-resolution sample image to obtain sample first type information in the low-resolution sample image and a sample mask image corresponding to each sample first type information.

[0057] In an embodiment of the present application, during the training process, semantic recognition of low-resolution sample images is also required to obtain sample first type information in the low-resolution sample images and sample mask images corresponding to each first type information.

[0058] S430: Input the sample first type information and the sample mask image into an initial super-resolution network model for processing to obtain a sample super-resolution image.

[0059] In the embodiments of the present application, in order to realize the construction of a super-resolution network model, it is generally necessary to construct an initial super-resolution network model with randomly initialized parameters, so as to facilitate the subsequent optimization and adjustment of the parameters in the initial super-resolution network model, thereby obtaining a sample super-resolution image that can ultimately realize image super-resolution processing. Specifically, the network model structure of the initial super-resolution network model is the same as that of the super-resolution network model mentioned above, and generally also includes a deep neural network layer and a linear controller.

[0060] S440: Fusing the sample super-resolution images to obtain a predicted super-resolution image.

[0061] In the embodiment of the present application, based on the above, by fusing the sample super-resolution images corresponding to each first type of information, a final complete predicted super-resolution image can be obtained. It can be understood that the image difference between the predicted super-resolution image and the initial high-resolution sample image can, to a certain extent, reflect the effect of the image super-resolution processing of the initial super-resolution network model, that is, when the difference between the predicted super-resolution image and the high-resolution sample image is large, it indicates that the image super-resolution processing effect of the super-resolution network model is not good. Conversely, when the difference between the predicted super-resolution image and the high-resolution sample image is small, it indicates that the image super-resolution processing effect of the super-resolution network model is relatively ideal and can be better used for image super-resolution processing.

[0062] S450, adjusting the initial super-resolution network model according to the difference between the predicted super-resolution image and the high-resolution sample image to obtain the super-resolution network model.

[0063] In the embodiment of the present application, in combination with the aforementioned related descriptions, it can be seen that the difference between the predicted super-resolution image and the high-resolution sample image can reflect the effect of the image super-resolution processing of the super-resolution network model. Therefore, based on the idea of ​​backpropagation, the difference between the predicted super-resolution image and the high-resolution sample image can be used to optimize and adjust the network model parameters in the initial super-resolution network model, thereby improving the image super-resolution processing effect of the initial super-resolution network model and obtaining the final trained super-resolution network model that can realize image super-resolution processing. Among them, the difference between the predicted super-resolution image and the high-resolution sample image can be calculated based on the difference in pixel values ​​of corresponding position points, that is, the smaller the difference in pixel values ​​of corresponding position points, the smaller the difference between the predicted super-resolution image and the high-resolution sample image.

[0064] Specifically, in the process of adjusting the initial super-resolution network model to obtain the final trained super-resolution network model, the network model is typically optimized and adjusted multiple times. That is, after adjusting the initial super-resolution network model based on the difference between the predicted super-resolution image and the high-resolution sample image, an optimized super-resolution network model is typically obtained. On this basis, the processed sample first type information and the sample mask image are again input into the optimized super-resolution network model to obtain a new predicted super-resolution image, which is then used to adjust the optimized super-resolution network model again using the difference between the new predicted super-resolution image and the high-resolution sample image until a certain condition is met. For example, when the number of adjustments to the super-resolution network model reaches a certain threshold, or when the difference between the predicted super-resolution image and the high-resolution sample image obtained after several consecutive processing times is less than a certain threshold, the optimization of the super-resolution network model can be considered complete. At this point, the optimized super-resolution network model is now capable of effectively super-resolving the low-resolution sample image into a high-resolution sample image. At this point, the optimized super-resolution network model can be determined as the final super-resolution network model obtained from the training.

[0065] In addition, as a feasible embodiment of the present application, in the aforementioned process of obtaining the super-resolution network model, taking into account the different requirements for super-resolution details of images in different scenes, the super-resolution network model can also be associated with the scene type of the high-resolution sample image and stored, so as to train and obtain super-resolution network models corresponding to different scene types for image super-resolution, so as to facilitate the subsequent process of using the super-resolution network model to complete the processing of the initial image. The regional mask image corresponding to each first type of information can be super-resolved based on the super-resolution network model trained corresponding to the scene type of the initial image. The specific implementation scheme will not be repeated in the embodiment of this application.

[0066] It can be seen that through the above description, the trained super-resolution network model can be used to differentially process the region mask images corresponding to different first-type information, thereby obtaining the region super-resolution image corresponding to each region mask image.

[0067] In addition, considering that the texture feature information in the image will also have a certain impact on the super-resolution image processing, for example, for the flat area in the image, that is, the weak texture area, if the intensity of the image super-resolution processing is high, additional noise will be introduced into the super-resolution image. Therefore, as an additional feasible embodiment of the present application, in the process of super-resolution processing of the regional mask images corresponding to each first type of information based on the first type of information, edge detection will be additionally performed on the regional mask images to extract the texture feature information therein, so as to facilitate the subsequent super-resolution processing of the regional mask images based on the texture feature information and the first type of information. The specific implementation scheme can be referred to below. Figure 5 and its explanatory contents.

[0068] like Figure 5 As shown, Figure 5 A flowchart of a method for extracting texture feature information to perform super-resolution processing on a region mask image provided in an embodiment of the present application, specifically comprising steps S510 to S520:

[0069] S510: Perform edge detection on the region mask image to obtain texture feature information corresponding to the region mask image.

[0070] In an embodiment of the present application, by performing edge detection on the region mask image, texture feature information in the region mask image can be extracted. Specifically, the texture feature information can reflect the flatness of the region in the image to a certain extent, that is, for regions with relatively weak texture feature information, the flatness of the region is relatively high. Conversely, for regions with relatively obvious texture feature information, the flatness of the region is relatively low.

[0071] S520 , performing super-resolution processing on each of the region mask images according to the first type information and the texture feature information to obtain a region super-resolution image corresponding to the region mask image.

[0072] Considering that when super-resolution processing is performed on regions with weak textures, if the intensity of the super-resolution processing is too high, additional noise may be introduced into the image, therefore, in the embodiment of the present application, during the super-resolution processing of each region mask image, the texture feature information in each region mask image is simultaneously considered for adjustment, that is, each region mask image is super-resolution processed based on the first type of information and the texture feature information to obtain a region super-resolution image corresponding to the region mask image. Specifically, similar to the step of super-resolution processing of each region mask image based on the first type of information, in the embodiment of the present application, the first type of information and the texture feature information are input into the super-resolution network model together with the region mask image for processing. The specific implementation scheme of the embodiment of the present application will not be repeated here.

[0073] S230: Fusing the regional super-resolved images to obtain target image information corresponding to the initial image.

[0074] In the embodiment of the present application, after the super-resolution processing of the region mask image corresponding to each first type of information is completed and the super-resolution images of the regions corresponding to each first type of information are obtained, the final target image information can be obtained by further fusing these super-resolution images of the regions. Specifically, the above steps can be implemented by an image semantic coupler corresponding to the above image semantic information decoupler, that is, the target image information I finally obtained is obtained. SR The calculation formula is as follows:

[0075]

[0076] in, That is, the regional super-resolution image corresponding to each region mask image, m i The corresponding mask information.

[0077] The image processing method provided in the embodiments of the present application, after acquiring an initial image to be super-resolved, first performs semantic recognition on the initial image to obtain first-type information in the initial image. The method then combines different first-type information to further super-resolve the initial image, thereby obtaining the final target image information. By identifying the subject in the image and executing the corresponding super-resolution process on the initial image, the present application can ensure the representation of different subjects in the final target image information.

[0078] Specifically, to facilitate understanding of the implementation scenario of the super-resolution image provided in the embodiment of the present application, the following will be explained in detail using the medical imaging field as an example. In this implementation scenario, the initial image is specifically a medical image. For example, it can be an initial medical image with low resolution obtained by scanning the human body through a specific scanning device, such as an electronic computed tomography device, a nuclear magnetic resonance device, etc. At this time, the first type of information obtained by identifying the initial medical image can specifically be the physiological tissue type. On this basis, the physiological tissue type information contained in the initial medical image is used to process the initial medical image. For example, the image areas corresponding to different physiological tissues are super-resolution processed according to different physiological tissue type information, and the final super-resolution medical image corresponding to the initial medical image can be obtained.

[0079] In addition, in the process of the aforementioned super-resolution processing, in order to further improve the performance of the obtained super-resolution image, the image can also be filtered in real time to remove the noise in the image during the image processing process. Specifically, Figure 6 As shown, Figure 6 Another flowchart of image processing steps provided in an embodiment of the present application includes steps S610 to S640:

[0080] S610: Perform filtering on the initial image to obtain a filtered image.

[0081] In the embodiment of the present application, by filtering the initial image, the noise in the image can be effectively removed, and the filtered image obtained can achieve a better super-resolution effect. Specifically, the filtering process can use Gaussian filtering or other filtering methods that can remove noise, which will not be described in detail in the embodiment of the present application.

[0082] S620: Perform semantic recognition on the filtered image to obtain first type information in the filtered image.

[0083] In the embodiment of the present application, the difference from the aforementioned step S120 is that semantic recognition is not performed on the initial image, but on the filtered image after the initial image is processed. For the specific implementation scheme of semantic recognition of the image, please refer to the explanation of the aforementioned step S120. For example, it can be implemented through an image semantic information decoupler, which will not be repeated in the embodiment of the present application.

[0084] S630: Process the filtered image according to the first type information to obtain an initial super-resolution image.

[0085] In the embodiment of the present application, the difference from the aforementioned step S130 is that the initial image is not processed to obtain the super-resolution image, but the filtered image is processed, and the obtained super-resolution image is not used as the target image information finally output, but is the initial super-resolution image that needs to be further processed. The specific implementation scheme of processing the filtered image according to the first type of information can be referred to the explanation of the aforementioned step 130. For example, it can be achieved through a designed super-resolution network model, which will not be repeated in the embodiment of the present application.

[0086] S640: Perform filtering processing on the initial super-resolution image to obtain target image information.

[0087] In the embodiment of the present application, considering that the super-resolution network model may introduce noise in some areas of the image, the image processing device further filters the initial super-resolution image obtained by processing the super-resolution network model to obtain the final noise-free target image information. Of course, the filtering of the initial super-resolution image can also use Gaussian filtering or other filtering methods that can remove noise, which will not be further described in this embodiment of the present application.

[0088] In order to more clearly understand the complete implementation process of the image processing method provided in the embodiment of the present application, the following will be combined with the above Figures 1 to 6 The content provided provides a complete implementation step of an image processing method, specifically including the following steps:

[0089] 1) Design an image semantic information decoupler Seg(·), which takes a low-resolution image I as input and outputs an n-dimensional decoupling coefficient, i.e., the first type of information c i And the mask information m corresponding to each first type of information i , i∈n, as follows:

[0090] The network structure of the image semantic information decoupler Seg(·) can be composed of a deep neural network, and the decoupler output is the decoupling coefficient c i and mask information m i , i∈n, decoupling coefficient c i It is a 1×n array used to represent the category labels of different subject categories in the image, and the mask information m i are n mask images of the same size as the input image I, which are used for image information separation of semantic super-resolution and image fusion after semantic super-resolution:

[0091] c i , m i =Seg(I)

[0092] 2) Design a multimodal semantic super-resolution network model N SR(·), the network model input is the first type of information c i , and the image m processed by mask information i I, the output is n semantic super-resolution images The details are as follows:

[0093] Multimodal semantic super-resolution network model N SR (·) consists of a deep neural network model and a linear controller, wherein the linear controller controls the super-resolution strength. Specifically, the super-resolution strength is different for different subjects and backgrounds. For example, for human subjects, especially faces, a high-intensity super-resolution effect is required, while for subjects in the background area, a low-intensity super-resolution effect can usually be designed. At this time, the output of the semantic information decoupler, i.e., the first type of information c i , mask information m i The dot product m with the low-resolution image I i I as a multimodal semantic super-resolution network model N SR The input of (·) can correspond to the output of n semantic super-resolution images, that is, regional super-resolution images Right now

[0094] 3) Design an image semantic coupler Mer(·), with the input being a semantic super-resolution image and mask information m i , the output is the final super-resolution result of the whole image I SR , as follows:

[0095] The function of the image semantic coupler is to combine n semantic super-resolution images, namely the aforementioned regional super-resolution images Fusion into a single high-resolution image I SS , which is the final super-resolution result of the low-resolution image I, that is, the target image information mentioned above. The main method is to use the mask information m provided by the image semantic information decoupler Seg(·) i right Perform mask fusion to get I SR ,Right now

[0096] Specifically, for ease of understanding, such as Figure 7 As shown in FIG, a schematic diagram of the effect of the above image processing method on the image is shown. It can be seen that after the initial image I is processed by the image semantic information decoupler Seg(·), an array c for representing the subject category in the image is obtained. i And the corresponding mask information m i , then array c i and mask information mi Input to super-resolution network model N SR In (·), the regional super-resolution image corresponding to each first type of information will be obtained Finally, an image semantic coupler Mer(·) is used to super-resolve the region image The corresponding mask information m i After multiplication and fusion, the final super-resolution image I can be obtained. SR .

[0097] In order to better implement the image processing method provided in the embodiment of the present application, on the basis of the image processing method provided in the embodiment of the present application, the embodiment of the present application also provides an image processing device, such as Figure 8 As shown, the image processing device 800 includes:

[0098] An acquisition module 810 is configured to acquire an initial image to be processed;

[0099] a recognition module 820 configured to perform semantic recognition on the initial image to obtain first type information in the initial image;

[0100] The processing module 830 is configured to process the initial image according to the first type of information to obtain target image information corresponding to the initial image.

[0101] Preferably, the processing module 830 is also used to obtain the regional mask image corresponding to each first type of information in the initial image; perform super-resolution processing on the regional mask image corresponding to each first type of information according to the first type of information to obtain the regional super-resolution image corresponding to the regional mask image; and fuse the regional super-resolution images to obtain the target image information corresponding to the initial image.

[0102] Preferably, the processing module 830 is also used to input the region mask image corresponding to the first type of information into the deep neural network layer in the super-resolution network model for processing to obtain the region mask feature information corresponding to each first type of information; adjust the parameters of the linear controller of the super-resolution network model according to each first type of information to obtain the target linear controller corresponding to each first type of information; input the region mask feature information corresponding to each first type of information into the corresponding target linear controller for processing to obtain the region super-resolution image corresponding to the region mask image.

[0103] Preferably, the super-resolution network model is trained by the processing module 830 by executing the following steps: obtaining a high-resolution sample image, and down-sampling the high-resolution sample image to obtain a low-resolution sample image; performing semantic recognition on the low-resolution sample image to obtain sample first type information in the low-resolution sample image and a sample mask image corresponding to each sample first type information; inputting the sample first type information and the sample mask image into the initial super-resolution network model for processing to obtain a sample super-resolution image; fusing the sample super-resolution images to obtain a predicted super-resolution image; and adjusting the initial super-resolution network model according to the difference between the predicted super-resolution image and the high-resolution sample image to obtain the super-resolution network model.

[0104] Preferably, the initial image is an initial medical image; the first type of information is physiological tissue type information; the processing module 830 is also used to process the initial medical image according to the physiological tissue type information to obtain a super-resolution medical image corresponding to the initial medical image.

[0105] Preferably, the processing module 830 is also used to perform edge detection on the region mask image to obtain texture feature information corresponding to the region mask image; and perform super-resolution processing on each region mask image according to the first type information and the texture feature information to obtain a region super-resolution image corresponding to the region mask image.

[0106] Preferably, the recognition module 820 is also used to filter the initial image to obtain a filtered image; perform semantic recognition on the filtered image to obtain the first type of information in the filtered image; the processing module 830 is also used to process the filtered image according to the first type of information to obtain an initial super-resolution image; and perform filtering on the initial super-resolution image to obtain target image information.

[0107] For the specific definition of the image processing device, please refer to the definition of the image processing method above and will not be repeated here. Each module in the above-mentioned image processing device can be implemented in whole or in part through software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each of the above modules.

[0108] In some embodiments of the present application, the image processing device 800 may be implemented in the form of a computer program. The computer program may be implemented in a computer program such as Figure 9 The computer device can store various program modules constituting the image processing apparatus 800, such as: Figure 8 The acquisition module 810, the recognition module 820 and the processing module 830 are shown. The computer program composed of various program modules enables the processor to execute the steps of the image processing method of each embodiment of the present application described in this specification.

[0109] For example, Figure 9 The computer device shown can be Figure 8 The acquisition module 810 in the image processing apparatus 800 shown executes step S110. The computer device can execute step S120 via the identification module 820. The computer device can execute step S130 via the processing module 830. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external computer device via a network connection. When the computer program is executed by the processor, an image processing method is implemented.

[0110] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0111] In some embodiments of the present application, a computer device is provided, comprising one or more processors; a memory; and one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the following steps:

[0112] Obtain the initial image to be processed;

[0113] Performing semantic recognition on the initial image to obtain first type information in the initial image;

[0114] The initial image is processed according to the first type information to obtain target image information corresponding to the initial image.

[0115] In some embodiments of the present application, a computer-readable storage medium is provided, storing a computer program. The computer program is loaded by a processor, causing the processor to perform the following steps:

[0116] Obtain the initial image to be processed;

[0117] Performing semantic recognition on the initial image to obtain first type information in the initial image;

[0118] The initial image is processed according to the first type information to obtain target image information corresponding to the initial image.

[0119] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Any reference to memory, storage, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0120] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0121] The above is a detailed introduction to an image processing method, device, computer equipment and storage medium provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present invention.

Claims

1. An image processing method, characterized in that: include: Obtain the initial image to be processed; Performing semantic recognition on the initial image to obtain first type information in the initial image; The initial image is processed according to the first type information to obtain target image information corresponding to the initial image.

2. The image processing method according to claim 1, wherein: The processing of the initial image according to the first type information to obtain target image information corresponding to the initial image includes: Obtaining a region mask image corresponding to each first type of information in the initial image; performing super-resolution processing on each region mask image corresponding to each of the first type information according to the first type information, to obtain a region super-resolution image corresponding to the region mask image; The regional super-resolved images are fused to obtain target image information corresponding to the initial image.

3. The image processing method according to claim 2, wherein: The performing super-resolution processing on each region mask image corresponding to each of the first type information according to the first type information to obtain a region super-resolution image corresponding to the region mask image includes: Inputting the region mask images corresponding to the first type of information into a deep neural network layer in a super-resolution network model for processing to obtain region mask feature information corresponding to each of the first type of information; The region mask feature information corresponding to each of the first type information is respectively input into the corresponding target linear controller for processing to obtain the region super-resolution image corresponding to the region mask image.

4. The image processing method according to claim 3, wherein: The super-resolution network model is trained by the following steps: Acquiring a high-resolution sample image, and performing downsampling processing on the high-resolution sample image to obtain a low-resolution sample image; Performing semantic recognition on the low-resolution sample image to obtain sample type information in the low-resolution sample image and a sample mask image corresponding to each sample type information; Inputting the sample type information and the sample mask image into an initial super-resolution network model for processing to obtain a sample super-resolution image; Fusing the sample super-resolution images to obtain a predicted super-resolution image; According to the difference between the predicted super-resolution image and the high-resolution sample image, the initial super-resolution network model is adjusted to obtain the super-resolution network model.

5. The image processing method according to claim 2, wherein: The performing super-resolution processing on each region mask image corresponding to each of the first type information according to the first type information to obtain a region super-resolution image corresponding to the region mask image includes: Performing edge detection on the region mask image to obtain texture feature information corresponding to the region mask image; Super-resolution processing is performed on each of the region mask images according to the first type information and the texture feature information to obtain a region super-resolution image corresponding to the region mask image.

6. The image processing method according to claim 1, wherein: The initial image is an initial medical image; the first type of information is physiological tissue type information; Processing the initial image according to the first type of information to obtain target image information corresponding to the initial image includes: The initial medical image is processed according to the physiological tissue type information to obtain a super-resolution medical image corresponding to the initial medical image.

7. The image processing method according to any one of claims 1 to 6, characterized in that: The performing semantic recognition on the initial image to obtain the first type of information in the initial image includes: Performing filtering on the initial image to obtain a filtered image; performing semantic recognition on the filtered image to obtain first type information in the filtered image; The processing of the initial image according to the first type information to obtain target image information corresponding to the initial image includes: Processing the filtered image according to the first type of information to obtain an initial super-resolution image; The initial super-resolution image is filtered to obtain target image information.

8. An image processing method, characterized in that: include: An acquisition module, used for acquiring an initial image to be processed; a recognition module, configured to perform semantic recognition on the initial image to obtain first type information in the initial image; a processing module, configured to process the initial image according to the first type of information to obtain target image information corresponding to the initial image; Preferably, the processing module is further configured to obtain a region mask image corresponding to each first type of information in the initial image; and perform super-resolution processing on each region mask image corresponding to each first type of information according to the first type of information to obtain a region super-resolution image corresponding to the region mask image; Fusing the regional super-resolved images to obtain target image information corresponding to the initial image; Preferably, the processing module is further configured to input the region mask image corresponding to the first type of information into a deep neural network layer in a super-resolution network model for processing, to obtain region mask feature information corresponding to each of the first type of information; Adjusting the parameters of the linear controller of the super-resolution network model according to each of the first types of information to obtain a target linear controller corresponding to each of the first types of information; inputting the region mask feature information corresponding to each of the first types of information into the corresponding target linear controller for processing to obtain a region super-resolved image corresponding to the region mask image; Preferably, the super-resolution network model is trained by a processing module by executing the following steps: obtaining a high-resolution sample image, and performing downsampling processing on the high-resolution sample image to obtain a low-resolution sample image; performing semantic recognition on the low-resolution sample image to obtain sample first type information in the low-resolution sample image and a sample mask image corresponding to each sample first type information; Inputting the sample first type information and the sample mask image into an initial super-resolution network model for processing to obtain a sample super-resolution image; The sample super-resolution images are fused to obtain a predicted super-resolution image; and the initial super-resolution network model is adjusted according to a difference between the predicted super-resolution image and the high-resolution sample image to obtain the super-resolution network model; Preferably, the initial image is an initial medical image; the first type of information is physiological tissue type information; the processing module is further configured to process the initial medical image according to the physiological tissue type information to obtain a super-resolution medical image corresponding to the initial medical image; Preferably, the processing module is further configured to perform edge detection on the region mask image to obtain texture feature information corresponding to the region mask image; and perform super-resolution processing on each of the region mask images according to the first type information and the texture feature information to obtain a region super-resolution image corresponding to the region mask image; Preferably, the recognition module is also used to filter the initial image to obtain a filtered image; perform semantic recognition on the filtered image to obtain the first type of information in the filtered image; the processing module is also used to process the filtered image according to the first type of information to obtain an initial super-resolution image; and perform filtering on the initial super-resolution image to obtain target image information.

9. A computer device, characterized in that: The computer device comprises: one or more processors; Memory; and One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the image processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that A computer program is stored thereon, and the computer program is loaded by a processor to execute the image processing method according to any one of claims 1 to 7.

Images