Information processing method and device

Abstract

The present application embodiment discloses an information processing method, including: obtaining a first image to be processed, a second image to be processed, and a regional image of a first abnormal skin area corresponding to the first image to be processed; determining target information representing a target association relationship between the first abnormal skin area and a second abnormal skin area corresponding to the second image to be processed; wherein the target information is obtained by performing feature analysis on the first image to be processed, the second image to be processed, and the regional image using a skin processing sub-model; performing first feature extraction on the target information using the skin processing sub-model to obtain a first feature map, and performing second feature extraction on the target information using a target object processing sub-model to obtain a second feature map; decoding the first feature map and the second feature map using a region determination model to obtain a target image for the second abnormal skin area. The present application embodiment also discloses an information processing device.

Description

Technical Field

[0001] The present application relates to information processing technology in the field of information processing, and in particular to an information processing method and device. Background Art

[0002] At present, there are some rare skin diseases that have serious effects on the patient's body. Therefore, extracting abnormal skin areas from rare abnormal skin images is particularly important for medical research. Among them, when determining the abnormal skin area, rare sample abnormal skin images are usually obtained first, and a region determination model for extracting abnormal skin areas is trained based on the rare sample abnormal skin images, so as to extract the abnormal skin area from the rare abnormal skin images through the region determination model. However, due to the lack of rare sample abnormal skin images, the sample data for training the region determination model is relatively small, resulting in a low accuracy rate of extracting abnormal skin areas through the trained region determination model. Summary of the Invention

[0003] In order to solve the above technical problems, the embodiments of the present application hope to provide an information processing method and device, which solves the problem that due to the lack of rare sample abnormal skin images, there is less sample data for training the region determination model, resulting in low accuracy in extracting abnormal skin areas through the trained region determination model.

[0004] The technical solution of this application is achieved as follows:

[0005] An information processing method, the method comprising:

[0006] Acquire a first image to be processed, a second image to be processed, and a regional image of a first abnormal skin area corresponding to the first image to be processed;

[0007] Determining target information representing a target association relationship between the first abnormal skin region and a second abnormal skin region corresponding to the second image to be processed; wherein the target information is obtained by performing feature analysis on the first image to be processed, the second image to be processed, and the region image using a skin processing sub-model;

[0008] Performing a first feature extraction on the target information using the skin processing sub-model to obtain a first feature map, and performing a second feature extraction on the target information using the target object processing sub-model to obtain a second feature map;

[0009] The first feature map and the second feature map are decoded and processed through the region determination model to obtain a target image for the second abnormal skin area; wherein, the region determination model is a model for determining the abnormal skin area; the region determination model includes the skin processing sub-model and the target object processing sub-model.

[0010] In the above solution, the target object processing sub-model is different from the skin processing sub-model. The target object processing sub-model is used to extract features of target objects, and the target objects include at least one type of object that is different from the skin.

[0011] In the above solution, the first feature extraction of the target information by the skin processing sub-model to obtain a first feature map, and the second feature extraction of the target information by the target object processing sub-model to obtain a second feature map include:

[0012] performing, by a first meta-learner in the skin processing sub-model, first feature extraction on the target information to obtain the first feature map;

[0013] A second feature extraction is performed on the target information through a second meta-learner in the target object processing sub-model to obtain the second feature map.

[0014] The above scheme also includes:

[0015] Acquire a first sample image set for abnormal skin and a second sample image set for a target object;

[0016] Using a first feature analysis module in the first initial model, perform feature analysis on images in the first sample image set to obtain first information, and using a second feature analysis module in the second initial model, perform feature analysis on images in the second sample image set to obtain second information; wherein the first information represents the association relationship between abnormal skin areas in the first sample image set; and the second information represents the association relationship between target objects in the second sample image set;

[0017] Performing a first feature extraction on the first information using the first feature extraction module in the first initial model to obtain a third feature map, and performing a second feature extraction on the first information using the second feature extraction module in the second initial model to obtain a fourth feature map; performing a first feature extraction on the second information using the first feature extraction module to obtain a fifth feature map, and performing a second feature extraction on the second information using the second feature extraction module to obtain a sixth feature map;

[0018] Based on the third feature map, the fourth feature map, the fifth feature map and the sixth feature map, the first initial model and the second initial model are trained to obtain the area determination model; wherein, the skin processing sub-model is obtained by training the first initial model; and the target object processing sub-model is obtained by training the second initial model.

[0019] In the above solution, the training of the first initial model and the second initial model based on the third feature map, the fourth feature map, the fifth feature map, and the sixth feature map to obtain the region determination model includes:

[0020] Based on the third feature map, the fourth feature map, the fifth feature map and the sixth feature map, the first initial model and the second initial model are alternately trained to obtain the region determination model.

[0021] In the above solution, the step of using the second feature analysis module in the second initial model to perform feature analysis on images in the second sample image set to obtain the second information includes:

[0022] During the N+1th training of the second initial model, processing a first subsample image having a first target object by a first convolutional layer in the second initial model to obtain a seventh feature map of the first target object, and processing a second subsample image having a second target object by the first convolutional layer to obtain an eighth feature map of the second target object; wherein the first target object and the second target object are of the same type;

[0023] Processing the seventh feature map through the dilated convolution layer in the second initial model to obtain a ninth feature map, and processing the eighth feature map through the dilated convolution layer to obtain a tenth feature map;

[0024] The second information is determined based on the ninth feature map, the tenth feature map and the first benchmark segmented image; wherein the first benchmark segmented image is a benchmark image of the first target object corresponding to the first subsample image; wherein the second sample image set includes the first subsample image, the second subsample image and the first benchmark segmented image.

[0025] In the above solution, the first feature extraction module of the first initial model is used to extract the first feature of the first information to obtain the third feature map, and the second feature extraction module of the second initial model is used to extract the second feature of the first information to obtain the fourth feature map, including:

[0026] During the Nth training of the first initial model, extracting a first feature from the first information by a first meta-learner in the first initial model to obtain a third feature map; wherein parameters of the first meta-learner in the first initial model are determined after the N-2th training of the first initial model; and N is a positive integer greater than or equal to 2.

[0027] A second meta-learner in a second initial model is used to perform second feature extraction on the first information to obtain a fourth feature map; wherein parameters in the second meta-learner in the second initial model are determined after the second initial model is trained for the N-1th time.

[0028] In the above solution, the first initial model and the second initial model are alternately trained based on the third feature map, the fourth feature map, the fifth feature map, and the sixth feature map to obtain the region determination model, including:

[0029] During the N-th training of the first initial model, decoding the third feature map and the fourth feature map to obtain a first segmented image of the abnormal skin area corresponding to the third subsample image;

[0030] updating the model parameters in the first initial model based on the first segmented image and the second benchmark segmented image; wherein the second benchmark segmented image is a benchmark image of the abnormal skin area corresponding to the third subsample image;

[0031] During the N+1th training of the second initial model, decoding the fifth feature map and the sixth feature map to obtain a second segmented image of the second target object corresponding to the second subsample image;

[0032] Based on the second segmented image and the third reference segmented image, the model parameters of the second initial model are updated to obtain the region determination model; wherein the third reference segmented image is a reference image of the second target object corresponding to the second subsample image.

[0033] In the above solution, the method further includes:

[0034] After training the first initial model for the Nth time, updating the weights of the first convolutional layer and the weights of the dilated convolutional layer in the second initial model based on the weights of the second convolutional layer in the first initial model;

[0035] After training the second initial model for the N+1th time, the weight of the second convolutional layer in the first initial model is updated based on the weight of the first convolutional layer and the weight of the dilated convolutional layer in the second initial model.

[0036] An information processing device, comprising:

[0037] an acquiring unit, configured to acquire a first image to be processed, a second image to be processed, and a regional image of a first abnormal skin area corresponding to the first image to be processed;

[0038] a processing unit, configured to determine target information representing a target association relationship between the first abnormal skin region and a second abnormal skin region corresponding to the second image to be processed; wherein the target information is obtained by performing feature analysis on the first image to be processed, the second image to be processed, and the region image using a skin processing sub-model;

[0039] The processing unit is further configured to extract a first feature of the target information using the skin processing sub-model to obtain a first feature map, and extract a second feature of the target information using the target object processing sub-model to obtain a second feature map;

[0040] The processing unit is also used to decode the first feature map and the second feature map through the area determination model to obtain a target image for the second abnormal skin area; wherein the area determination model is a model for determining the abnormal skin area; the area determination model includes the skin processing sub-model and the target object processing sub-model.

[0041] The information processing method and device provided by the embodiments of the present application obtain a first image to be processed, a second image to be processed, and a regional image of a first abnormal skin area corresponding to the first image to be processed; determine target information representing a target association relationship between the first abnormal skin area and the second abnormal skin area corresponding to the second image to be processed; wherein the target information is obtained by performing feature analysis on the first image to be processed, the second image to be processed, and the regional image through a skin processing sub-model; extract a first feature of the target information through the skin processing sub-model to obtain a first feature map, and extract a second feature of the target information through the target object processing sub-model to obtain a second feature map; decode the first feature map and the second feature map through a regional determination model to obtain a target association relationship for the second abnormal skin area. A target image of an abnormal skin area; wherein the area determination model is a model for determining the abnormal skin area; the area determination model includes a skin processing sub-model and a target object processing sub-model; in this way, the trained area determination model includes images based on rare abnormal skin images and target objects, and the trained skin processing sub-model and target object processing sub-model no longer rely solely on rare abnormal skin images to train the area determination model, thereby increasing the sample data for training the area determination model, improving the accuracy of the determined area determination model, and thus improving the accuracy of extracting abnormal skin areas through the trained area determination model, solving the problem that the sample data for training the area determination model is small, resulting in low accuracy in extracting abnormal skin areas through the trained area determination model. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A flowchart of an information processing method provided in an embodiment of the present application;

[0043] Figure 2 A flowchart of another information processing method provided in an embodiment of the present application;

[0044] Figure 3 A flowchart of another information processing method provided in an embodiment of the present application;

[0045] Figure 4 A schematic diagram of the structure of a region determination model provided in an embodiment of the present application;

[0046] Figure 5 A schematic diagram of the structure of an information processing device provided in an embodiment of the present application;

[0047] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application.

[0049] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0050] The present application provides an information processing method for an electronic device. Figure 1 As shown, the method includes the following steps:

[0051] Step 101: Acquire a first image to be processed, a second image to be processed, and a regional image of a first abnormal skin area corresponding to the first image to be processed.

[0052] In an embodiment of the present application, the first image to be processed and the second image to be processed are both abnormal skin images; the abnormal type of the abnormal skin corresponding to the first image to be processed is the same as the abnormal type of the abnormal skin corresponding to the second image to be processed; the regional image can be another device with image processing capabilities that processes the first image to be processed and sends it to the electronic device; of course, the regional image can also be the electronic device that analyzes the pixel values ​​of the pixel points in the first image to be processed and obtains the first image to be processed by segmenting the image.

[0053] In a feasible implementation, the first image to be processed and the second image to be processed may both be rare skin disease images; and the regional image may be a binary image obtained by hiding the area outside the first abnormal skin area in the first image to be processed.

[0054] Step 102: Determine target information representing a target association relationship between the first abnormal skin area and the second abnormal skin area corresponding to the second image to be processed.

[0055] The target information is obtained by performing feature analysis on the first image to be processed, the second image to be processed, and the regional image through the skin processing sub-model.

[0056] In an embodiment of the present application, the skin processing sub-model is a partial model in the region determination model; the region determination model is a model for determining the abnormal skin region; the skin processing sub-model is used to perform feature analysis on the abnormal skin image; the electronic device can use the skin processing sub-model to perform feature extraction on the first image to be processed based on the region image to obtain a feature map for the first abnormal skin region, and perform feature extraction on the second image to be processed to obtain an initial feature map, and perform similarity analysis on the feature map of the first abnormal skin region and the initial feature map to obtain target information; wherein the target information can be similar feature information between the first abnormal skin region and the second abnormal skin region.

[0057] In a feasible implementation, the feature information can be presented in the form of an image; the electronic device can splice the feature map of the first abnormal skin area and the initial feature map to obtain a spliced ​​feature map, and perform similarity analysis on the spliced ​​feature map to obtain a feature map that is similar between the first abnormal skin area and the second abnormal skin area.

[0058] Step 103: extract a first feature of the target information using the skin processing sub-model to obtain a first feature map, and extract a second feature of the target information using the target object processing sub-model to obtain a second feature map.

[0059] In this embodiment of the present application, a first feature extraction module in the skin processing sub-model may extract a first feature from the target information to obtain a first feature map; and a second feature extraction module in the target object processing sub-model may extract a second feature from the target information to obtain a second feature map. The first feature map may be an image representing deep-layer skin features of the second abnormal skin area, and the second feature map may be an image representing superficial general features of the second abnormal skin area.

[0060] It should be noted that by extracting the first and second feature maps, the completeness and accuracy of the features of the second abnormal skin area determined are improved. Extracting the first and second feature maps can be performed simultaneously. Alternatively, the first feature map can be extracted first, followed by the second feature map, or the second feature map can be extracted first, followed by the first feature map. The present embodiment does not limit the order in which the first and second feature maps are extracted.

[0061] Step 104: Decode the first feature map and the second feature map using the region determination model to obtain a target image for the second abnormal skin region.

[0062] The region determination model is used to determine abnormal skin areas and includes a skin processing sub-model and a target object processing sub-model. The target object processing sub-model is different from the skin processing sub-model in that it is used to extract features of the target object.

[0063] In an embodiment of the present application, the first feature map and the second feature map can be decoded by a decoder in the region determination model to obtain a target image for the second abnormal skin region.

[0064] In a feasible implementation, the target image may be a binary image obtained by performing concealment processing on the area outside the second abnormal skin area in the second image to be processed.

[0065] The information processing method provided in the embodiment of the present application obtains a first image to be processed, a second image to be processed and a regional image of a first abnormal skin area corresponding to the first image to be processed; determines target information representing the target association relationship between the first abnormal skin area and the second abnormal skin area corresponding to the second image to be processed; wherein the target information is obtained by performing feature analysis on the first image to be processed, the second image to be processed and the regional image through a skin processing sub-model; performs a first feature extraction on the target information through the skin processing sub-model to obtain a first feature map, and performs a second feature extraction on the target information through a target object processing sub-model to obtain a second feature map; in this way, the trained region determination model includes images based on rare abnormal skin images and target objects, and the trained skin processing sub-model and target object processing sub-model no longer rely solely on rare abnormal skin images to train the region determination model, increases the sample data of the training region determination model, improves the accuracy of the determined region determination model, and thereby improves the accuracy of extracting abnormal skin areas through the trained region determination model, solving the problem that the sample data of the training region determination model is small, resulting in low accuracy in extracting abnormal skin areas through the trained region determination model.

[0066] Based on the above embodiments, the embodiments of the present application provide an information processing method, referring to Figure 2 As shown, the method includes the following steps:

[0067] Step 201: The electronic device obtains a first sample image set for abnormal skin and a second sample image set for a target object.

[0068] The number of images in the first sample image set is less than the number of images in the second sample image set; and the target objects include at least one type of object that is different from the skin.

[0069] In an embodiment of the present application, the first sample image set and the second sample image set may be pre-acquired; the first sample image set includes rare abnormal skin images and images of abnormal skin areas; wherein each abnormal skin type has a corresponding plurality of rare abnormal skin images; the number of rare abnormal skin images is greater than the number of images of abnormal skin areas; wherein the images of abnormal skin areas may be images of the area where the abnormal skin is located extracted from the rare abnormal skin images; the images of the abnormal skin areas may be referred to as labeled images corresponding to the rare abnormal skin images. The second sample image set includes images having target objects and images of target object areas; wherein each type of target object corresponds to a plurality of images having the target object; the number of images having the target object is greater than the number of images of the target object areas; wherein the images of the target object areas may be images of the area where the target object is located extracted from the images having the target object; the images of the target object areas may also be referred to as labeled images corresponding to the images having the target object.

[0070] It should be noted that the images in the second sample image set have a rich variety of categories. Using images from this set when training the region determination model can alleviate the limited categories of rare skin disease images and improve the accuracy of the trained region determination model. The images in the second sample image set include categories (types) such as animals, plants, and scenery.

[0071] In a feasible implementation, the images in the second sample image set may be images of aircraft in a natural field.

[0072] Step 202: The electronic device uses the first feature analysis module in the first initial model to perform feature analysis on images in the first sample image set to obtain first information, and uses the second feature analysis module in the second initial model to perform feature analysis on images in the second sample image set to obtain second information.

[0073] The first information represents the association relationship between abnormal skin areas in the first sample image set; and the second information represents the association relationship between target objects in the second sample image set.

[0074] In an embodiment of the present application, the first feature analysis module and the second feature analysis module are different; the first feature analysis module includes a first feature extractor and a first feature fusion module; the second feature analysis module includes a second feature extractor and a second feature fusion module; wherein the structure of the first feature extractor is different from the structure of the second feature extractor; the structure of the first feature fusion module is the same as the structure of the second feature fusion module. The first feature extractor is used to extract deep-level features of images in the first sample image set, wherein the deep-level features generally relate to specific object concepts, such as object categories; the second feature extractor is used to extract shallow-level features of images in the second sample image set; wherein the shallow-level features may be basic colors and edges. The first feature extractor and the second feature extractor may be pre-trained to improve the feature extraction capability of the encoder.

[0075] When determining the first information, specifically, a first feature extractor may be used to first extract features from the image of the i-th abnormal skin area in the first sample image set, corresponding to the image of the i-th abnormal skin area, to obtain a feature map for the i-th abnormal skin image, and then feature extraction may be performed on the m-th abnormal skin image in the first sample image set to obtain a feature map for the m-th abnormal skin image. Subsequently, a first feature fusion module may be used to perform a similarity analysis on the feature maps of the i-th abnormal skin image and the m-th abnormal skin image to obtain the first information. The abnormality type of the abnormal skin corresponding to the i-th abnormal skin image is the same as the abnormality type of the abnormal skin corresponding to the m-th abnormal skin image. i and m are positive integers.

[0076] Specifically, when determining the second information, a second feature extractor may be first used to extract features from the hth image with the target object corresponding to the image of the hth target object region in the second sample image set, thereby obtaining a feature map for the hth image with the target object. Feature extraction is then performed on the jth image with the target object in the second sample image set, thereby obtaining a feature map for the jth image with the target object. Subsequently, a second feature fusion module may be used to perform a similarity analysis on the feature maps of the hth image with the target object and the jth image with the target object, thereby obtaining the second information. The type of target object corresponding to the hth image with the target object is the same as the type of target object corresponding to the jth image with the target object; and j and h are positive integers.

[0077] Step 203: The electronic device uses the first feature extraction module in the first initial model to extract the first feature of the first information to obtain a third feature map, and uses the second feature extraction module in the second initial model to extract the second feature of the first information to obtain a fourth feature map.

[0078] The first information may be a feature map representing the correlation relationship between abnormal skin areas in the first sample image set.

[0079] In an embodiment of the present application, the first information can be input into the first feature extraction module and the second feature extraction module respectively, the first information is analyzed by the first feature extraction module, and the third feature map representing the characteristics of deep abnormal skin is extracted from the first information, and the first information is analyzed by the second feature extraction module, and the fourth feature map representing the general characteristics of shallow abnormal skin is extracted from the first information.

[0080] Step 204: The electronic device uses the first feature extraction module to extract the first feature of the second information to obtain a fifth feature map, and uses the second feature extraction module to extract the second feature of the second information to obtain a sixth feature map.

[0081] The second information may be a feature map representing the correlation relationship between abnormal skin areas in the second sample image set.

[0082] In an embodiment of the present application, the second information can be input into the first feature extraction module and the second feature extraction module respectively, the second information is analyzed by the first feature extraction module, and the fifth feature map representing the deep-level specific features of the target object is extracted from the second information, and the second information is analyzed by the second feature extraction module, and the sixth feature map representing the shallow-level general features of the target object is extracted from the second information.

[0083] Step 205: The electronic device trains the first initial model and the second initial model based on the third feature map, the fourth feature map, the fifth feature map, and the sixth feature map to obtain a region determination model.

[0084] The skin processing sub-model is obtained by training the first initial model; and the target object processing sub-model is obtained by training the second initial model.

[0085] In an embodiment of the present application, a neural network algorithm may be used to train the first initial model and the second initial model based on the third feature map, the fourth feature map, the fifth feature map, the sixth feature map, a first loss function corresponding to the first initial model, and a second loss function corresponding to the second initial model to obtain a region determination model. The first loss function and the second loss function may be the same; however, the first loss function and the second loss function may also be different.

[0086] It should be noted that step 205 can be implemented through step a1:

[0087] a1. Based on the third feature map, the fourth feature map, the fifth feature map, and the sixth feature map, alternately train the first initial model and the second initial model to obtain a region determination model.

[0088] In an embodiment of the present application, during the Nth training of the first initial model, the model parameters of the first initial model are updated based on the third feature map, the fourth feature map and the first loss function. Then, during the N+1 training of the second initial model, the model parameters of the second initial model are updated based on the fifth feature map, the sixth feature map and the second loss function to determine the region determination model; wherein, some weights of the first initial model and the second initial model are shared during the alternating training process.

[0089] Step 206: The electronic device obtains the first image to be processed, the second image to be processed, and a regional image of the first abnormal skin area corresponding to the first image to be processed.

[0090] Step 207: The electronic device determines target information representing a target association relationship between the first abnormal skin area and the second abnormal skin area corresponding to the second image to be processed.

[0091] The target information is obtained by performing feature analysis on the first image to be processed, the second image to be processed, and the regional image through the skin processing sub-model.

[0092] Step 208: The electronic device extracts a first feature of the target information using the skin processing sub-model to obtain a first feature map, and extracts a second feature of the target information using the target object processing sub-model to obtain a second feature map.

[0093] Step 209: The electronic device decodes the first feature map and the second feature map using the region determination model to obtain a target image for the second abnormal skin region.

[0094] Among them, the region determination model is a model used to determine abnormal skin areas; the region determination model includes a skin processing sub-model and a target object processing sub-model.

[0095] It should be noted that, for the description of the same steps and contents in this embodiment as those in other embodiments, reference can be made to the description in other embodiments and will not be repeated here.

[0096] The information processing method provided in the embodiments of the present application includes a trained skin processing sub-model and a target object processing sub-model based on rare abnormal skin images and images of target objects in the trained region determination model. It no longer relies solely on rare abnormal skin images to train the region determination model, increases the sample data of the training region determination model, improves the accuracy of the determined region determination model, and further improves the accuracy of extracting abnormal skin areas through the trained region determination model, solving the problem that the sample data of the training region determination model is small, resulting in low accuracy in extracting abnormal skin areas through the trained region determination model.

[0097] Based on the above embodiments, the embodiments of the present application provide an information processing method, referring to Figure 3 As shown, the method includes the following steps:

[0098] Step 301: The electronic device obtains a first sample image set for abnormal skin and a second sample image set for a target object.

[0099] Step 302: The electronic device uses the first feature analysis module in the first initial model to perform feature analysis on images in the first sample image set to obtain first information.

[0100] The first information represents the correlation between abnormal skin areas in the first sample image set.

[0101] Step 303: During the N+1th training of the second initial model, the electronic device processes the first subsample image having the first target object through the first convolutional layer in the second initial model to obtain the seventh feature map of the first target object, and processes the second subsample image having the second target object through the first convolutional layer to obtain the eighth feature map of the second target object.

[0102] The first target object and the second target object are of the same type; the target objects include the first target object and the second target object; the second sample image set includes a first subsample image, a second subsample image, and a first reference segmented image; the first reference segmented image is a reference image of the first target object corresponding to the first subsample image; and the type of the first target object corresponding to the first subsample image is the same as the type of the second target object corresponding to the second subsample image.

[0103] In the embodiment of the present application, the Visual Geometry Group Network (VGG)-16 has a total of 6 modules. The first convolutional layer can be the first two modules of VGG-16. The first convolutional layer is used to extract features from the first sub-sample image to obtain a seventh feature map representing the shallow features of the first sub-sample image. Similarly, the first convolutional layer is used to extract features from the second sub-sample image to obtain an eighth feature map representing the shallow features of the second sub-sample image. Among them, the first two modules of VGG-16 can be denoted as E 1,2 .

[0104] It should be noted that when training the second initial model, it is expected that the second initial model will acquire the ability to learn general features by learning the images in the second sample image set. Therefore, the number of the first convolutional layer does not need to be too many, and the first two modules of VGG-16 are preferably used.

[0105] Step 304: The electronic device processes the seventh feature map through the dilated convolution layer in the second initial model to obtain a ninth feature map, and processes the eighth feature map through the dilated convolution layer to obtain a tenth feature map.

[0106] In the embodiment of the present application, a dilated convolution layer can be used to extract features from the seventh feature map to obtain a ninth feature map, and a dilated convolution layer can be used to extract features from the eighth feature map to obtain a tenth feature map. The second feature extractor includes a first convolution layer and a dilated convolution layer; the structure of the first feature extractor is different from that of the second feature extractor, and the first feature extractor does not have a dilated convolution layer. The dilated convolution layer is used to extract features from the eighth feature map to obtain a tenth feature map. dila By designing the second feature extractor in this way, the second feature extractor can extract more robust and more general features, thereby improving the generalized learning ability of the region determination model.

[0107] It should be noted that by processing the seventh and eighth feature maps through the atrous convolution layer, the receptive field can be expanded without losing resolution, thereby capturing multi-scale information and contextual information of general features from the first sub-sample image and the second sub-sample image, and improving the generalized learning ability of the trained second initial model.

[0108] Step 305: The electronic device determines second information based on the ninth feature map, the tenth feature map, and the first reference segmented image.

[0109] The first reference segmented image is a reference image of the first target object corresponding to the first sub-sample image; and the second information represents the association relationship between the target objects in the second sample image set.

[0110] In an embodiment of the present application, the electronic device can perform feature extraction on the ninth feature map based on the first benchmark segmented image to obtain a feature map of the first target object corresponding to the first sub-sample image, and through a second feature fusion module, splice the feature map of the first target object corresponding to the first sub-sample image and the tenth feature map, and perform similarity analysis on the spliced ​​feature maps to obtain second information.

[0111] In a feasible implementation, the ninth feature map can be resized to obtain a first sub-feature map of the same size as the first benchmark segmentation image, and the vector extracted from the first benchmark segmentation image and the vector extracted from the first sub-feature map are multiplied to generate a second sub-feature map, and the second sub-feature map is subjected to Mask Averagepooling (average pooling) processing to obtain a third sub-feature map, and then the third sub-feature map is upsampled to obtain a fourth sub-feature map, and the fourth sub-feature map and the tenth feature map are spliced ​​to obtain a fifth sub-feature map, thereby realizing the fusion of features extracted from the first sub-sample image and the second sub-sample image, and then the fifth sub-feature map is subjected to similarity analysis to obtain the sixth sub-feature map; wherein, the role of upsampling is to transform the size of the third sub-feature map to the same size as the tenth feature map; and the second information includes the sixth sub-feature map.

[0112] It should be noted that, by performing average pooling processing on the second sub-feature map, the first target object area and the background area in the first sub-sample image can be learned to filter out the background area, thereby obtaining a third sub-feature map; wherein, the third sub-feature map represents the features corresponding to the target type, and the target type is the type of the first target object corresponding to the first sub-sample image and the second target object corresponding to the second sub-sample image; the features corresponding to the target type can be calculated by the following formula (1):

[0113]

[0114] Where (x, y) represents the coordinates of the pixel in the first subsample image, p c Indicates the features corresponding to the target type; W and H represent the width and height of the first subsample image respectively; Y c represents the first benchmark segmented image, and F represents the first sub-feature map. The first sub-sample image and the second sub-sample image have the same size.

[0115] Performing similarity analysis on the fifth sub-feature map to obtain the sixth sub-feature map can specifically include performing convolution and comparison on the fifth sub-feature map, activating pixels that are most similar to the features of the target type, and obtaining the sixth feature map.

[0116] Step 306 : During the N-th training of the first initial model, the electronic device extracts the first feature of the first information through the first meta-learner in the first initial model to obtain a third feature map.

[0117] The parameters of the first meta-learner in the first initial model are determined after the first initial model is trained for the N-2th time; the parameters of the first meta-learner include at least a weight of the first meta-learner. N is a positive integer greater than or equal to 2.

[0118] In an embodiment of the present application, the first meta-learner is composed of a third convolutional layer and a ReLU (activation function) layer; the first meta-learner is used to extract specific features from the image; the first information can be input into the first meta-learner, and the first meta-learner performs feature extraction on the first information to obtain a third feature map of deep abnormal skin features. The first meta-learner can be referred to as a dedicated learner and can be represented by a specific learner. The weights in the first meta-learner are updated when training the first initial model.

[0119] In an embodiment of the present application, the first sample image set includes a third subsample image having an abnormal skin area, a fourth subsample image having an abnormal skin area, and a fourth benchmark segmented image; wherein the fourth benchmark segmented image is a benchmark image of the abnormal skin area corresponding to the fourth subsample image; the first information can be the first feature analysis module in the first initial model, obtained by processing the third subsample image, the fourth subsample image, and the fourth benchmark segmented image.

[0120] Step 307: The electronic device uses the second meta-learner in the second initial model to extract the second feature of the first information to obtain a fourth feature map.

[0121] The parameters of the second meta-learner in the second initial model are determined after the second initial model is trained for the N-1th time; and the parameters of the second meta-learner include at least a weight of the second meta-learner.

[0122] In an embodiment of the present application, the second meta-learner is composed of a third convolutional layer and a ReLU (activation function) layer; the second meta-learner is used to extract shallow general features of the image; the second information can be input into the second meta-learner, and the second meta-learner performs feature extraction on the first information to obtain a fourth feature map of the general features of shallow abnormal skin.

[0123] It should be noted that the second meta-learner can be called a universal learner, and the first meta-learner can be represented by a generic learner. The first meta-learner and the second meta-learner can be identical in structure.

[0124] Step 308: The electronic device uses the first feature extraction module to extract the first feature of the second information to obtain a fifth feature map, and uses the second feature extraction module to extract the second feature of the second information to obtain a sixth feature map.

[0125] Step 309: During the Nth training of the first initial model, the electronic device decodes the third feature map and the fourth feature map to obtain a first segmented image of the abnormal skin area corresponding to the third subsample image.

[0126] In an embodiment of the present application, the electronic device may input the third feature map and the fourth feature map into a decoder, and process them through the decoder to obtain a first segmented image output by the decoder.

[0127] In a feasible implementation, the first segmented image may be a binary image obtained by performing a hiding process on the area outside the abnormal skin area in the third subsample image.

[0128] Step 310: The electronic device updates the model parameters in the first initial model based on the first segmented image and the second reference segmented image.

[0129] The second reference segmented image is a reference image of the abnormal skin area corresponding to the third sub-sample image.

[0130] In an embodiment of the present application, during the Nth training of the first initial model, a first loss value between the first segmented image and the second reference segmented image may be calculated using a first loss function corresponding to the first initial model, and model parameters in the first initial model may be adjusted based on the first loss value to update the model parameters in the first initial model. The second reference segmented image is acquired in advance.

[0131] Step 311: During the N+1th training of the second initial model, the electronic device decodes the fifth feature map and the sixth feature map to obtain a second segmented image of the second target object corresponding to the second subsample image.

[0132] In an embodiment of the present application, the fifth feature map and the sixth feature map can be input into a decoder and processed by the decoder to obtain a second segmented image output by the decoder.

[0133] In a feasible implementation, the second segmented image may be a binary image obtained by performing a hiding process on the area outside the area where the second target object is located in the second subsample image.

[0134] Step 312: The electronic device updates the model parameters of the second initial model based on the second segmented image and the third reference segmented image to obtain a region determination model.

[0135] The third reference segmented image is a reference image of the second target object corresponding to the second subsample image.

[0136] In an embodiment of the present application, during the N+1th training of the second initial model, the second loss value between the second segmented image and the third benchmark segmented image can be calculated by the second loss function corresponding to the second initial model, and the model parameters in the second initial model can be adjusted based on the second loss value to achieve the update of the model parameters in the second initial model until the first loss value and the second loss value meet the target loss value condition, thereby obtaining the region determination model.

[0137] Step 313: The electronic device obtains the first image to be processed, the second image to be processed, and a regional image of the first abnormal skin area corresponding to the first image to be processed.

[0138] Step 314: The electronic device determines target information representing a target association relationship between the first abnormal skin area and the second abnormal skin area corresponding to the second image to be processed.

[0139] The target information is obtained by performing feature analysis on the first image to be processed, the second image to be processed, and the regional image through the skin processing sub-model.

[0140] Step 315: The electronic device performs first feature extraction on the target information through the first meta-learner in the skin processing sub-model to obtain a first feature map.

[0141] In an embodiment of the present application, the target information may be input into the first meta-learner of the skin processing sub-model, so that the first meta-learner can extract a first feature map representing the specific features of deep abnormal skin from the target information.

[0142] Step 316: The electronic device performs second feature extraction on the target information through the second meta-learner in the target object processing sub-model to obtain a second feature map.

[0143] In an embodiment of the present application, the target information can be input into the second meta-learner of the target object processing sub-model, so that the second meta-learner can extract the general features of shallow abnormal skin from the target information to obtain a second feature map.

[0144] Step 317: Decode the first feature map and the second feature map using the region determination model to obtain a target image for the second abnormal skin region.

[0145] Among them, the region determination model is a model used to determine abnormal skin areas; the region determination model includes a skin processing sub-model and a target object processing sub-model.

[0146] Based on the foregoing embodiment, in other embodiments of the present application, the information processing method further includes the following steps:

[0147] Step 318: After training the first initial model for the Nth time, update the weights of the first convolutional layer and the dilated convolutional layer in the second initial model based on the weights of the second convolutional layer in the first initial model.

[0148] In an embodiment of the present application, after the first initial model is trained for the Nth time, the model parameters in the first initial model can be updated, and the weights of the second convolutional layer in the updated first initial model can be shared with the weights of the first convolutional layer and the weights of the dilated convolutional layer in the second initial model to achieve the update of the weights of the first convolutional layer and the weights of the dilated convolutional layer in the second initial model.

[0149] It should be noted that after training the first initial model for the Nth time, the weights of the modules other than the second meta-learner in the second initial model before training the second initial model for the N+1th time can also be updated based on the weights of the modules other than the first meta-learner in the updated first initial model.

[0150] Specifically, the weight of the second feature analysis module in the second initial model can be updated before the second initial model is trained for the N+1th time based on the weight of the first feature analysis module in the updated first initial model obtained after training the first initial model for the Nth time.

[0151] Step 319: After training the second initial model for the N+1th time, update the weights of the second convolutional layer in the first initial model based on the weights of the first convolutional layer and the weights of the dilated convolutional layer in the second initial model.

[0152] In an embodiment of the present application, after the second initial model is trained for the N+1th time, the model parameters in the second initial model can be updated, and the weights of the first convolutional layer and the weights of the dilated convolutional layer in the updated second initial model are shared with the weights of the second convolutional layer in the first initial model to update the weights of the second convolutional layer in the first initial model. The second feature extractor includes a second convolutional layer; the number of layers of the second convolutional layer is greater than the number of layers of the first convolutional layer; the second convolutional layer can be the first four modules in VGG-16, and the first four modules in VGG-16 can be denoted as E 1,2 and E 3,4 .

[0153] It should be noted that after the second initial model is trained for the N+1th time, the weights of the modules other than the first meta-learner in the first initial model before the first initial model is trained for the N+2th time can also be updated based on the weights of the modules other than the second meta-learner in the updated second initial model.

[0154] Specifically, the weight of the first feature analysis module in the first initial model can be updated before the first initial model is trained for the N+2th time based on the weight of the second feature analysis module in the updated second initial model obtained after training the second initial model for the N+1th time.

[0155] Based on the above embodiments, the following combination Figure 4 , and a detailed explanation of the region determination model is given.

[0156] In an embodiment of the present application, the region determination model is composed of a skin processing sub-model, a target object processing sub-module and a decoder; the region determination model is trained by combining small sample segmentation and meta-learning algorithm.

[0157] Before the training area determination model is used, it is expected that the initial model will acquire the ability to learn general features by learning from small samples of target objects of different categories in the second sample image set, and acquire the ability to learn specific skin lesions by learning from small samples of abnormal skin of different categories in the first sample image set. Based on these goals, the first feature extractor and the second feature extractor are designed respectively. The first feature extractor can be obtained by using E sp To express, E sp＝{ E 1,2 , E 3,4}； The second feature extractor can be used with E ge To express, E ge＝{ E 1,2 , E dila} The initial model includes a first initial model and a second initial model. The first initial model has the same structure as the skin processing sub-model; the second initial model has the same structure as the target object processing sub-model.

[0158] In addition, considering that abnormal skin in the medical field and target objects in the natural field are very different in domain, by designing the first meta-learner and the second meta-learner, adaptive learning is performed in the training stage respectively. During the training process, the first meta-learner and the second meta-learner can be alternately optimized to intensively compare and fuse specialized features and general features.

[0159] When applying the region determination model, the first image to be processed is used as the support image input in the skin processing sub-model, the second image to be processed is used as the query image input in the skin processing sub-model, and the region image of the first abnormal skin region corresponding to the first image to be processed is the marker image corresponding to the support image (e.g. Figure 4 The skin processing submodule in the region determination model can use the second convolution layer (E 1,2 and E3,4 ) performs feature extraction on the first image to be processed, obtains a high-level feature map corresponding to the first image to be processed, and uses the second convolutional layer (E 1,2 and E 3,4 ) Perform feature extraction on the second image to be processed to obtain a high-level feature map corresponding to the second image to be processed, resize the high-level feature map corresponding to the first image to be processed to obtain a resized high-level feature map, and multiply the resized high-level feature map and the regional image in the vector dimension to generate a new feature map, and perform average pooling and upsampling on the new feature map in sequence to obtain a feature map to be analyzed, splice the feature map to be analyzed and the high-level feature map corresponding to the second image to be processed (i.e., the initial feature map), and perform similarity analysis on the spliced ​​feature map to obtain target information, and input the target information into the first meta-learner in the skin processing sub-model and the second meta-learner in the target processing sub-model, respectively, so that the first meta-learner outputs a first feature map representing the specific features of the second abnormal skin area, and the second meta-learner outputs a second feature map representing the general features of the second abnormal skin area, and then inputs the first feature map and the second feature map into the decoder, so that the decoder outputs a target image of the second abnormal skin area. Wherein, the feature map to be analyzed is the feature map of the first abnormal skin area, and the feature map to be analyzed represents the features corresponding to the type of abnormal skin corresponding to the first image to be processed.

[0160] It should be noted that if Figure 4 As shown in the figure, the difference between the structure of the skin processing sub-model and the structure of the target processing sub-model in the region determination model is that the first feature extractor in the skin processing sub-model is composed of the first four modules of VGG16, while the second feature extractor in the target processing sub-model is composed of the first two modules of VGG16 and the hole convolution layer. During training, the first initial model and the second initial model are alternately trained to obtain the skin processing sub-model and the target object processing sub-model. The second feature analysis module is required during the training process, but it is not used in the application process. Only the second meta-learner is used. In other words, Figure 4 The data flow of each module before the second meta-learner in the target object processing sub-model shown in the figure only appears in the training process and does not appear in the application. During the application, the target information obtained in the skin processing sub-model is directly given to the first meta-learner of the skin processing sub-model and the second meta-learner of the target object processing sub-model for processing.

[0161] like Figure 4As shown, in the process of training the first initial model for the Nth time, the first information obtained by processing the first initial model needs to be output to the first meta learner and the second meta learner respectively, and the feature maps output by the first meta learner and the second meta learner are input to the decoder to obtain a binary image of the abnormal skin, and the model parameters of the first initial model are adjusted based on the first loss value between the binary image of the abnormal skin and the baseline binary image of the abnormal skin; in the process of training the second initial model for the N+1th time, the second information obtained by processing the second initial model needs to be output to the first meta learner and the second meta learner respectively, and the feature maps output by the first meta learner and the second meta learner are input to the decoder to obtain a binary image of the target object, and the model parameters of the second initial model are updated based on the second loss value between the binary image of the target object and the baseline binary image of the target object, until the first loss value and the second loss value meet the target loss value condition, and the area determination model is obtained.

[0162] It should be noted that the region determination model provided in the embodiment of the present application enables the model to acquire learning capabilities from images in a natural domain different from the skin domain, thereby effectively learning the segmentation of rare skin disease regions; average pooling is used in the dedicated learning process to generate specific features related to the skin disease region to integrate contextual information from skin images of rare visible classes; in the general learning process, general shallow feature representations from a large number of natural images different from skin images can be explored, thereby helping dedicated learning to identify skin disease regions. The first initial model corresponding to the dedicated learning and the second initial model corresponding to the general learning are collaboratively optimized and alternately updated through a cross-domain meta-learning strategy, achieving mutual promotion and improving the accuracy of the determined region determination model.

[0163] It should be noted that, for the description of the same steps and contents in this embodiment as those in other embodiments, reference can be made to the description in other embodiments and will not be repeated here.

[0164] The information processing method provided in the embodiments of the present application includes a trained skin processing sub-model and a target object processing sub-model based on rare abnormal skin images and images of target objects in the trained region determination model. It no longer relies solely on rare abnormal skin images to train the region determination model, increases the sample data of the training region determination model, improves the accuracy of the determined region determination model, and further improves the accuracy of extracting abnormal skin areas through the trained region determination model, solving the problem that the sample data of the training region determination model is small, resulting in low accuracy in extracting abnormal skin areas through the trained region determination model.

[0165] Based on the above embodiments, the embodiments of the present application provide an information processing device, which can be applied to Figures 1 to 3In the information processing method provided in the corresponding embodiment, refer to Figure 5 As shown, the information processing device 4 includes:

[0166] An acquisition unit 41 is configured to acquire a first image to be processed, a second image to be processed, and a regional image of a first abnormal skin region corresponding to the first image to be processed;

[0167] a processing unit 42 configured to determine target information representing a target association relationship between the first abnormal skin region and the second abnormal skin region corresponding to the second image to be processed; wherein the target information is obtained by performing feature analysis on the first image to be processed, the second image to be processed, and the region image using a skin processing sub-model;

[0168] The processing unit 42 is further configured to extract a first feature of the target information using the skin processing sub-model to obtain a first feature map, and extract a second feature of the target information using the target object processing sub-model to obtain a second feature map;

[0169] The processing unit 42 is also used to decode the first feature map and the second feature map through a region determination model to obtain a target image for the second abnormal skin area; wherein the region determination model is a model for determining the abnormal skin area; the region determination model includes a skin processing sub-model and a target object processing sub-model.

[0170] In other embodiments of the present application, the target object processing sub-model is different from the skin processing sub-model. The target object processing sub-model is used to extract features of the target object, and the target object includes at least one object of a different category from the skin.

[0171] In other embodiments of the present application, the processing unit 42 is specifically configured to perform the following steps:

[0172] Performing a first feature extraction on the target information through the first meta-learner in the skin processing sub-model to obtain a first feature map;

[0173] The second meta-learner in the target object processing sub-model is used to extract the second feature of the target information to obtain a second feature map.

[0174] In other embodiments of the present application, the processing unit 42 is specifically configured to perform the following steps:

[0175] Acquire a first sample image set for abnormal skin and a second sample image set for a target object;

[0176] Using a first feature analysis module in the first initial model, feature analysis is performed on images in the first sample image set to obtain first information, and using a second feature analysis module in the second initial model, feature analysis is performed on images in the second sample image set to obtain second information; wherein the first information represents the association relationship between abnormal skin areas in the first sample image set; and the second information represents the association relationship between target objects in the second sample image set;

[0177] Using the first feature extraction module in the first initial model to extract the first feature of the first information to obtain a third feature map, and using the second feature extraction module in the second initial model to extract the second feature of the first information to obtain a fourth feature map; and using the first feature extraction module to extract the first feature of the second information to obtain a fifth feature map, and using the second feature extraction module to extract the second feature of the second information to obtain a sixth feature map;

[0178] Based on the third feature map, the fourth feature map, the fifth feature map and the sixth feature map, the first initial model and the second initial model are trained to obtain a region determination model; wherein, the skin processing sub-model is obtained by training the first initial model; and the target object processing sub-model is obtained by training the second initial model.

[0179] In other embodiments of the present application, the processing unit 42 is specifically configured to perform the following steps:

[0180] Based on the third feature map, the fourth feature map, the fifth feature map and the sixth feature map, the first initial model and the second initial model are alternately trained to obtain a region determination model.

[0181] In other embodiments of the present application, the processing unit 42 is specifically configured to perform the following steps:

[0182] During the N+1th training of the second initial model, a first subsample image having the first target object is processed by the first convolutional layer in the second initial model to obtain a seventh feature map of the first target object, and a second subsample image having the second target object is processed by the first convolutional layer to obtain an eighth feature map of the second target object; wherein the first target object and the second target object are of the same type;

[0183] The seventh feature map is processed by the dilated convolution layer in the second initial model to obtain a ninth feature map, and the eighth feature map is processed by the dilated convolution layer to obtain a tenth feature map;

[0184] The second information is determined based on the ninth feature map, the tenth feature map and the first benchmark segmented image; wherein the first benchmark segmented image is a benchmark image of the first target object corresponding to the first subsample image; wherein the second sample image set includes the first subsample image, the second subsample image and the first benchmark segmented image.

[0185] In other embodiments of the present application, the processing unit 42 is specifically configured to perform the following steps:

[0186] During the Nth training of the first initial model, extracting a first feature from the first information by a first meta-learner in the first initial model to obtain a third feature map; wherein parameters of the first meta-learner in the first initial model are determined after the N-2th training of the first initial model; and N is a positive integer greater than or equal to 2.

[0187] A second meta-learner in the second initial model is used to extract a second feature of the first information to obtain a fourth feature map; wherein parameters in the second meta-learner in the second initial model are determined after the second initial model is trained for the N-1th time.

[0188] In other embodiments of the present application, the processing unit 42 is specifically configured to perform the following steps:

[0189] During the N-th training of the first initial model, decoding the third feature map and the fourth feature map is performed to obtain a first segmented image of the abnormal skin area corresponding to the third subsample image;

[0190] updating the model parameters in the first initial model based on the first segmented image and the second benchmark segmented image; wherein the second benchmark segmented image is a benchmark image of the abnormal skin area corresponding to the third subsample image;

[0191] During the N+1th training of the second initial model, decoding the fifth feature map and the sixth feature map is performed to obtain a second segmented image of the second target object corresponding to the second subsample image;

[0192] Based on the second segmented image and the third reference segmented image, the model parameters of the second initial model are updated to obtain a region determination model; wherein the third reference segmented image is a reference image of the second target object corresponding to the second subsample image.

[0193] In other embodiments of the present application, the processing unit 42 is specifically configured to perform the following steps:

[0194] After training the first initial model for the Nth time, updating the weights of the first convolutional layer and the dilated convolutional layer in the second initial model based on the weights of the second convolutional layer in the first initial model;

[0195] After training the second initial model for the N+1th time, the weights of the second convolutional layer in the first initial model are updated based on the weights of the first convolutional layer and the weights of the dilated convolutional layer in the second initial model.

[0196] It should be noted that the interaction process between the various units in the embodiment of the present application can refer to Figures 1 to 3 The implementation process of the information processing method provided in the corresponding embodiment will not be repeated here.

[0197] The information processing device provided in the embodiment of the present application includes a trained skin processing sub-model and a target object processing sub-model based on rare abnormal skin images and images of target objects in the trained region determination model. The region determination model is no longer trained solely on rare abnormal skin images, but the sample data for training the region determination model is increased, the accuracy of the determined region determination model is improved, and the accuracy of extracting abnormal skin regions through the trained region determination model is thereby improved, thus solving the problem of low accuracy in extracting abnormal skin regions through the trained region determination model due to the small amount of sample data for the training region determination model.

[0198] Based on the above embodiments, the embodiments of the present application provide an electronic device which can be applied to Figures 1 to 3 In the information processing method provided in the corresponding embodiment, refer to Figure 6 As shown, the electronic device 5 may include: a processor 51, a memory 52 and a communication bus 53, wherein:

[0199] The communication bus 53 is used to realize the communication connection between the processor 51 and the memory 52;

[0200] The processor 51 is used to execute the information processing program stored in the memory 52 to achieve the above Figures 1 to 3 The corresponding embodiments provide steps of the information processing method.

[0201] It should be noted that the specific implementation process of the steps executed by the processor in this embodiment can refer to the implementation process of the information processing method provided in the corresponding embodiments 1 to 3, and will not be repeated here.

[0202] The electronic device provided in the embodiments of the present application includes a trained region determination model based on rare abnormal skin images and images of target objects, a trained skin processing sub-model and a target object processing sub-model. The region determination model is no longer solely dependent on rare abnormal skin images to train the region determination model, but the sample data for training the region determination model is increased, the accuracy of the determined region determination model is improved, and the accuracy of extracting abnormal skin regions through the trained region determination model is thereby improved, thus solving the problem of low accuracy in extracting abnormal skin regions through the trained region determination model due to the small amount of sample data for the training region determination model.

[0203] Based on the above embodiments, the embodiments of the present application provide a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement Figures 1 to 3 The corresponding embodiments provide steps in the information processing method.

[0204] It should be noted that the above-mentioned computer-readable storage medium can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory (Flash Memory), a magnetic surface storage, an optical disc, or a compact disc read-only memory (CD-ROM); it can also be various electronic devices that include one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0205] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0206] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0207] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0208] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0209] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0210] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0211] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An information processing method, comprising: Acquire a first image to be processed, a second image to be processed, and a regional image of a first abnormal skin area corresponding to the first image to be processed; Determining target information representing a target association relationship between the first abnormal skin region and a second abnormal skin region corresponding to the second image to be processed; wherein the target information is obtained by performing feature analysis on the first image to be processed, the second image to be processed, and the region image using a skin processing sub-model; Performing a first feature extraction on the target information using the skin processing sub-model to obtain a first feature map, and performing a second feature extraction on the target information using the target object processing sub-model to obtain a second feature map; the target object processing sub-model is different from the skin processing sub-model, and is used to extract features of target objects, wherein the target objects include at least one type of object different from the skin; The first feature map and the second feature map are decoded and processed through the region determination model to obtain a target image for the second abnormal skin area; wherein, the region determination model is a model for determining the abnormal skin area; the region determination model includes the skin processing sub-model and the target object processing sub-model.

2. The method according to claim 1, wherein the step of extracting a first feature of the target information using the skin processing sub-model to obtain a first feature map, and extracting a second feature of the target information using the target object processing sub-model to obtain a second feature map, comprises: performing, by a first meta-learner in the skin processing sub-model, first feature extraction on the target information to obtain the first feature map; A second feature extraction is performed on the target information through a second meta-learner in the target object processing sub-model to obtain the second feature map.

3. The method according to claim 1, further comprising: Acquire a first sample image set for abnormal skin and a second sample image set for a target object; Using a first feature analysis module in the first initial model, perform feature analysis on images in the first sample image set to obtain first information, and using a second feature analysis module in the second initial model, perform feature analysis on images in the second sample image set to obtain second information; wherein the first information represents the association relationship between abnormal skin areas in the first sample image set; and the second information represents the association relationship between target objects in the second sample image set; Performing a first feature extraction on the first information using the first feature extraction module in the first initial model to obtain a third feature map, and performing a second feature extraction on the first information using the second feature extraction module in the second initial model to obtain a fourth feature map; performing a first feature extraction on the second information using the first feature extraction module to obtain a fifth feature map, and performing a second feature extraction on the second information using the second feature extraction module to obtain a sixth feature map; Based on the third feature map, the fourth feature map, the fifth feature map and the sixth feature map, the first initial model and the second initial model are trained to obtain the area determination model; wherein, the skin processing sub-model is obtained by training the first initial model; and the target object processing sub-model is obtained by training the second initial model.

4. The method according to claim 3, wherein the training of the first initial model and the second initial model based on the third feature map, the fourth feature map, the fifth feature map, and the sixth feature map to obtain the region determination model comprises: Based on the third feature map, the fourth feature map, the fifth feature map and the sixth feature map, the first initial model and the second initial model are alternately trained to obtain the region determination model.

5. The method according to claim 4, wherein the step of using the second feature analysis module in the second initial model to perform feature analysis on images in the second sample image set to obtain the second information comprises: During the N+1th training of the second initial model, processing a first subsample image having a first target object by a first convolutional layer in the second initial model to obtain a seventh feature map of the first target object, and processing a second subsample image having a second target object by the first convolutional layer to obtain an eighth feature map of the second target object; wherein the first target object and the second target object are of the same type; Processing the seventh feature map through the dilated convolution layer in the second initial model to obtain a ninth feature map, and processing the eighth feature map through the dilated convolution layer to obtain a tenth feature map; The second information is determined based on the ninth feature map, the tenth feature map and the first benchmark segmented image; wherein the first benchmark segmented image is a benchmark image of the first target object corresponding to the first subsample image; wherein the second sample image set includes the first subsample image, the second subsample image and the first benchmark segmented image.

6. The method according to claim 4, wherein the first feature extraction module of the first initial model is used to extract the first feature of the first information to obtain the third feature map, and the second feature extraction module of the second initial model is used to extract the second feature of the first information to obtain the fourth feature map, comprising: During the Nth training of the first initial model, extracting a first feature from the first information by a first meta-learner in the first initial model to obtain a third feature map; wherein parameters of the first meta-learner in the first initial model are determined after the N-2th training of the first initial model; and N is a positive integer greater than or equal to 2. A second meta-learner in a second initial model is used to perform second feature extraction on the first information to obtain a fourth feature map; wherein parameters in the second meta-learner in the second initial model are determined after the second initial model is trained for the N-1th time.

7. The method according to claim 4, wherein the step of alternately training the first initial model and the second initial model based on the third feature map, the fourth feature map, the fifth feature map, and the sixth feature map to obtain the region determination model comprises: During the N-th training of the first initial model, decoding the third feature map and the fourth feature map to obtain a first segmented image of the abnormal skin area corresponding to the third subsample image; updating the model parameters in the first initial model based on the first segmented image and the second benchmark segmented image; wherein the second benchmark segmented image is a benchmark image of the abnormal skin area corresponding to the third subsample image; During the N+1th training of the second initial model, decoding the fifth feature map and the sixth feature map to obtain a second segmented image of the second target object corresponding to the second subsample image; Based on the second segmented image and the third reference segmented image, the model parameters of the second initial model are updated to obtain the region determination model; wherein the third reference segmented image is a reference image of the second target object corresponding to the second subsample image.

8. The method according to claim 7, further comprising: After training the first initial model for the Nth time, updating the weights of the first convolutional layer and the weights of the dilated convolutional layer in the second initial model based on the weights of the second convolutional layer in the first initial model; After training the second initial model for the N+1th time, the weight of the second convolutional layer in the first initial model is updated based on the weight of the first convolutional layer and the weight of the dilated convolutional layer in the second initial model.

9. An information processing device, comprising: an acquiring unit, configured to acquire a first image to be processed, a second image to be processed, and a regional image of a first abnormal skin area corresponding to the first image to be processed; a processing unit, configured to determine target information representing a target association relationship between the first abnormal skin region and a second abnormal skin region corresponding to the second image to be processed; wherein the target information is obtained by performing feature analysis on the first image to be processed, the second image to be processed, and the region image using a skin processing sub-model; The processing unit is further configured to perform a first feature extraction on the target information using the skin processing sub-model to obtain a first feature map, and to perform a second feature extraction on the target information using the target object processing sub-model to obtain a second feature map; the target object processing sub-model is different from the skin processing sub-model, and is configured to perform feature extraction on target objects, wherein the target objects include at least one type of object different from the skin; The processing unit is also used to decode the first feature map and the second feature map through a region determination model to obtain a target image for the second abnormal skin area; wherein the region determination model is a model for determining the abnormal skin area; the region determination model includes the skin processing sub-model and the target object processing sub-model.

Images