A skin image segmentation method to reduce the influence of confounding factors

Abstract

The embodiment of this specification discloses a skin image segmentation method for reducing the influence of confounding factors, and relates to the field of image processing technology. The method includes: obtaining a target skin image to be segmented, and an initial noise image obtained by processing the target skin image; performing feature extraction processing on the initial noise image using a first feature conversion model to obtain an initial noise feature vector; searching for neighboring vectors of the initial noise feature vector in the initial feature space; performing hidden space feature representation constraints on the neighboring vectors to obtain multiple target neighboring vectors, and constructing a hidden space feature vector corresponding to each target neighboring vector; performing feature mapping processing on the latent space feature vector through a second feature conversion model to obtain multiple predicted mixed noises for simulating confounding factors; denoising the target skin image based on the predicted mixed noise to obtain a skin image segmentation result corresponding to each predicted mixed noise.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a skin image segmentation method for reducing the influence of confounding factors. Background Art

[0002] Image segmentation is very important in the automated diagnosis process of skin diseases. Accurate lesion boundaries can provide a reliable basis for the subsequent analysis of lesion types. However, due to the confounding factors such as unclear boundaries, complex and diverse skin texture and color, and severe hair occlusion in the lesion area, the segmentation of skin images (such as skin photos, dermoscopic images, etc.) has become a very challenging task.

[0003] In recent years, deep learning methods have been widely used in medical image processing, and a large number of skin image segmentation methods based on convolutional neural networks (CNN) have been proposed to reduce the influence of confounding factors. However, the segmentation methods currently proposed cannot effectively remove the interference of confounding factors, and are prone to problems such as missed recognition of noise, which ultimately leads to the inability to obtain very accurate skin lesion boundaries.

[0004] Based on this, it is necessary to study a skin image segmentation method that can better reduce the influence of confounding factors in order to improve the accuracy of skin image segmentation. Summary of the invention

[0005] To solve the above problems, an embodiment of the present specification provides a skin image segmentation method for reducing the influence of confounding factors, the method comprising:

[0006] Acquire a target skin image to be segmented and an initial noise image obtained by processing the target skin image;

[0007] Using a first feature conversion model to perform feature extraction processing on the initial noise image to obtain an initial noise feature vector;

[0008] Searching for neighboring vectors of the initial noise feature vector in the initial feature space;

[0009] Performing latent space feature representation constraints on the neighboring vectors to obtain multiple target neighboring vectors, and constructing a latent space feature vector corresponding to each target neighboring vector;

[0010] Performing feature mapping processing on the latent space feature vector through a second feature conversion model to obtain a plurality of predicted confounding noises for simulating confounding factors, wherein the first feature conversion model and the second feature conversion model are obtained by performing noise diffusion simulation training using sample skin images;

[0011] Based on the predicted mixed noise, the target skin image is denoised through inverse processing to obtain a skin image segmentation result corresponding to each predicted mixed noise.

[0012] In some embodiments, the initial noise image includes noise corresponding to at least one confounding factor, wherein the confounding factor includes hair occlusion, skin texture, and blurred skin lesion boundary;

[0013] The initial noise image is obtained based on the following method:

[0014] Performing a grayscale closing operation on the target skin image;

[0015] Based on the target skin image and the result of the grayscale closing operation, obtaining a trough area image of the target skin image;

[0016] Performing threshold segmentation on the trough area image to obtain an initial noise mask;

[0017] The initial noise image is obtained based on the initial noise mask and the target skin image.

[0018] In some embodiments, the step of constraining the neighboring vectors to represent hidden space features to obtain a plurality of target neighboring vectors includes:

[0019] Performing pooling processing on the neighboring vectors to process the neighboring vectors into channel-level vectors;

[0020] Performing a filtering operation on the channel-level vector, and obtaining a significance coefficient corresponding to each channel in the channel-level vector based on a processing result of the filtering operation;

[0021] Applying hidden layer processing constraints to the significance coefficient corresponding to each channel according to the KL distance between different channels to obtain a constraint coefficient corresponding to each channel;

[0022] A plurality of target neighboring vectors are obtained according to the channel-level vector and the constraint coefficient corresponding to each channel.

[0023] In some embodiments, constructing a hidden space feature vector corresponding to each target neighboring vector includes:

[0024] Obtaining the closest discrete vector representation corresponding to each target neighboring vector in the initial feature space;

[0025] The latent vector generated by the encoding network in the first feature conversion model for the target neighboring vector is replaced with the closest discrete vector representation to obtain a hidden space feature vector corresponding to each target neighboring vector.

[0026] In some embodiments, the performing feature mapping processing on the latent space feature vector by the second feature conversion model to obtain a plurality of predicted confounding noises for simulating confounding factors includes:

[0027] Mapping the latent space feature vectors corresponding to the plurality of target neighboring vectors to obtain a plurality of predicted mixed noise latent vectors;

[0028] A processing instruction is determined according to the target skin image, and a decoding operation is performed on each of the predicted mixed noise latent vectors according to the processing instruction through a second feature conversion model to determine the predicted mixed noise corresponding to each predicted mixed noise latent vector.

[0029] In some embodiments, the noise diffusion simulation training includes:

[0030] Acquire multiple sample skin images and a sample noise mask corresponding to each sample skin image;

[0031] Introducing noise data with different degrees and within a preset condition range into each of the sample noise masks in sequence, so as to obtain a diffuse noise mask of each sample noise mask under the influence of the noise data with different degrees;

[0032] Performing noise diffusion simulation training on the initial first feature conversion model and the initial second feature conversion model through the diffusion noise mask to obtain noise errors, weight errors and image reconstruction errors corresponding to noise data of different degrees;

[0033] According to the noise error, weight error and image reconstruction error, model parameters of the initial first feature conversion model and the initial second feature conversion model are iteratively optimized to obtain a first feature conversion model and a second feature conversion model.

[0034] In some embodiments, the noise error determination process includes:

[0035] Performing image fusion of the maximum diffuse noise mask corresponding to each sample noise mask and the sample skin image to obtain a sample noise mask fusion image corresponding to each sample noise mask, wherein the maximum diffuse noise mask is a diffuse noise mask that introduces the maximum degree of noise data;

[0036] Performing feature extraction processing on the sample noise mask fusion image through the initial first feature conversion model, and performing latent space feature representation constraints on the result of the feature extraction processing to obtain latent space feature vectors corresponding to each sample noise mask fusion image;

[0037] Performing a decoding operation on the latent space feature vector through an initial second feature conversion model to obtain predicted maximum diffusion degree noise data;

[0038] Determine a predicted second diffuse noise mask corresponding to each sample noise mask according to a difference between a maximum diffuse noise mask and maximum diffuse noise data corresponding to each sample noise mask;

[0039] Determine the sample noise error corresponding to the maximum diffusion degree noise data based on the second diffusion noise mask corresponding to each sample noise mask and the predicted second diffusion noise mask, wherein the second diffusion noise mask is the most recent diffusion noise mask that introduces noise data to a degree lower than the maximum diffusion noise mask;

[0040] The sample noise errors corresponding to all sample noise masks are added together to obtain the noise error.

[0041] In some embodiments, the process of determining the weight error and the image reconstruction error includes:

[0042] Reconstructing the sample noise mask fusion image through the initial second feature conversion model to obtain a reconstructed skin image;

[0043] Mapping the latent space feature vector corresponding to the sample noise mask fusion image to the reconstruction space of the reconstructed skin image to obtain predicted maximum diffusion degree noise data;

[0044] Comparing the sample skin image with the reconstructed skin image to obtain an image reconstruction error corresponding to the maximum diffusion degree noise data;

[0045] Obtain the significance coefficient and constraint coefficient of the hidden space feature vector corresponding to the sample noise mask fusion image in each channel, and obtain the weight error corresponding to the maximum diffusion degree noise data.

[0046] In some embodiments, the iterative optimization of the model parameters of the initial first feature conversion model and the initial second feature conversion model according to the noise error, the weight error and the image reconstruction error to obtain the first feature conversion model and the second feature conversion model includes:

[0047] Preliminarily optimizing the model parameters of the initial first feature conversion model and the initial second feature conversion model according to the noise error, the weight error and the image reconstruction error to obtain a preliminary first feature conversion model and a preliminary second feature conversion model;

[0048] Inputting the verification sample image into the preliminary first feature conversion model and the preliminary second feature conversion model to obtain a model test output;

[0049] Comparative analysis is performed on the model test output and the skin image segmentation result corresponding to the verification sample image to obtain an output matching evaluation result and a distribution variance of the model test output and the skin image segmentation result corresponding to the verification sample image;

[0050] Determining a diffusion rationality index based on the maximum and minimum values ​​of the distribution variance;

[0051] A model performance index is obtained based on the output matching evaluation result and the diffusion rationality index. When the model performance index is higher than a preset threshold, the training is terminated, and the preliminary first feature conversion model is used as the trained first feature conversion model, and the preliminary second feature conversion model is used as the trained second feature conversion model.

[0052] In some embodiments, based on the predicted mixed noise, the target skin image is denoised by inverse processing to obtain a skin image segmentation result corresponding to each predicted mixed noise, including:

[0053] The predicted mixed noise is removed from the target skin image in sequence, and the result obtained after each removal of the current predicted mixed noise is used as the skin image segmentation result corresponding to the current predicted mixed noise.

[0054] The skin image segmentation method provided in the embodiments of this specification to reduce the influence of confounding factors may bring about at least the following beneficial effects:

[0055] (1) Based on the initial noise image, multiple predicted mixed noises are obtained through noise diffusion simulation, and then the target skin image is denoised by the multiple predicted mixed noises, thereby obtaining the corresponding skin image segmentation result, which can effectively improve the segmentation effect of the skin image and reduce the probability of noise caused by mixed factors being missed;

[0056] (2) By constraining the latent space feature representation of the result of feature extraction, the distribution or feature measurement of the latent space can be restricted, avoiding the loss of the feature representation ability of the latent space due to the small weight value of the channel during the feature conversion process, thereby improving the robustness and stability of the model;

[0057] (3) By comparing and analyzing the model test output obtained by processing the validation sample set and the skin image segmentation result corresponding to the validation sample image, the output matching evaluation result is obtained, and the diffusion rationality index of the model is determined based on the distribution variance of the model test output and the skin image segmentation result corresponding to the validation sample image. Then, the model performance index is calculated based on the output matching evaluation result and the diffusion rationality index to evaluate the overall performance of the first feature conversion model and the second feature conversion model. This can improve the accuracy and reliability of the model, ensure that the model performs stably under different conditions, and thus have higher credibility in actual skin image segmentation applications.

[0058] Additional features will be described in part in the following description. For those skilled in the art, it will become apparent by reviewing the following and accompanying drawings, or it may be understood by the production or operation of the examples. The features of this specification may be implemented and obtained by practicing or using various aspects of the methods, tools, and combinations described in the following detailed examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0060] Figure 1 is a schematic diagram of an exemplary application scenario of a skin image segmentation system for reducing the influence of confounding factors according to some embodiments of this specification;

[0061] Figure 2 is an exemplary flow chart of a skin image segmentation method for reducing the influence of confounding factors according to some embodiments of this specification;

[0062] Figure 3 is an exemplary flow chart of a method for determining an initial noise image according to some embodiments of this specification;

[0063] Figure 4 is a schematic diagram of a process for determining an initial noise image according to some embodiments of this specification;

[0064] Figure 5 is an exemplary flow chart of a noise diffusion simulation training process according to some embodiments of the present specification. DETAILED DESCRIPTION

[0065] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of this specification. For ordinary technicians in this field, this specification can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.

[0066] It should be understood that the "system", "device", "unit" and / or "module" used in this specification is a method for distinguishing different components, elements, parts, parts or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0067] As shown in this specification and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "comprise" and "include" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0068] Flowcharts are used in this specification to illustrate the operations performed by the system according to the embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed precisely in order. Instead, the steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.

[0069] The skin image segmentation method and system for reducing the influence of confounding factors provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0070] Figure 1 It is a schematic diagram of an exemplary application scenario of a skin image segmentation system for reducing the influence of confounding factors according to some embodiments of this specification.

[0071] Reference Figure 1 In some embodiments, an application scenario 100 of a skin image segmentation system for reducing the influence of confounding factors may include a skin image acquisition device 110 , a storage device 120 , a processing device 130 , a terminal device 140 , and a network 150 .

[0072] The skin image acquisition device 110 can be used to acquire skin images for a target object (e.g., a patient with a skin disease or a user using the software). In some embodiments, the skin image acquisition device 110 may include a dermatoscope device or a user terminal with an image acquisition function (e.g., a smart phone). In some embodiments, the skin image acquisition device 110 may have an independent power supply, which can send the acquired skin images to other components (e.g., storage device 120, processing device 130, terminal device 140) in the application scenario 100 via wired or wireless (e.g., Bluetooth, WiFi, etc.).

[0073] In some embodiments, the skin image acquisition device 110 can send the skin image it has acquired to the storage device 120, the processing device 130, the terminal device 140, etc. through the network 150. In some embodiments, the skin image acquired by the skin image acquisition device 110 can be processed by the processing device 130. For example, the processing device 130 can obtain a plurality of predicted mixed noises for simulating confounding factors based on the skin image, and then denoise the skin image based on the predicted mixed noises to obtain the skin image segmentation results corresponding to each predicted mixed noise. In some embodiments, the data processed by the processing device 130 can be sent to the storage device 120 for recording, or sent to the terminal device 140 for feedback to the user (e.g., a doctor, a skin disease patient, a software user, etc.).

[0074] The network 150 may facilitate the exchange of information and / or data. The network 150 may include any suitable network that can facilitate the exchange of information and / or data of the application scenario 100. In some embodiments, at least one component of the application scenario 100 (e.g., the skin image acquisition device 110, the storage device 120, the processing device 130, the terminal device 140) may exchange information and / or data with at least one other component in the application scenario 100 through the network 150. For example, the processing device 130 may obtain a skin image acquired for a target object from the skin image acquisition device 110 and / or the storage device 120 through the network 150. For another example, the processing device 130 may obtain a user operation instruction from the terminal device 140 through the network 150, and exemplary operation instructions may include but are not limited to calling up a skin image, reading a skin image segmentation result determined based on the skin image, etc.

[0075] In some embodiments, the network 150 may be any form of wired or wireless network, or any combination thereof. As an example only, the network 150 may include a cable network, a wired network, an optical fiber network, a telecommunications network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a public switched telephone network (PSTN), a Bluetooth network, a ZigBee network, a near field communication (NFC) network, etc., or any combination thereof. In some embodiments, the network 150 may include at least one network access point, and at least one component of the application scenario 100 may be connected to the network 150 through the access point to exchange data and / or information.

[0076] The storage device 120 may store data, instructions and / or any other information. In some embodiments, the storage device 120 may store data obtained from the skin image acquisition device 110, the processing device 130 and / or the terminal device 140. For example, the storage device 120 may store the skin image acquired by the skin image acquisition device 110; for another example, the storage device 120 may store the predicted mixed noise processed by the processing device 130 and the skin image segmentation results corresponding to each predicted mixed noise. In some embodiments, the storage device 120 may store data and / or instructions used by the processing device 130 to execute or use to complete the exemplary method described in this specification. In some embodiments, the storage device 120 may include a large capacity memory, a removable memory, a volatile read-write memory, a read-only memory (ROM), etc., or any combination thereof. Exemplary large capacity memory may include a magnetic disk, an optical disk, a solid-state disk, etc. In some embodiments, the storage device 120 may be implemented on a cloud platform. By way of example only, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multi-layer cloud, etc., or any combination thereof.

[0077] In some embodiments, the storage device 120 may be connected to the network 150 to communicate with at least one other component in the application scenario 100 (e.g., the skin image acquisition device 110, the processing device 130, the terminal device 140). At least one component in the application scenario 100 may access data, instructions, or other information stored in the storage device 120 through the network 150. In some embodiments, the storage device 120 may be directly connected or communicate with one or more components in the application scenario 100 (e.g., the skin image acquisition device 110, the terminal device 140). In some embodiments, the storage device 120 may be part of the skin image acquisition device 110 and / or the processing device 130.

[0078] The processing device 130 can process data and / or information obtained from the skin image acquisition device 110, the storage device 120, the terminal device 140 and / or other components of the application scenario 100. In some embodiments, the processing device 130 can obtain a skin image from any one or more of the skin image acquisition device 110, the storage device 120 or the terminal device 140, and process the skin image to determine the predicted confounding noise used to simulate the confounding factor and the skin image segmentation result corresponding to each predicted confounding noise. In some embodiments, the processing device 130 can obtain a pre-stored computer instruction from the storage device 120, and execute the computer instruction to implement the skin image segmentation method for reducing the influence of confounding factors described in this specification.

[0079] In some embodiments, the processing device 130 may be a single server or a server group. The server group may be centralized or distributed. In some embodiments, the processing device 130 may be local or remote. For example, the processing device 130 may access information and / or data from the skin image acquisition device 110, the storage device 120, and / or the terminal device 140 via the network 150. For another example, the processing device 130 may be directly connected to the skin image acquisition device 110, the storage device 120, and / or the terminal device 140 to access information and / or data. In some embodiments, the processing device 130 may be implemented on a cloud platform. For example, a cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, a cloud-to-cloud cloud, a multi-cloud, etc., or any combination thereof.

[0080] The terminal device 140 can receive, send and / or display data. The received data may include data collected by the skin image acquisition device 110, data stored in the storage device 120, data processed by the processing device 130, etc. For example, the data received and / or displayed by the terminal device 140 may include skin images collected by the skin image acquisition device 110, predicted mixed noises for simulating mixed factors determined by the processing device 130 based on the skin images, and skin image segmentation results corresponding to each predicted mixed noise, etc. The sent data may include user input data and instructions, etc. For example, the terminal device 140 may send the data processing instructions input by the user to the processing device 130 through the network 150 to control the processing device 130 to perform corresponding data processing.

[0081] In some embodiments, the terminal device 140 may include a mobile device 141, a tablet computer 142, a laptop computer 143, etc., or any combination thereof. For example, the mobile device 141 may include a mobile phone, a personal digital assistant (PDA), a dedicated mobile terminal (such as a skin detection analyzer, etc.), etc., or any combination thereof. In some embodiments, the terminal device 140 may include an input device (such as a keyboard, a key, a touch screen), an output device (such as a display, a speaker), etc. In some embodiments, the processing device 130 may be a part of the terminal device 140.

[0082] It should be noted that the above description of the application scenario 100 is only for example and explanation, and does not limit the scope of application of this specification. For those skilled in the art, various modifications and changes can be made to the application scenario 100 under the guidance of this specification. However, these modifications and changes are still within the scope of this specification. For example, the skin image acquisition device 110 may include more or fewer functional components.

[0083] Figure 2 is an exemplary flow chart of a skin image segmentation method for reducing the influence of confounding factors according to some embodiments of this specification. In some embodiments, the method can be performed by processing logic, which can include hardware (e.g., circuits, dedicated logic, programmable logic, microcode, etc.), software (instructions running on a processing device to perform hardware simulation), etc. or any combination thereof. In some embodiments, Figure 2 One or more operations in the flowchart of the skin image segmentation method for reducing the influence of confounding factors can be performed by Figure 1 The method is implemented by the processing device 130 and / or the terminal device 140 shown. For example, the method can be stored in the storage device 120 in the form of a computer program or instruction, and called and / or executed by the processing device 130 and / or the terminal device 140. The execution process of the method is described below by taking the processing device 130 as an example.

[0084] Reference Figure 2 In some embodiments, a skin image segmentation method for reducing the influence of confounding factors may include:

[0085] Step S210, obtaining a target skin image to be segmented and an initial noise image obtained by processing the target skin image.

[0086] Among them, the target skin image refers to the skin image that needs to be processed by the skin image segmentation method for reducing the influence of confounding factors provided by the embodiment of the present application. For example, the target skin image can be a facial skin image taken by a high-resolution camera, or it can be a skin image of other parts of the body obtained by a professional skin detection device (such as a dermatoscope). These images usually contain rich detailed information such as skin texture, pores, hair, and spots, and may also be interfered by factors such as ambient light and equipment noise. In the process of skin lesion area segmentation, these confounding factors may affect the accuracy of skin image segmentation.

[0087] Based on this, in the embodiment of the present application, a noise diffusion simulation method is used to determine multiple predicted mixed noises for simulating mixed factors based on the initial noise, and then denoising is performed based on the multiple predicted mixed noises to obtain skin image segmentation results corresponding to each predicted mixed noise. Among them, the initial noise image can be understood as the noise data obtained by preliminary processing of the target skin image. The noise data may be incomplete or incomplete. If the target skin image is denoised directly based on the initial noise image, there may be problems such as noise leakage recognition.

[0088] Specifically, in the embodiment of the present application, when the user needs to segment the target skin image, the terminal device 140 can submit the target skin image to be segmented to the processing device 130. After obtaining the target skin image, the processing device 130 can perform preliminary noise extraction processing on the target skin image to obtain an initial noise image. Figure 3 In some embodiments of the present application, the process of determining the initial noise image may include the following steps:

[0089] Step S211, performing a grayscale closing operation on the target skin image.

[0090] Grayscale closing operation is a basic operation in mathematical morphology, which can effectively fill small holes and cracks in the target image while maintaining the edge information of the target area. In the embodiment of the present application, the purpose of grayscale closing operation is to reduce small noise points and discontinuous areas in the target skin image, so as to obtain a smoother initial noise image. Specifically, in the embodiment of the present application, the target skin image can be a color image, which can be converted into a corresponding grayscale image (such as Figure 4As shown in Figure A in the figure). In some embodiments, a grayscale closing operation can be performed on the grayscale image corresponding to the target skin image through a structure element with a specific size (for example, 5×5). Among them, the structure element can be understood as a matrix for defining a neighborhood, and its shape and size can be adjusted according to actual needs. Through the grayscale closing operation, small holes and cracks in the target skin image can be filled, so that the subsequent noise extraction is more accurate. The specific implementation content of the grayscale closing operation can be regarded as the prior art and will not be discussed in detail in this specification.

[0091] Step S212, obtaining a valley area image of the target skin image based on the target skin image and the result of the grayscale closing operation.

[0092] In some embodiments, the target skin image can be subtracted from the result of the grayscale closing operation to obtain a valley image of the target skin image (e.g. Figure 4 As shown in Figure B in the figure). It can be understood that the trough image refers to the area with lower grayscale value in the target skin image, which usually corresponds to detail information such as skin texture, pores, hair, etc. By acquiring the trough image, the detail features in the target skin image can be further highlighted, providing a more accurate basis for subsequent noise extraction.

[0093] Step S213, performing threshold segmentation on the trough area image to obtain an initial noise mask.

[0094] After obtaining the valley image of the target skin image through the above steps, the initial noise mask (such as Figure 4 (as shown in Figure C in the figure). Specifically, in the embodiment of the present application, the purpose of threshold segmentation is to distinguish the noise area in the trough area image from the normal skin area. In order to achieve this purpose, a suitable threshold value can be set so that the pixels with grayscale values ​​lower than the threshold in the trough area image are marked as noise areas, and the initial noise mask is obtained based on the noise area. On the contrary, the pixels higher than the threshold are marked as normal skin areas. In some embodiments, the threshold value can be fixed or dynamically determined according to the characteristics of the target skin image. The threshold value is not specifically limited in this specification.

[0095] Step S214: obtaining the initial noise image based on the initial noise mask and the target skin image.

[0096] After obtaining the initial noise mask, the mask can be used to perform a masking operation on the target skin image to extract the initial noise image. Specifically, the value of each pixel in the initial noise image can be determined by the value of the pixel at the corresponding position in the target skin image. If the pixel at this position is marked as a noise area in the initial noise mask, the pixel value at this position is retained; otherwise, the pixel value at this position can be set to zero or other preset values. In this way, an initial noise image containing only noise information can be obtained, which provides a basis for subsequent noise diffusion simulation and denoising processing.

[0097] Furthermore, after obtaining the initial noise image, a noise diffusion simulation method can be used to simulate the influence of confounding factors on the target skin image. Specifically, a mathematical model can be used to perform diffusion processing on the initial noise image so that the noise diffuses in the target skin image in a certain pattern. In some embodiments, a Gaussian diffusion model, a Poisson diffusion model or other suitable diffusion models can be used for noise diffusion simulation. Through noise diffusion simulation, multiple predicted confounding noise images for simulating confounding factors can be obtained, providing a more comprehensive basis for subsequent denoising processing.

[0098] Step S220: Perform feature extraction processing on the initial noise image using the first feature conversion model to obtain an initial noise feature vector.

[0099] In an embodiment of the present application, the first feature conversion model can be a coding module, which may include multiple convolutional layers, activation layers, pooling layers, and fully connected layers. Through the combination of these layers, the initial noise image can be converted into a vector representation in a high-dimensional feature space. In an embodiment of the present application, the purpose of the first feature conversion model is to extract key features in the initial noise image and provide a basis for noise diffusion simulation in subsequent processes.

[0100] In an embodiment of the present application, the first feature conversion model can be obtained by using a large number of sample skin images for noise diffusion simulation training. It can be understood that in the embodiment of the present application, the first feature conversion model is trained by means of noise diffusion simulation, which can improve its ability to mine the relevant complex structure of the real feature distribution in the feature extraction process to a certain extent, and improve the problem of inaccurate segmentation results caused by incomplete noise extraction. The specific training process of the first feature conversion model can be referred to later, and it will not be described in detail here.

[0101] Step S230: searching for neighboring vectors of the initial noise feature vector in the initial feature space.

[0102] In the embodiment of the present application, the initial feature space refers to a high-dimensional feature space generated by the first feature conversion model. In the feature space, each initial noise feature vector corresponds to a point. In order to further improve the recognition accuracy of the noise feature, a nearest neighbor search algorithm can be used to find the nearest neighbor vector closest to the initial noise feature vector in the feature space.

[0103] Specifically, algorithms such as K-nearest neighbor (K-NN), local sensitive hashing (LSH) or other efficient neighboring search methods can be used. Through these methods, several neighboring vectors that are closest to the initial noise feature vector in the feature space can be quickly found (the neighboring vectors can be generated by noise diffusion simulation based on the initial noise feature vector through a noise diffusion simulation model). These neighboring vectors usually contain noise feature information similar to the initial noise feature vector, thereby facilitating subsequent diffusion noise prediction and denoising. In an embodiment of the present application, the number of the neighboring vectors can be set according to actual needs. As an example only, in some embodiments, the number of the neighboring vectors can be set to 10.

[0104] Step S240 , performing latent space feature representation constraints on the neighboring vectors to obtain a plurality of target neighboring vectors, and constructing a latent space feature vector corresponding to each target neighboring vector.

[0105] In an embodiment of the present application, the aforementioned feature extraction process can be understood as a process of mapping the target skin image into a latent space through an encoding module (for example, a first feature conversion model) to obtain a corresponding latent space feature vector. In this processing process, it is also necessary to perform latent space feature representation constraints on the results of the feature extraction process, thereby limiting the distribution or feature measurement of the latent space, avoiding the loss of the feature representation capability of the latent space due to the small weight value of the channel during the feature conversion process, and thereby improving the robustness and stability of the model.

[0106] Exemplarily, in some embodiments, the neighboring vector obtained by the above process can be pooled and processed into a channel-level vector. Then, the channel-level vector is filtered, and the significance coefficient corresponding to each channel in the channel-level vector is obtained based on the processing result of the filtering operation. Further, a hidden layer processing constraint can be applied to the significance coefficient corresponding to each channel according to the KL distance between different channels to obtain the constraint coefficient corresponding to each channel. Finally, multiple target neighboring vectors are obtained according to the channel-level vector and the constraint coefficient corresponding to each channel.

[0107] In an embodiment of the present application, the above-mentioned pooling processing can be understood as a data dimensionality reduction method, which can map the neighboring vectors in the aforementioned initial feature space to a specific coordinate system, thereby projecting the high-dimensional noise features into a low-dimensional feature space and processing each neighboring vector as a channel-level vector.

[0108] Furthermore, the channel-level vector can be filtered, and the significance coefficient corresponding to each channel in the channel-level vector can be obtained based on the processing result of the filtering operation. The significance coefficient can be understood as an indicator to measure the importance of each channel in the feature representation. By calculating the KL distance (Kullback-Leibler divergence) between different channels, the differences between the channels can be evaluated. Based on these differences, hidden layer processing constraints can be applied to the significance coefficient of each channel, thereby obtaining the constraint coefficient corresponding to each channel.

[0109] Specifically, the KL distance is a method for measuring the difference between two probability distributions. In an embodiment of the present application, the significance coefficient of each channel can be obtained by calculating the KL distance between the probability distribution of each channel in the neighboring vector and the reference distribution. Then, according to these significance coefficients, different weights can be applied to each channel to ensure that important channels can get more attention during the feature representation process, while less important channels are appropriately suppressed. It should be pointed out that in an embodiment of the present application, the aforementioned reference distribution refers to a pre-set distribution, which can be determined based on the feature distribution in the training data set. In this way, it can be ensured that the model can better capture the key information in the data during the feature extraction process, thereby improving the generalization ability of the model.

[0110] In some embodiments, the above significance coefficient and constraint coefficient can be expressed by the following formula:

[0111]

[0112]

[0113] in, Indicates The significance coefficient corresponding to each channel; Indicates The probability distribution of channels, represents the reference distribution, Indicates The KL distance between the probability distribution of each channel and the reference distribution; Indicates the number of channels. Indicates The constraint coefficient corresponding to each channel; Represents a constraint function for the significance coefficient Apply hidden layer processing constraints. In the embodiment of the present application, the constraint function It can be a nonlinear transformation, such as an activation function or other form of weight adjustment function.

[0114] The significance coefficient and constraint coefficient calculated by the above formula can be further used to construct the target proximity vector. Specifically, in some embodiments, the target proximity vector can be obtained by the following method: First, according to the channel level vector The corresponding constraint coefficient Multiply to get the constrained channel-level vector Furthermore, by concatenating the constrained channel-level vectors corresponding to each channel, the target neighboring vector can be obtained.

[0115] It should be noted that in practical applications, the above process can be embedded in a deep learning framework, and models such as convolutional neural networks (CNNs) can be used for feature extraction and processing. Through training and optimization, the performance of the model in mixed noise prediction and denoising tasks can be further improved. In addition, in some embodiments, other technologies, such as generative adversarial networks (GANs), can be combined to further improve the denoising effect.

[0116] In some embodiments of the present application, the above-mentioned latent space feature representation constraint process can be implemented using various pooling layers and filtering layers in the deep learning framework. For example, a maximum pooling layer (Max Pooling) or an average pooling layer (Average Pooling) can be used to implement pooling processing, and a convolutional layer (Convolutional Layer) can be used to implement filtering operations. Through the combination of these deep learning components, the feature representation can be effectively constrained, thereby improving the accuracy of noise feature recognition and the robustness of the model.

[0117] Through the above steps, multiple target neighboring vectors can be obtained. Next, the hidden space feature vectors corresponding to each target neighboring vector can be constructed. The hidden space feature vectors corresponding to the target neighboring vectors refer to the closest discrete vector representation corresponding to each target neighboring vector in the aforementioned initial feature space. These hidden space feature vectors will serve as the basis for subsequent processing steps, such as for mixed noise prediction and denoising processing, thereby improving the denoising effect and segmentation effect of skin images.

[0118] Specifically, in some embodiments of the present application, the closest discrete vector representation corresponding to each target neighboring vector in the initial feature space can be obtained, and then the latent vector generated by the encoding network in the first feature conversion model for the target neighboring vector is replaced with the closest discrete vector representation to obtain the hidden space feature vector corresponding to each target neighboring vector.

[0119] Step S250, performing feature mapping processing on the latent space feature vector through a second feature conversion model to obtain multiple predicted confounding noises for simulating confounding factors, wherein the first feature conversion model and the second feature conversion model are obtained by performing noise diffusion simulation training using sample skin images.

[0120] In an embodiment of the present application, the second feature conversion model is a decoding module corresponding to the first feature conversion model, and the second feature conversion model may include a fully connected layer, a conversion layer and a transposed convolution layer, which can convert the latent space feature vector obtained by the aforementioned process into predicted mixed noise.

[0121] Specifically, after obtaining multiple latent space feature vectors corresponding to the initial noise image through the above steps, these latent space feature vectors can be input into the second feature conversion model, and then the processing instructions are determined according to the target skin image, and the latent space feature vectors are decoded according to the processing instructions by the second feature conversion model to obtain the corresponding predicted mixed noise. Exemplarily, in some embodiments, the aforementioned processing instructions may be instructions for noise prediction based on specific areas of the skin image. For example, if there are obvious confounding factors such as hair occlusion in the target skin image, the processing instructions may specify diffuse noise prediction for these areas. In this way, the second feature conversion model can more accurately simulate the impact of confounding factors on the skin image, thereby generating more realistic predicted mixed noise, so as to reduce the impact of these confounding factors on the skin image segmentation through denoising in the subsequent process.

[0122] It should be understood that the process of the second feature conversion model performing feature mapping processing on the aforementioned latent space feature vector is performed by the second feature conversion model based on the data processing capabilities learned during the training process. This process is an abstract process and will not be described in detail in this specification. In some embodiments of the present application, the data conversion process between the aforementioned second feature conversion model and the first feature conversion model can be implemented based on a codebook.

[0123] In the embodiment of the present application, the second feature conversion model and the first feature conversion model can be obtained by performing noise diffusion simulation training using sample skin images. The specific content of the noise diffusion simulation training process can be found in the following text, and will not be discussed in detail here.

[0124] Step S260: Based on the predicted mixed noise, the target skin image is denoised by inverse processing to obtain a skin image segmentation result corresponding to each predicted mixed noise.

[0125] After obtaining multiple predicted mixed noises through the above steps, the target skin image can be further denoised through inverse processing based on the multiple predicted mixed noises to obtain skin image segmentation results corresponding to each predicted mixed noise. The inverse processing can be understood as a process of removing noise from the target skin image based on a mask corresponding to the noise when the noise is known. In some embodiments, the area where the noise is removed can be filled using mean filling or other methods. For example, if there is noise caused by hair occlusion in the target skin image, the inverse processing process can identify these areas and fill them with the pixel mean of the surrounding normal skin area to restore the skin details occluded by the noise.

[0126] In an embodiment of the present application, the above-mentioned predicted mixed noise can be removed from the target skin image in sequence, and the result obtained after each removal of the current predicted mixed noise is used as the skin image segmentation result corresponding to the current predicted mixed noise. Specifically, in an embodiment of the present application, the target skin image can be denoised in an iterative manner. First, the first predicted mixed noise is taken as input, and the noise is removed through a reverse processing step to obtain the first denoised skin image segmentation result. Subsequently, the second predicted mixed noise is taken as input, and reverse processing is performed based on the skin image segmentation result after the previous denoising to obtain the second denoised skin image segmentation result. And so on, until all predicted mixed noises are removed in sequence, and finally a clean skin image segmentation result is obtained.

[0127] In the iterative process, in order to improve the denoising effect, a multi-scale denoising strategy can be used. Specifically, the target skin image can be processed at different scales, such as quickly removing large-scale noise at a coarse scale and finely removing noise at a fine scale. In this way, the denoising speed and denoising accuracy can be effectively balanced, ensuring that the final skin image segmentation result is both clean and has a high degree of detail retention.

[0128] In addition, image enhancement technology can be introduced during the denoising process to further improve the quality of skin images. For example, image sharpening technology can be used to enhance the clarity of skin texture, or color correction technology can be used to adjust the tone of skin images to make them closer to reality. These enhancement operations can be performed on the denoised skin image segmentation results to obtain more ideal visual effects.

[0129] Finally, through the above steps, a high-quality skin image segmentation result after denoising can be obtained. This result can not only effectively reduce the influence of confounding factors, but also retain the important features of skin images, providing reliable image data support for subsequent skin disease diagnosis, skin condition analysis, etc.

[0130] It can be understood that skin image segmentation is the technology and process of dividing the skin image into several specific image areas with unique properties and extracting the area of ​​interest from them. This process is an important step from image processing to image analysis. In the technical solution provided in the embodiment of the present application, multiple predicted mixed noises are obtained by noise diffusion simulation on the basis of the initial noise image, and then the target skin image is denoised by the multiple predicted mixed noises, thereby obtaining the corresponding skin image segmentation result. In this way, the segmentation effect of the skin image can be effectively improved, and the probability of noise caused by confounding factors being missed can be reduced.

[0131] The following is a detailed description of the noise diffusion simulation training process of the first feature conversion model and the second feature conversion model involved in the embodiment of the present application. Figure 5 In some embodiments, the noise diffusion simulation training process may include:

[0132] Step S310, obtaining a plurality of sample skin images and a sample noise mask corresponding to each sample skin image.

[0133] The sample skin image refers to the skin image data in the historical skin image collection process, which can be used for model training. The sample noise mask is the mask corresponding to the initial noise feature obtained by feature extraction of the sample skin image. The determination process of the initial noise feature can be referred to above and will not be repeated here.

[0134] It should be noted that in the embodiments of the present application, the aforementioned sample skin images may include various different confounding factors (multiple confounding factors may also be included in the same sample skin image), which can improve the diversity of training data to a certain extent, thereby enabling the model obtained in subsequent training to have better generalization capabilities.

[0135] Step S320: introducing noise data with different degrees and within a preset condition range into each of the sample noise masks in turn, so as to obtain a diffuse noise mask of each sample noise mask under the influence of the noise data with different degrees.

[0136] In an embodiment of the present application, step S320 can be performed by a noise diffusion simulation model, which can be used to sequentially introduce noise data with different degrees and within a preset condition range for each sample noise mask, and obtain a diffusion noise mask of each sample noise mask under the influence of noise data of different degrees. Among them, the preset condition range can be understood as a specific noise level interval, and the noise level in this interval can be uniformly distributed or change according to a certain specific rule. For example, the noise level can be adjusted according to a Gaussian distribution to simulate the noise conditions that may occur in actual applications.

[0137] Specifically, in an embodiment of the present application, the noise diffusion simulation model may include a neural network structure based on deep learning, which can learn the diffusion law of noise data through training. During the training process, the model receives a sample noise mask and noise data within a preset condition range as input, and outputs a corresponding diffusion noise mask. In order to improve the generalization ability of the model, a variety of different noise data can be used for training, such as Gaussian noise, Poisson noise, salt and pepper noise, etc.

[0138] It should also be noted that, in the embodiments of the present application, the aforementioned noise data of different degrees can be understood as noise data with different noise intensities or levels within the preset condition range. In some embodiments, the noise data of different degrees can constitute a sequence in which the noise intensity gradually increases or decreases. For example, the noise level can change gradually from low to high, or from high to low, to simulate the impact under different noise environments. In some embodiments, noise data with different degrees and within the preset condition range can be introduced into each sample noise mask in turn according to the noise order in the sequence, so as to obtain a diffuse noise mask of each sample noise mask under the influence of noise data of different degrees.

[0139] Step S330, performing noise diffusion simulation training on the initial first feature conversion model and the initial second feature conversion model through the diffusion noise mask to obtain noise errors, weight errors and image reconstruction errors corresponding to noise data of different degrees.

[0140] In an embodiment of the present application, the noise diffusion simulation training of the initial first feature conversion model and the initial second feature conversion model through the above-mentioned diffusion noise mask can be understood as training the noise feature extraction capability of the first feature conversion network based on the diffusion noise mask, and the second feature conversion model obtains the ability to predict mixed noise based on multiple latent space feature vectors corresponding to the noise feature vector extracted by the first feature conversion network. Specifically, in an embodiment of the present application, searching for neighboring vectors of the initial noise feature vector in the initial feature space can be understood as searching for neighboring diffusion noise masks corresponding to the initial noise image in the initial feature space, in other words, searching for neighboring diffusion noise masks with similar features to the initial noise image in the initial feature space. In this way, it can be ensured that the first feature conversion model can more accurately simulate the noise feature extraction and noise diffusion process in the actual noise environment during the noise diffusion simulation training, while enabling the second feature conversion model to more accurately predict mixed noise.

[0141] In the process of performing noise diffusion simulation training on the initial first feature conversion model and the initial second feature conversion model using the diffuse noise masks obtained by introducing noise data of different degrees, the noise error, weight error and image reconstruction error corresponding to the noise data of different degrees can be obtained.

[0142] The process of determining the noise error may include:

[0143] First, a maximum diffuse noise mask corresponding to each sample noise mask is fused with the sample skin image (for example, by weighted superposition) to obtain a sample noise mask fused image corresponding to each sample noise mask, wherein the maximum diffuse noise mask is a diffuse noise mask that introduces the maximum degree of noise data.

[0144] Secondly, the sample noise mask fusion image is subjected to feature extraction processing through the initial first feature conversion model, and the result of the feature extraction processing is subjected to latent space feature representation constraints to obtain latent space feature vectors corresponding to each sample noise mask fusion image. The specific implementation method of the latent space feature representation constraint can be referred to above and will not be repeated here.

[0145] Then, the latent space feature vector is decoded by the initial second feature conversion model to obtain predicted maximum diffusion degree noise data.

[0146] Furthermore, according to the difference between the maximum diffusion noise mask (the maximum diffusion noise mask is generated based on the noise diffusion simulation model) and the maximum diffusion degree noise data (the maximum diffusion degree noise data is predicted based on the second feature conversion model) corresponding to each sample noise mask, the predicted second diffusion noise mask corresponding to each sample noise mask is determined, and based on the second diffusion noise mask and the predicted second diffusion noise mask corresponding to each sample noise mask, the sample noise error corresponding to the maximum diffusion degree noise data is determined, wherein the second diffusion noise mask is the most recent diffusion noise mask that introduces noise data to a degree lower than the maximum diffusion noise mask.

[0147] Finally, the sample noise errors corresponding to all sample noise masks are added together to obtain the noise error.

[0148] In some embodiments, the calculation process of the noise error can be expressed by the following formula:

[0149]

[0150] in, represents the noise error; represents the total number of sample skin images (that is, the total number of sample noise masks); Indicates Sample skin images ( The maximum diffuse noise mask corresponding to the sample noise mask, Indicates Sample skin images ( The (predicted) maximum diffuse noise data corresponding to the sample noise mask, Indicates Sample skin images ( The predicted second diffuse noise mask corresponding to the sample noise mask; Indicates Sample skin images ( It should be noted that in the above formula, the difference between different masks can be understood as the difference between the pixel matrices corresponding to different masks.

[0151] In some embodiments, the sample noise mask fusion image can be reconstructed through the initial second feature conversion model to obtain a reconstructed skin image; then the hidden space feature vector corresponding to the sample noise mask fusion image is mapped to the reconstruction space of the reconstructed skin image to obtain the predicted maximum diffusion degree noise data; finally, by comparing the sample skin image (the sample skin image is the original skin image corresponding to the sample noise mask fusion image) and the reconstructed skin image, the image reconstruction error corresponding to the maximum diffusion degree noise data is obtained. Specifically, in some embodiments, the calculation process of the image reconstruction error can be expressed as follows:

[0152]

[0153] in, represents the image reconstruction error (the error is the error under the maximum noise diffusion degree), Represents the total number of sample skin images; Indicates The original skin image corresponding to the sample skin image, Indicates that based on The reconstructed skin image is obtained by reconstructing the sample noise mask fusion image corresponding to the sample skin image. In the above formula, It can refer to finding the difference between the pixel matrices corresponding to the two images.

[0154] In some embodiments of the present application, the weight error corresponding to the maximum diffusion degree noise data can be obtained by obtaining the significance coefficient and constraint coefficient corresponding to each channel of the latent space feature vector corresponding to the aforementioned sample noise mask fusion image. The calculation process of the weight error can be expressed as follows:

[0155]

[0156] in, represents the weight error, represents the total number of sample skin images (that is, the total number of sample noise mask fusion images), indicates; The number of channels of the latent space feature vector corresponding to the sample noise mask fusion image corresponding to the sample skin image; Indicates The latent space feature vector corresponding to the sample noise mask fusion image corresponding to the sample skin image is The significance coefficient corresponding to the channel is Indicates The latent space feature vector corresponding to the sample noise mask fusion image corresponding to the sample skin image is The constraint coefficient corresponding to each channel.

[0157] Step S340, iteratively optimizing the model parameters of the initial first feature conversion model and the initial second feature conversion model according to the noise error, weight error and image reconstruction error, to obtain a first feature conversion model and a second feature conversion model.

[0158] The noise error is calculated through the above steps. , image reconstruction error And the weight error Then, the noise error can be , image reconstruction error And the weight error The model parameters of the initial first feature conversion model and the initial second feature conversion model are iteratively optimized to obtain the first feature conversion model and the second feature conversion model.

[0159] Specifically, in the embodiment of the present application, the noise error , image reconstruction error And the weight error , the model parameters of the initial first feature conversion model and the initial second feature conversion model are preliminarily optimized to obtain a preliminary first feature conversion model and a preliminary second feature conversion model. The preliminary first feature conversion model and the preliminary second feature conversion model can be understood as feature conversion models obtained by preliminary training.

[0160] Specifically, in some embodiments of the present application, based on the noise error , image reconstruction error And the weight error Get the final LOSS (LOSS = + + ), and then optimize the model parameters of the initial first feature conversion model and the initial second feature conversion model based on the back propagation and gradient descent algorithm so that the final LOSS is gradually reduced. When the final LOSS is reduced to a certain extent, the preliminary first feature conversion model and the preliminary second feature conversion model can be obtained.

[0161] Furthermore, the verification sample image can be input into the preliminary first feature conversion model and the preliminary second feature conversion model to obtain the model test output; then the model test output and the skin image segmentation result corresponding to the verification sample image are compared and analyzed to obtain the output matching evaluation result and the distribution variance of the model test output and the skin image segmentation result corresponding to the verification sample image, and the diffusion rationality index is determined based on the maximum and minimum values ​​of the distribution variance; finally, the model performance index can be obtained based on the output matching evaluation result and the diffusion rationality index, and the training can be terminated when the model performance index is higher than the preset threshold, and the preliminary first feature conversion model is used as the trained first feature conversion model, and the preliminary second feature conversion model is used as the trained second feature conversion model. On the contrary, if the model performance index is lower than the preset threshold, the model parameters of the preliminary first feature conversion model and the preliminary second feature conversion model can be further optimized based on the back propagation and gradient descent algorithms to further reduce the final LOSS.

[0162] Specifically, in some embodiments of the present application, the calculation method of the model performance index can be expressed as follows:

[0163]

[0164] in, represents the model performance index; Indicates the output matching evaluation result; represents the diffusion rationality index; and Respectively represent the corresponding weight coefficients. In some embodiments, and can be equal, that is, both are 0.5. In some embodiments, and They may not be equal, so that the calculation process of the model performance index is more focused on outputting the matching evaluation result or the diffusion rationality index. As an example only, in some embodiments of the present application, It can be 0.4, It can be 0.6. In the embodiment of the present application, the model performance index can be used to characterize the overall performance of the first feature conversion model and the second feature conversion model in processing the skin image segmentation task. and The value of can flexibly control the sensitivity of the model performance index to the output matching evaluation results and the diffusion rationality index, thereby achieving fine control of the model performance.

[0165] In the above formula, and It can be calculated as follows:

[0166]

[0167]

[0168] in, Indicates the output matching evaluation result; represents the diffusion rationality index; Represents the total number of validation sample images; (*) indicates normalization operation; Indicates The model test output corresponding to the validation sample image is Indicates The real skin image segmentation result corresponding to the verification sample image (also called the expected skin image segmentation result); It represents the maximum value of the distribution variance of the skin image segmentation results corresponding to the model test output and the verification sample image. represents the minimum value of the distribution variance of the skin image segmentation result corresponding to the model test output and the verification sample image. It should be noted that in the embodiment of the present application, the difference between the maximum value and the minimum value of the distribution variance ( ) can be used to characterize the generalization ability of the model when processing different samples. If the difference is small, it means that the performance of the model on different samples is relatively consistent and the generalization ability is strong; otherwise, it means that the performance of the model on different samples is quite different and the generalization ability is weak.

[0169] It should also be noted that in the embodiment of the present application, the model test output may include skin image segmentation results corresponding to multiple predicted mixed noises. It may refer to the sum of the differences between the skin image segmentation results corresponding to the multiple predicted mixed noises and the real skin image segmentation result. In other words, in the embodiment of the present application, the difference between the skin image segmentation result corresponding to each predicted mixed noise and the real skin image segmentation result can be calculated, and then these differences can be accumulated to obtain the output matching evaluation result corresponding to a single verification sample image. Furthermore, by calculating the mean of the output matching evaluation results corresponding to the multiple verification sample images, the final output matching evaluation result can be obtained.

[0170] The above has described the basic concepts involved in this application. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements and corrections to this specification. Such modifications, improvements and corrections are suggested in this specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

[0171] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0172] In summary, the beneficial effects that may be brought about by the embodiments of this specification include but are not limited to:

[0173] (1) In the skin image segmentation method for reducing the influence of confounding factors provided in some embodiments of the present specification, multiple predicted confounding noises are obtained by noise diffusion simulation based on the initial noise image, and then the target skin image is denoised by the multiple predicted confounding noises, thereby obtaining the corresponding skin image segmentation result, which can effectively improve the segmentation effect of the skin image and reduce the probability of noise caused by confounding factors being missed;

[0174] (2) In the skin image segmentation method for reducing the influence of confounding factors provided in some embodiments of the present specification, by constraining the latent space feature representation of the result of feature extraction processing, the distribution or feature measurement of the latent space can be restricted, and the loss of the feature representation ability of the latent space due to the excessively small weight value of the channel during the feature conversion process can be avoided, thereby improving the robustness and stability of the model;

[0175] (3) In the skin image segmentation method and system for reducing the influence of confounding factors provided in some embodiments of the present specification, an output matching evaluation result is obtained by comparing and analyzing the model test output obtained by processing the verification sample set and the skin image segmentation result corresponding to the verification sample image, and the diffusion rationality index of the model is determined based on the distribution variance of the model test output and the skin image segmentation result corresponding to the verification sample image. Then, the model performance index is calculated based on the output matching evaluation result and the diffusion rationality index to evaluate the overall performance of the first feature conversion model and the second feature conversion model. This can improve the accuracy and reliability of the model, ensure that the model performs stably under different conditions, and thus have higher credibility in actual skin image segmentation applications.

[0176] It should be noted that different embodiments may produce different beneficial effects. In different embodiments, the beneficial effects that may be produced may be any one or a combination of the above, or may be any other possible beneficial effects.

[0177] At the same time, this specification uses specific words to describe the embodiments of this specification. For example, "one embodiment", "an embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more in different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures or characteristics in one or more embodiments of this specification can be appropriately combined.

[0178] In addition, it will be understood by those skilled in the art that various aspects of this specification may be illustrated and described by a number of patentable categories or situations, including any new and useful process, machine, product or combination of substances, or any new and useful improvements thereto. Accordingly, various aspects of this specification may be performed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software may be referred to as "data blocks", "modules", "engines", "units", "components" or "systems". In addition, various aspects of this specification may be represented as a computer product located in one or more computer-readable media, which includes computer-readable program code.

[0179] A computer storage medium may include a propagated data signal containing computer program code, for example, in baseband or as part of a carrier wave. The propagated signal may be in a variety of forms, including electromagnetic, optical, etc., or a suitable combination. A computer storage medium may be any computer-readable medium other than a computer-readable storage medium, which can be connected to an instruction execution system, device or apparatus to communicate, propagate or transmit the program for use. The program code on the computer storage medium may be transmitted via any suitable medium, including radio, cable, fiber optic cable, RF, or similar media, or any combination of the above media.

[0180] The computer program codes required for the operation of each part of this specification can be written in any one or more programming languages, including object-oriented programming languages ​​such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, Python, etc., conventional procedural programming languages ​​such as C language, Visual Basic, Fortran2003, Perl, COBOL2002, PHP, ABAP, dynamic programming languages ​​such as Python, Ruby and Groovy, or other programming languages, etc. The program code can be run entirely on the user's computer, or run on the user's computer as an independent software package, or run partly on the user's computer and partly on a remote computer, or run entirely on a remote computer or processing device. In the latter case, the remote computer can be connected to the user's computer through any network form, such as a local area network (LAN) or a wide area network (WAN), or connected to an external computer (e.g., via the Internet), or in a cloud computing environment, or used as a service such as software as a service (SaaS).

[0181] In addition, unless explicitly stated in the claims, the order of the processing elements and sequences described in this specification, the use of alphanumeric characters, or the use of other names are not intended to limit the order of the processes and methods of this specification. Although the above disclosure discusses some invention embodiments that are currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the attached claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the essence and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing processing device or mobile device.

[0182] Similarly, it should be noted that in order to simplify the description disclosed in this specification and thus help understand one or more embodiments of the invention, in the above description of the embodiments of this specification, multiple features are sometimes combined into one embodiment, figure or description thereof. However, this disclosure method does not mean that the features required by the subject matter of this specification are more than the features mentioned in the claims. In fact, the features of the embodiments are less than all the features of the single embodiment disclosed above.

[0183] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. A skin image segmentation method for reducing the influence of confounding factors, characterized in that: include: Acquire a target skin image to be segmented and an initial noise image obtained by processing the target skin image; Using a first feature conversion model to perform feature extraction processing on the initial noise image to obtain an initial noise feature vector; Searching for neighboring vectors of the initial noise feature vector in the initial feature space; Performing latent space feature representation constraints on the neighboring vectors to obtain multiple target neighboring vectors, and constructing latent space feature vectors corresponding to each target neighboring vector; Performing feature mapping processing on the latent space feature vector through a second feature conversion model to obtain a plurality of predicted confounding noises for simulating confounding factors, wherein the first feature conversion model and the second feature conversion model are obtained by performing noise diffusion simulation training using sample skin images; Based on the predicted mixed noise, the target skin image is denoised by inverse processing to obtain a skin image segmentation result corresponding to each predicted mixed noise; wherein, The step of constraining the neighboring vectors to represent hidden space features to obtain a plurality of target neighboring vectors includes: Performing pooling processing on the neighboring vectors to process the neighboring vectors into channel-level vectors; Performing a filtering operation on the channel-level vector, and obtaining a significance coefficient corresponding to each channel in the channel-level vector based on a processing result of the filtering operation; Applying hidden layer processing constraints to the significance coefficient corresponding to each channel according to the KL distance between different channels to obtain a constraint coefficient corresponding to each channel; According to the channel-level vector and the constraint coefficient corresponding to each channel, a plurality of target neighboring vectors are obtained; the noise diffusion simulation training includes: Acquire multiple sample skin images and a sample noise mask corresponding to each sample skin image; Introducing noise data with different degrees and within a preset condition range into each of the sample noise masks in sequence, so as to obtain a diffuse noise mask of each sample noise mask under the influence of the noise data with different degrees; Performing noise diffusion simulation training on the initial first feature conversion model and the initial second feature conversion model through the diffusion noise mask to obtain noise errors, weight errors and image reconstruction errors corresponding to noise data of different degrees; Iteratively optimizing the model parameters of the initial first feature conversion model and the initial second feature conversion model according to the noise error, the weight error and the image reconstruction error to obtain a first feature conversion model and a second feature conversion model; The noise error is calculated based on the following formula: Among them, E noise represents the noise error, T represents the total number of sample skin images, M max，t represents the maximum diffuse noise mask corresponding to the t-th sample skin image, M' max，t M represents the maximum diffusion noise data corresponding to the t-th sample skin image; pred，t represents the second diffuse noise mask corresponding to the t-th sample skin image, the maximum diffuse noise mask is the diffuse noise mask that introduces the maximum degree of noise data, and the second diffuse noise mask is the most recent diffuse noise mask that introduces noise data to a degree lower than the maximum diffuse noise mask; The image reconstruction error is calculated based on the following formula: Among them, E recon represents the image reconstruction error, I orig，t represents the original skin image corresponding to the t-th sample skin image, I recon，t represents the reconstructed skin image obtained by reconstructing the sample noise mask fusion image corresponding to the t-th sample skin image, I orig，t -I recon，t It refers to the difference between the pixel matrices corresponding to the two images; the weighted error is calculated based on the following formula: Among them, E weight represents the weight error, n_t represents the number of channels of the hidden space feature vector corresponding to the sample noise mask fusion image corresponding to the tth sample skin image; S t，i represents the significance coefficient of the latent space feature vector corresponding to the sample noise mask fusion image corresponding to the t-th sample skin image in the ith channel, C t，i Represents the constraint coefficient corresponding to the i-th channel of the latent space feature vector corresponding to the sample noise mask fusion image corresponding to the t-th sample skin image.

2. The method according to claim 1, characterized in that The initial noise image includes noise corresponding to at least one confounding factor, wherein the confounding factor includes hair occlusion, skin texture, and blurred skin lesion boundary; The initial noise image is obtained based on the following method: Performing a grayscale closing operation on the target skin image; Based on the target skin image and the result of the grayscale closing operation, obtaining a trough area image of the target skin image; Performing threshold segmentation on the trough area image to obtain an initial noise mask; The initial noise image is obtained based on the initial noise mask and the target skin image.

3. The method according to claim 2, characterized in that The step of constructing a hidden space feature vector corresponding to each target neighboring vector includes: Obtaining the closest discrete vector representation corresponding to each target neighboring vector in the initial feature space; The latent vector generated by the encoding network in the first feature conversion model for the target neighboring vector is replaced with the closest discrete vector representation to obtain a hidden space feature vector corresponding to each target neighboring vector.

4. The method according to claim 3, characterized in that The step of performing feature mapping processing on the latent space feature vector by the second feature conversion model to obtain a plurality of predicted confounding noises for simulating confounding factors includes: Mapping the latent space feature vectors corresponding to the plurality of target neighboring vectors to obtain a plurality of predicted mixed noise latent vectors; A processing instruction is determined according to the target skin image, and a decoding operation is performed on each of the predicted mixed noise latent vectors according to the processing instruction through a second feature conversion model to determine the predicted mixed noise corresponding to each predicted mixed noise latent vector.

5. The method according to claim 1, characterized in that The iterative optimization of the model parameters of the initial first feature conversion model and the initial second feature conversion model according to the noise error, the weight error and the image reconstruction error to obtain the first feature conversion model and the second feature conversion model includes: Preliminarily optimizing the model parameters of the initial first feature conversion model and the initial second feature conversion model according to the noise error, the weight error and the image reconstruction error to obtain a preliminary first feature conversion model and a preliminary second feature conversion model; Inputting the verification sample image into the preliminary first feature conversion model and the preliminary second feature conversion model to obtain a model test output; Comparative analysis is performed on the skin image segmentation results corresponding to the model test output and the verification sample image to obtain an output matching evaluation result and a distribution variance of the skin image segmentation results corresponding to the model test output and the verification sample image; Determining a diffusion rationality index based on the maximum and minimum values ​​of the distribution variance; A model performance index is obtained based on the output matching evaluation result and the diffusion rationality index. When the model performance index is higher than a preset threshold, the training is terminated, and the preliminary first feature conversion model is used as the trained first feature conversion model, and the preliminary second feature conversion model is used as the trained second feature conversion model.

6. The method according to any one of claims 1 to 5, characterized in that: The step of denoising the target skin image based on the predicted mixed noise by inverse processing to obtain a skin image segmentation result corresponding to each predicted mixed noise includes: The predicted mixed noise is removed from the target skin image in sequence, and the result obtained after each removal of the current predicted mixed noise is used as the skin image segmentation result corresponding to the current predicted mixed noise.

Images