Skin cancer diagnosis model training method based on neural network ensemble architecture search

Abstract

The present invention discloses a skin cancer diagnosis model training method based on neural network integrated architecture search, comprising the following steps: step 1, collecting a dermatoscope image data set and annotating each image in the data set; step 2, designing a search space: using the ResNet50 convolutional neural network as the basic structure, by expanding and changing the basic components in the basic model, a comprehensive architecture search space is constructed; step 3, implementing a search strategy: using a one-time learning neural network architecture search architecture and a bagging-based integrated learning method; step 4, evaluating the performance and diversity of the sub-architecture; step 5, after completing the search, integrating multiple sub-architectures with higher performance in the sub-architecture group. The present invention solves the problems of low efficiency, limited performance, and difficulty in coping with complex data challenges in traditional skin cancer intelligent auxiliary diagnosis technology, and improves the accuracy and robustness of skin cancer classification.

Description

Technical Field

[0001] The present invention relates to the field of deep learning and medical image analysis, and in particular to a method for training a skin cancer diagnosis model based on neural network integrated architecture search. The method aims to optimize and improve the classification performance of neural networks in intelligent auxiliary diagnosis of skin cancer, and is widely used in medical image analysis, skin disease detection and intelligent diagnosis systems. Background Art

[0002] Skin cancer is a common malignancy with an increasing incidence worldwide. The main types include basal cell carcinoma (BCC), squamous cell carcinoma (SCC) and malignant melanoma (MM), which are usually associated with long-term ultraviolet (UV) exposure. As the early symptoms of skin cancer are not easy to detect and the lesion site is highly specific, early diagnosis and timely treatment are crucial to improve survival rates. Traditional methods for diagnosing skin cancer include clinical examination, dermoscopy and tissue biopsy. Clinical examination is performed by dermatologists to make a preliminary assessment by observing the appearance characteristics of the lesion area. However, this method has limited accuracy in identifying early lesions or lesions similar to normal skin, which can easily lead to missed or misdiagnosis. Dermoscopy is a non-invasive imaging technology that can magnify the details of the skin lesion area and enhance the recognition of lesion characteristics. Although dermoscopy improves the sensitivity of detection, its results rely on the professional experience and subjective judgment of the physician and have certain limitations. Tissue biopsy is considered the gold standard for the diagnosis of skin cancer, and the presence of cancer is confirmed by taking samples and observing the lesion tissue under a microscope. However, as an invasive procedure, biopsy may cause discomfort and complications, and the acquisition cycle of pathological results is long, which may lead to delayed diagnosis. Although traditional diagnostic methods have their own application value, they still have certain limitations. With the development of deep learning technology, deep neural network methods have been widely used in skin cancer auxiliary diagnosis research, providing new solutions for the early detection and diagnosis of skin cancer.

[0003] Deep Neural Networks (DNNs) are a type of artificial neural network that consists of multiple hidden layers and can learn and recognize complex patterns through progressive nonlinear transformations. The core of the deep neural network model lies in its multi-level structural design. Each layer of neurons receives the input from the previous layer, and after linear transformation and nonlinear activation function processing, it outputs to the next layer. This multi-level processing can extract high-level features of the input data, thereby showing excellent performance in pattern recognition and classification tasks. In the field of skin cancer diagnosis, DNNs, especially convolutional neural networks (CNNs), are widely used in the analysis and classification of skin lesion images. The application of deep neural networks (DNNs) in skin cancer diagnosis mainly includes the following steps: First, skin surface lesion images are collected through dermatoscopes or other imaging devices, and standardized processing such as resizing, color normalization and noise removal is performed on them. Then, a large amount of preprocessed image data is used to train the deep neural network model. The model gradually extracts features in the image through multi-layer convolution and pooling, and finally outputs the classification results. In this process, the back propagation algorithm is used to optimize the network weights to improve the accuracy of classification. In order to improve model performance, it is usually necessary to repeatedly adjust the model architecture and parameter settings. It can be seen that although deep neural networks have the advantages of high accuracy and automation in skin cancer diagnosis, can greatly reduce the workload of doctors and adapt to different types of medical images, building an efficient neural network model often requires repeated adjustments to the model architecture, which is inefficient and takes a lot of time.

[0004] In order to improve the efficiency of model construction, the Neural Architecture Search method (NAS) was proposed to improve the design efficiency of skin cancer intelligent auxiliary diagnosis model. Traditional neural network design usually relies on the experience of experts and requires a lot of experiments and tuning processes. However, NAS greatly reduces the design time and manpower investment by automatically finding the best structure in a pre-defined architecture search space. The basic operation of NAS includes three main steps: First, define a search space containing all possible network architectures, which vary in the number of layers, connection methods, and activation functions. Then, use search strategies to explore in this space. Common strategies include reinforcement learning, evolutionary algorithms, or Bayesian optimization. Finally, the candidate architectures are tested and compared through performance evaluation to select the optimal structure. In the design of skin cancer intelligent auxiliary diagnosis model, NAS can significantly improve the construction efficiency. However, early neural network architecture search methods require training a large number of neural networks for architecture evaluation, which is computationally expensive and limits its widespread application. In 2016, Google proposed to generate neural network architectures through recurrent neural networks and use reinforcement learning to optimize controllers. This method achieved breakthrough results on the CIFAR-10 dataset. However, this method is difficult to promote due to the high resource consumption of the evaluation process. To solve this problem, the One-Shot NAS search was born. This method greatly reduces the computational cost by training a super network and using a weight sharing strategy, making related research more popular. However, since skin cancer medical image data usually has problems such as small sample size, unbalanced categories, and a large number of difficult samples, a single architecture may not be able to perform evenly on all categories. Models that rely on a single architecture cannot fully capture the diversity and complexity in the data, resulting in limited classification performance. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a method for searching skin surface lesion image data collected by a dermatoscope through a neural network integrated architecture, searching for multiple architectures with excellent performance and high diversity in one search, and improving the performance of the skin cancer classification diagnosis model through an integrated method. A skin cancer diagnosis model training method based on neural network integrated architecture search.

[0006] The objective of the present invention is achieved through the following technical solution: a skin cancer diagnosis model training method based on neural network integrated architecture search, comprising the following steps:

[0007] Step 1: Collect a dermoscopic image dataset and annotate each image in the dataset, including the lesion type and lesion area, to form a labeled dermoscopic image dataset, which is a training set and a validation set;

[0008] Step 2: Design the search space: Use the ResNet50 convolutional neural network as the basic structure, and combine it with the CAM-based global lesion localization module and the global feature-based intra-class center learning module; build a comprehensive architecture search space by expanding and changing the basic components in the basic model; including the following sub-steps:

[0009] Step 21: Build the ResNet50 basic model;

[0010] Step 22: Integrate the global lesion localization module and the intra-class center learning module: Add a CAM-based global lesion localization module after the last convolutional layer of ResNet50, and add an intra-class center learning module based on global features after the pooling layer;

[0011] Step 23: Expansion and optimization of search space: First, expand the network's expressive power by diversifying the types of convolutional layers: use convolutional layers of different numbers in the residual block, introduce convolutional kernels of different sizes in the convolutional layer, and use different convolution methods to form a variety of convolution combinations;

[0012] Secondly, in each residual block, a variety of different module connection methods are used; attention mechanism modules are added to some residual blocks;

[0013] In addition, the pooling layer uses a variety of pooling techniques;

[0014] Step 3: Implement the search strategy: In the neural network architecture search process, adopt a one-shot neural network architecture search architecture and a bagging-based ensemble learning method; including the following sub-steps:

[0015] Step 31, initialize the neural network architecture: search for all combinations of residual blocks and pooling layers in the search space, and combine the initial convolutional layer and the fully connected layer to form a convolutional neural network model similar to ResNet50, and then integrate the global lesion localization module and the intra-class center learning module to form a sub-architecture; then all sub-architectures will be embedded into a super network;

[0016] Step 32: Divide the training data set: Generate several sub-datasets D by Bootstrap sampling of the training data set i ;

[0017] Step 33: Train the super network using one-shot architecture search: In each training iteration, randomly select a sub-architecture A from the super network. i , and take a sub-dataset D i Training; then, using the loss function L(A i (θ i ),D i) to update the shared weights θ of the super network i ;

[0018] The training goal is to minimize the loss function of the super network, which is:

[0019]

[0020] Among them, L(A i (θ i ),D i ) is the loss function of the i-th sub-architecture, A i represents the i-th sub-architecture, θ i The corresponding parameters;

[0021] Step 34: Sub-architecture selection and training based on the Bagging method: After the super network training is completed, different sub-architectures are selected for independent training using the Bagging method;

[0022] Step 4: Evaluate sub-architecture performance and diversity: During the search process, the classification performance of each sub-architecture is evaluated in real time;

[0023] Step 5: Sub-architecture integration: Integrate multiple sub-architectures with higher performance in the sub-architecture group.

[0024] The beneficial effect of the present invention is that the present invention realizes the construction of a more efficient and robust intelligent auxiliary diagnosis model for skin cancer by introducing a diversity-driven integration method in the neural network integrated architecture search. Compared with the traditional single architecture design method, the present invention significantly improves the classification performance and generalization ability of the model on diversified and complex data sets, and overcomes the shortcomings of traditional methods in dealing with data complexity and category imbalance. First, in terms of efficiency, the present invention greatly reduces the time and manpower required for model design and optimization through automated neural network integrated architecture search. Compared with the manual tuning method, the present invention can explore the architecture space more systematically and quickly find the best model combination. Secondly, in terms of performance, the multi-architecture combination based on the Bagging integration strategy significantly enhances the classification accuracy and robustness of the model, especially in dealing with the common category imbalance and difficult sample problems in skin cancer data sets. Through the diversity evaluation mechanism, the complementarity of each sub-architecture in the classification task is ensured, thereby effectively improving the overall performance of the final model. This technological breakthrough will provide clinicians with more accurate auxiliary diagnosis tools, helping to comprehensively improve the early diagnosis and treatment effects of skin cancer. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is the overall flow chart of the present invention;

[0026] Figure 2It is a schematic diagram of the search space design of step 2 of the present invention;

[0027] Figure 3 It is a search strategy flow chart of step 3 of the present invention. DETAILED DESCRIPTION

[0028] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0029] like Figure 1 As shown, a skin cancer diagnosis model training method based on neural network integrated architecture search of the present invention comprises the following steps:

[0030] Step 1: Collect a dermoscopic image dataset and annotate each image in the dataset, including the lesion type and lesion area, to form a labeled dermoscopic image dataset;

[0031] First, the existing ISIC (International Skin Imaging Collaboration) dataset is used as the main data source. The ISIC dataset is the world's largest public dermoscopic image database, covering a variety of skin lesions, including melanoma, basal cell carcinoma, squamous cell carcinoma, etc. This dataset has been professionally annotated and can provide reliable data support for the initial development and verification of the model. In order to ensure the diversity and applicability of the dataset, it is recommended to combine the ISIC dataset with newly collected images. The following are the specific implementation details:

[0032] Step 11. Use the ISIC dataset and selectively supplement it: Select appropriate images from the ISIC dataset to ensure that all required skin lesion types are covered. These images have been professionally annotated and can be directly used for model training and testing. To further enhance the diversity and representativeness of the dataset, some images of lesion types or population characteristics that are not covered in the ISIC dataset can be supplemented according to specific needs. At this time, it is necessary to ensure that the newly collected images are consistent with the ISIC dataset in terms of annotation standards.

[0033] Step 12: Professional collection and annotation: In image collection, use a high-resolution dermatoscope to ensure image clarity and detail. Use professional annotation tools such as Labelbox to accurately annotate images, ensuring that the annotation standards are consistent with the ISIC dataset. The annotation content includes the boundaries of the lesion area, lesion type labels, and other key features. After integration, all images will be included in a unified dataset to ensure the consistency and integrity of the dataset.

[0034] Step 13: Dataset division and data enhancement: The integrated dataset is divided into a training set and a validation set in a ratio of 70:30. In addition, data enhancement processing is performed on the dataset, such as random rotation, horizontal and vertical flipping, and scaling, to enrich the diversity of the dataset and further improve the robustness of the model.

[0035] Step 2: Design the search space: Use the ResNet50 convolutional neural network as the basic structure, and combine it with the global lesion localization module based on the Class Activation Mapping (CAM) and the intra-class center learning module based on global features; build a comprehensive architecture search space by expanding and changing the basic components in the basic model. The specific process is as follows: Figure 2 As shown, it includes the following sub-steps:

[0036] Step 21. Build the ResNet50 basic model: ResNet50 is used as the basic convolutional neural network model. ResNet50 includes an initial convolutional layer, four residual blocks, a global average pooling layer and a fully connected layer connected in sequence. Each residual block is composed of multiple convolutional layers connected by residuals. The initial convolutional layer uses a 7x7 convolution kernel for feature extraction. Batch normalization and ReLU activation function are applied after the initial convolutional layer, and then the data is input into the maximum pooling layer. The maximum pooling layer is used to quickly reduce the size of the feature map so that the subsequent residual blocks can perform convolution operations on smaller feature maps. Each residual block includes three convolutional layers, of which the first and third convolutional layers use 1x1 and convolution kernels to ensure that the input and output dimensions of the residual block match. The second convolutional layer uses a standard 3x3 convolution kernel for feature extraction. A global average pooling layer is added after the last residual block to convert the feature map into a fixed-size feature vector to provide input for subsequent classification tasks.

[0037] Step 22, integrate the global lesion localization module and the intra-class center learning module: add a CAM-based global lesion localization module after the last convolutional layer of ResNet50 (i.e., the last convolutional layer of the last residual block). This module uses the CAM generated for different classification tasks in the dermoscopic image to highlight the image area that the model pays most attention to in the classification task, thereby further enhancing the learning effect of the inter-class discriminant feature. Add the intra-class center learning module based on global features after the pooling layer; by minimizing the intra-class distance between the sample features and the class center, the discreteness of the intra-class features is reduced, and the samples of the same category are concentrated in the feature space, thereby improving the discriminative ability of the model. Finally, a comprehensive loss function is constructed by combining cross entropy loss, center loss, and symmetric KL-Divergence Loss (SKL), and the training effect of the model is improved by multi-objective optimization. The specific principles and implementation methods of the CAM-based global lesion localization module and the intra-class center learning module refer to the following paper: Intra-class consistency and inter-class discrimination feature learning for automatic skin lesion classification.

[0038] Step 23, expansion and optimization of search space: First, expand the network's expressive power by diversifying the types of convolutional layers, use convolutional layers of different numbers in the residual block, introduce convolution kernels of different sizes (such as 1x1, 3x3, 5x5, etc.) in the convolutional layer, and use different convolution methods such as depthwise separable convolution (DSC) and standard convolution to form a variety of possible convolution combinations; the depthwise separable convolution and standard convolution operations are decomposed into two steps, which can reduce the amount of calculation and parameters.

[0039] Secondly, in each residual block, a variety of different module connection methods are used, including dense connection, skip connection, and multi-branch connection (MBC), to further enrich the structural diversity of the residual block, thereby enhancing the flexibility of the network. In order to improve the model's attention to key features, attention mechanism modules can be added to some residual blocks, such as Squeeze-and-Excitation Module (SE) or Convolutional Block Attention Module (CBAM), to dynamically adjust feature weights from two dimensions: channel and space.

[0040] In addition, the pooling layer can also use a variety of pooling techniques such as global average pooling (GAP), global maximum pooling (GMP), local pooling, and dynamic pooling to further enhance the model's ability to represent multi-scale features.

[0041] These expanded search space designs will provide diverse choices during the architecture search process, ensuring that the final selected network structure has stronger feature extraction and classification capabilities.

[0042] Step 3: Implement the search strategy: In the neural network architecture search process, adopt a one-shot neural network architecture search architecture and an integrated learning method based on Bagging (Bootstrap Aggregating); when training the parameters of each sub-architecture, try to use different sub-data sets to ensure that each sub-architecture has strong diversity.

[0043] Step 3: Specific process as follows Figure 3 As shown, it includes the following sub-steps:

[0044] Step 31, initialize the neural network architecture: In the search space for the skin cancer image classification task, search for all combinations of residual blocks and pooling layers, and combine the initial convolution layer and the fully connected layer to form a convolutional neural network model similar to ResNet50, and then integrate the global lesion localization module and the intra-class center learning module to form a sub-architecture; in each sub-architecture, the residual block can use different numbers of convolutional layers, different sizes of convolutional kernels, different connection methods for the convolutional layers in the residual block, and can also choose to add attention mechanisms to some residual blocks; the pooling layer of the sub-architecture can also choose a variety of different pooling methods. Then all sub-architectures will be embedded in a super network; each layer in the super network contains different candidate operations, such as convolutional kernels of different sizes, partial or full addition of attention mechanisms in residual blocks, different types of pooling layers, etc. Each sub-architecture is formed by selecting a specific path from these candidate operations; in the super network, all sub-architectures share the same weight parameters, which means that the weights updated in one training will be shared by all sub-architectures, greatly reducing the training time and computing resource consumption.

[0045] Step 32: Divide the training data set: Generate several sub-data sets D from the training data set D through Bootstrap sampling i , each sub-dataset D iIt is a new sample set formed by randomly extracting samples from the original data set with replacement, ensuring that the samples in each sub-dataset are independent. Since the extracted samples will be put back into the original training data set to join the next sample extraction after being put into the sub-dataset, the samples may be repeatedly extracted, and there are certain differences between the sub-datasets. In order to ensure that each sub-dataset has a sufficient number of samples so that a meaningful model can be trained, the size of each sub-dataset is usually the same as the original dataset. Dataset D i Defined as:

[0046] D i ={(x j ,y j )|(x j ,y j )∈D,j∈S i}

[0047] Among them, D i represents the i-th sub-dataset, D is the training data set; S i is the index set of the i-th sampling, which records the index of the samples randomly selected from the training data set D during the i-th Bootstrap sampling process.

[0048] Step 33: Train the super network using one-shot architecture search: Use one-shot architecture search to train a super network that contains all possible sub-architectures. Use randomly initialized weights to build the super network. During the training process, the super network will share weight parameters so that subsequent sub-architectures can directly inherit these shared parameters, thereby reducing training time and resource consumption. In each training iteration, a sub-architecture A is randomly selected from the super network. i , and take a sub-dataset D i Training; then, using the loss function L(A i (θ i ),D i ) to update the shared weights θ of the super network i ; The training goal is to minimize the loss function of the super network; when the performance of the super network reaches the predetermined standard on the validation set or the number of training iterations reaches the upper limit, stop training;

[0049] The loss function of the super network is:

[0050]

[0051] Among them, L(A i (θ i ),D i ) is the loss function of the i-th sub-architecture, A i represents the i-th sub-architecture, θi The corresponding parameters;

[0052] Step 34, sub-architecture selection and training based on the bagging method: After the super network training is completed, different sub-architectures are selected for independent training using the bagging method; when each sub-architecture is selected, it will inherit the corresponding shared parameters in the super network, and then during the fine-tuning process on the corresponding sub-dataset, the parameters are further optimized using the stochastic gradient descent (SGD) method to minimize the loss function of the sub-architecture on the sub-dataset; the process of optimizing parameters is expressed as:

[0053]

[0054] Among them, τ is the learning rate, is the updated parameter.

[0055] Step 4: Evaluate sub-architecture performance and diversity: During the search process, evaluate the classification performance of each sub-architecture in real time, such as accuracy (ACC), area under the curve (AUC), and diversity (differences in architecture encoding, differences in prediction vector logits), to ensure that each sub-architecture has a certain degree of diversity while maintaining high classification performance; the specific implementation method is as follows:

[0056] Step 41: Real-time evaluation of sub-architecture classification performance: During the search process, the classification performance of each sub-architecture needs to be evaluated in real time after each training; the ACC and AUC of each sub-architecture are verified using the validation set. The performance data of each sub-architecture should be recorded and compared with the predetermined performance threshold, and compared with the predetermined thresholds respectively, and the sub-architectures whose ACC and / or AUC do not reach the threshold are eliminated; this process ensures that the selected sub-architectures are competitive in performance.

[0057] Step 42: Evaluate the diversity of sub-architectures: Calculate the diversity of sub-architecture encodings and the diversity of predicted logits respectively;

[0058] Calculate the difference in architecture encoding: In the search space, represent each sub-architecture as a fixed-length binary or discrete numerical vector. Assuming that the encoding length of the sub-architecture is n, the encoding of each sub-architecture is represented as: A i =(a i1 ,a i2 ,…,a in );where a ijRepresents the encoding value of the jth position of sub-architecture i; the length and specific value of each code depends on the search space of the design and the components of the sub-architecture, and the encoding length is the sum of the number of encoding bits required for all design options. Design options refer to parameters or components that need to be optimized or selected during the network architecture search process. They cover a variety of factors that determine the network structure and performance, from the number of network layers, convolution kernel size, activation function type to network width. The number of encoding bits represents the number of bits of the binary or discrete value of an option. For example: the number of convolutional layers can be 1, 2, 3 or 4 layers, which requires 2 bits of encoding, represented by 00, 01, 10, 11 respectively.

[0059] The difference in architecture encoding is defined as 1 minus the similarity. The Hamming distance (HD) is used to calculate the number of different encoding values ​​at the same position of the two sub-architectures. The specific method is to compare the encodings of the two sub-architectures bit by bit. If the encoding value of a certain position is different, the counter is increased by 1, and finally the number of bits of the different encoding values ​​is counted. For example, if the number of convolutional layers of one sub-architecture is 3 and the other is 4, then (10) and (11) are compared bit by bit. If the first 1 and 1 are the same, and the second 1 and 0 are different, then there is 1 bit difference. The similarity is calculated by the normalized Hamming distance, which is 1 minus the ratio of the Hamming distance to the encoding length; the difference is defined as 1 minus the similarity; during the architecture search process, the difference between each sub-architecture and all other sub-architectures is calculated in real time. After calculating the difference between each sub-architecture and other sub-architectures, if the difference between a sub-architecture and any other sub-architecture is lower than the threshold, the sub-architecture (the sub-architecture currently being evaluated) is eliminated; ensure that the difference between sub-architectures remains above a certain threshold to ensure diversity, avoid generating overly similar sub-architectures, and ultimately improve the generalization and robustness of the model.

[0060] Compute the differences in architecture prediction logits: In each training iteration, for each sub-architecture A i , input the validation set into the model to obtain the corresponding predicted logits vector. These logits refer to the model output values ​​without activation function, which are used to represent the network's confidence in each category. i and A j (Each sub-architecture is compared with all other sub-architectures) and the KL divergence (Kullback-Leibler Divergence, KLD) between the logits vectors predicted on the same validation set is calculated, which can be used to measure the similarity of two probability distributions. If the difference value calculated by KL divergence is large, it means that sub-architecture A i and A jIf there is a significant difference in the logits distribution of , it means that the two sub-architectures may have made different judgments on the same input data, and therefore have complementarity. On the contrary, if the difference value is small, it means that the judgment results of the two sub-architectures are similar and the information redundancy is high. For each pair of sub-architectures, calculate the KL divergence and record the results. In all comparisons, select the sub-architecture pairs with KL divergence higher than the threshold and keep them as candidate architectures.

[0061] Step 43: Comprehensive performance and diversity score evaluation: Calculate the comprehensive score for each sub-architecture by weighted sum to strike a balance between classification performance and diversity:

[0062] Score i =w1ACC i +w2AUC i +w3Diversity encoding,i +w4Diversity logits,i

[0063] Among them, ACC i It is sub-structure A i The ACC value and AUC values ​​obtained for the validation set i Substructure A i The AUC value obtained for the validation set, Diversity encoding,i and Diversity logits,i Sub-structure A i The average value of the encoding diversity and the average value of the predicted logits diversity in the entire architecture set; w1 to w4 are the weights of each indicator, which can be dynamically adjusted according to actual needs to balance the priorities of performance and diversity.

[0064] Step 5, sub-architecture integration: After the search is completed, multiple sub-architectures with higher performance in the sub-architecture group are integrated to improve the final classification performance of the skin cancer intelligent auxiliary diagnosis model; including the following steps:

[0065] Step 51, sub-architecture selection and integration: When selecting sub-architectures, give priority to those sub-architectures that perform well in classification performance (ACC and AUC), while ensuring that they have high diversity scores in encoding and predicting logits, and select Score i Sub-architectures larger than a preset threshold;

[0066] Step 52, design of sub-architecture integration strategy: multiple integration strategies are used to integrate the selected sub-architectures, including simple weighted average, voting and stacking. In simple weighted average, the output results of each sub-architecture are weighted averaged according to their comprehensive scores to obtain the final prediction result. The voting rule is to determine the final classification by majority voting based on the independent prediction results of each sub-architecture. In the stacking method, multiple base learners (i.e., sub-architectures) are first trained, and then the prediction results of these base learners are secondary learned through a meta-learner (logistic regression model or simple neural network) to optimally combine the prediction results of each base learner, thereby further improving the classification accuracy.

[0067] Step 53: Integration weight adjustment: In the initial stage of integration, an initial weight is assigned to each sub-architecture; this initial weight can be adjusted based on the comprehensive score of each sub-architecture. i Then, we enter the iterative adjustment phase, evaluate the overall performance of the integrated model on the validation set, and gradually adjust the weights of the sub-architectures so that the final integrated model achieves the best balance in various indicators, that is, minimizes the value of the following loss function:

[0068] L=-(w′1ACC+w′2AUC+w′3Diversity encoding +w′4Diversity logits )

[0069] ACC, AUC, Diversity encoding 、Diversity logits are the ACC, AUC, average of encoding differences and average of predicted logits differences of the integrated model on the validation set. The results obtained by each sub-architecture in the integrated model are integrated through the integration strategy; w′1, w′2, w′3, and w′4 are the corresponding weights respectively;

[0070] During training, the gradient descent method is used to optimize the loss function and dynamically adjust the weights of each sub-architecture:

[0071]

[0072] Where η is the learning rate, is the weight after update;

[0073] When the model performance evaluated on the validation set no longer improves significantly in several consecutive iterations, that is, the loss function value no longer decreases significantly, stop weight adjustment and get the final weight distribution plan;

[0074] Step 54: Generate the final ensemble model: After completing the performance evaluation on the validation set, comprehensively consider the performance of each ensemble strategy, select the ensemble strategy with the best performance, and then build the final ensemble model based on the selected sub-architecture and the optimized weight distribution scheme. This ensemble model shows the best classification performance on the validation set and has strong generalization ability. At the same time, consider the advantages and disadvantages of multiple strategies to ensure the robustness and reliability of the ensemble model in practical applications.

[0075] The sub-architecture integration strategy is shown in the following table:

[0076]

[0077] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A skin cancer diagnosis model training method based on neural network integrated architecture search, characterized in that: The steps include: Step 1: Collect a dermoscopic image dataset and annotate each image in the dataset, including the lesion type and lesion area, to form a labeled dermoscopic image dataset, which is divided into a training set and a validation set; Step 2: Design the search space: Use the ResNet50 convolutional neural network as the basic structure, and combine it with the CAM-based global lesion localization module and the global feature-based intra-class center learning module; build a comprehensive architecture search space by expanding and changing the basic components in the basic model; including the following sub-steps: Step 21: Build the ResNet50 basic model; Step 22: Integrate the global lesion localization module and the intra-class center learning module: Add a CAM-based global lesion localization module after the last convolutional layer of ResNet50, and add an intra-class center learning module based on global features after the pooling layer; Step 23: Expansion and optimization of search space: First, expand the network's expressive power by diversifying the types of convolutional layers: use convolutional layers of different numbers in the residual block, introduce convolutional kernels of different sizes in the convolutional layer, and use different convolution methods to form a variety of convolution combinations; Secondly, in each residual block, a variety of different module connection methods are used; attention mechanism modules are added to some residual blocks; In addition, the pooling layer uses a variety of pooling techniques; Step 3: Implement the search strategy: In the neural network architecture search process, adopt a one-shot neural network architecture search architecture and a bagging-based ensemble learning method; including the following sub-steps: Step 31, initialize the neural network architecture: search for all combinations of residual blocks and pooling layers in the search space, and combine the initial convolutional layer and the fully connected layer to form a convolutional neural network model similar to ResNet50, and then integrate the global lesion localization module and the intra-class center learning module to form a sub-architecture; then embed all sub-architectures into a super network; Step 32: Divide the training data set: Generate several sub-datasets D by Bootstrap sampling of the training data set i ; Step 33: Train the super network using one-shot architecture search: In each training iteration, randomly select a sub-architecture A from the super network. i , and take a sub-dataset D i Training; then, using the loss function L(A i (θ i ),D i ) to update the shared weights θ of the super network i ; The training goal is to minimize the loss function of the super network, which is: Among them, L(A i (θ i ),D i ) is the loss function of the i-th sub-architecture, A i represents the i-th sub-architecture, θ i The corresponding parameters; Step 34: Sub-architecture selection and training based on the Bagging method: After the super network training is completed, the Bagging method is used to select different sub-architectures for independent training; Step 4: Evaluate sub-architecture performance and diversity: During the search process, the classification performance of each sub-architecture is evaluated in real time; Step 5: Sub-architecture integration: Integrate multiple sub-architectures with higher performance in the sub-architecture group.

2. The method for training a skin cancer diagnosis model based on neural network integrated architecture search according to claim 1, characterized in that: The specific implementation method of step 4 is: Step 41, real-time evaluation of sub-architecture classification performance: During the search process, the classification performance of each sub-architecture needs to be evaluated in real time after each training; the ACC and AUC of each sub-architecture are verified using the validation set and compared with the predetermined performance thresholds, and the sub-architectures whose ACC and / or AUC do not reach the thresholds are eliminated; Step 42: Evaluate the diversity of sub-architectures: Calculate the diversity of sub-architecture encodings and the diversity of predicted logits respectively; Step 43: Comprehensive performance and diversity score evaluation: Calculate the comprehensive score for each sub-architecture by weighted sum: Score i =w1ACC i +w2AUC i +w3Diversity encoding,i +w4Diversity logits,i Among them, ACC i It is sub-structure A i The ACC value and AUC values ​​obtained for the validation set i Substructure A i The AUC value obtained for the validation set, Diversity encoding,i and Diversity logits,i Sub-structure A i The average of the encoded variance and the average of the predicted logits variance across the entire set of architectures; w1 to w4 are the weights of each metric.

3. The method for training a skin cancer diagnosis model based on neural network integrated architecture search according to claim 1, characterized in that: The specific implementation method of step 5 is: Step 51, Sub-architecture selection and integration: When selecting a sub-architecture, select Score i Sub-architectures larger than a preset threshold; Step 52, designing the integration strategy of the sub-architecture: using a variety of integration strategies to integrate the selected sub-architectures, including simple weighted average method, voting method and stacking method; Step 53: Adjust the ensemble weights: In the initial stage of the ensemble, an initial weight is assigned to each sub-architecture; then the iterative adjustment stage is entered to evaluate the overall performance of the ensemble model on the validation set, and the weights of the sub-architectures are gradually adjusted so that the final ensemble model achieves the best balance in terms of various indicators, that is, the value of the following loss function is minimized: L=-(w′1ACC+w′2AUC+w′3Diversity encoding +w′4Diversity logits ) ACC, AUC, Diversity encoding 、Diversity logits They are the ACC, AUC, average of encoding differences, and average of predicted logits differences of the integrated model on the validation set. The results obtained by each sub-architecture in the integrated model are integrated through the integration strategy; w′1, w′2, w′3, and w′4 are the corresponding weights respectively; During training, the gradient descent method is used to optimize the loss function and dynamically adjust the weights of each sub-architecture: Where η is the learning rate, is the weight after update; When the model performance evaluated on the validation set no longer improves significantly in several consecutive iterations, that is, the loss function value no longer decreases significantly; stop Adjust the weights to get the final weight distribution plan; Step 54: Generate the final ensemble model: After completing the performance evaluation on the validation set, comprehensively consider the performance of each ensemble strategy, select the ensemble strategy with the best performance, and then build the final ensemble model based on the selected sub-architecture and the optimized weight distribution scheme.

Images