NAC curative effect prediction method and device based on multi-modal feature fusion and river horse optimization

Abstract

The invention relates to an NAC curative effect prediction method and device based on multi-modal feature fusion and Heman optimization, and the method comprises the steps: firstly, extracting radiomics features, including texture, morphology, first-order statistics, wavelet and other features, from ultrasonic images of three modalities of two-dimensional gray scale, elasticity and color Doppler blood flow imaging, splicing the feature vectors, and carrying out the fusion of the feature vectors; a high-dimensional feature vector is formed, and lesion information is comprehensively represented. Then, an improved Hemma optimization algorithm is adopted to carry out dimension reduction on the features, and penalty factors and kernel parameters of a support vector machine are optimized. According to the algorithm, chaotic mapping is introduced, so that the global search capability is enhanced, local optimum is avoided, and the optimal feature subset and SVM parameters are screened out. Finally, an SVM model is trained based on the optimized features and parameters, an independent test set is used for model evaluation, and compared with various existing optimization algorithms, the advantages of the improved HO algorithm in the aspect of improving breast cancer NAC curative effect prediction accuracy are verified.

Description

Technical Field

[0001] The present invention relates to the fields of medical image processing and deep learning, and particularly to a method and device for predicting the efficacy of NAC based on multi-modal feature fusion and hippopotamus optimization. Background Art

[0002] Neoadjuvant chemotherapy (NAC) has become the first-line treatment for patients with locally advanced breast cancer. However, the responses of different patients to NAC vary greatly. Approximately 19%-30% of breast cancer patients will achieve pathological complete remission (pCR) after NAC, and approximately 5%-20% of patients will experience disease progression. Currently, the imaging methods commonly used to evaluate the efficacy of breast cancer NAC include MRI, PET / CT, and ultrasound. However, the interpretation of imaging images mainly relies on the subjective visual assessment of doctors, and there is still a lack of an evaluation system with objective criteria. At the same time, due to the differences in the training experiences and experience accumulations of clinical doctors, there are also significant differences in the diagnostic and evaluation abilities. Currently, there is still no standard method or biomarker in clinical practice that can be used to accurately predict whether breast cancer patients can achieve pCR after NAC.

[0003] Radiomics extracts a large number of quantitative features from medical images, providing a new tool for clinical decision-making. However, due to the high dimensionality of the radiomics features of multi-modal images, directly using them in machine learning models will lead to problems such as high computational complexity and poor model generalization ability.

[0004] Traditional feature selection methods usually show certain limitations when dealing with high-dimensional data, especially when searching for the global optimal feature subset, they are prone to falling into local optima.

[0005] Support Vector Machine (SVM) is a classification algorithm based on statistical learning theory, which mainly distinguishes data points of different categories by finding the optimal hyperplane in a high-dimensional space. How to effectively optimize the SVM model parameters has become the key to improving the SVM classification performance.

[0006] Currently, there are many optimization algorithms combined with SVM for classification tasks. These combined algorithms have achieved certain results in improving the classification accuracy, but there is still room for further optimization when dealing with high-dimensional feature data. Summary of the Invention

[0007] The purpose of the present invention is to overcome the defects of the above-mentioned existing technologies and provide a method and device for predicting the efficacy of NAC based on multi-modal feature fusion and hippopotamus optimization, so as to solve or partially solve the problems of unsatisfactory accuracy of NAC efficacy prediction, inability to effectively perform dimensionality reduction on high-dimensional data, and being prone to falling into local optima during the dimensionality reduction process.

[0008] The object of the present invention can be achieved by the following technical solutions:

[0009] In one aspect of the present invention, a method for predicting the efficacy of NAC based on multi-modal feature fusion and hippopotamus optimization is provided. The high-dimensional feature vector of the object to be predicted is used as the input of the trained SVM model to obtain the prediction result of whether the pathology is completely relieved after NAC treatment, so as to realize the prediction of the efficacy of NAC for non-treatment purposes. Among them, the training process of the SVM model includes the following steps:

[0010] Obtain multi-modal ultrasound images;

[0011] Perform region of interest annotation and image preprocessing on the multi-modal ultrasound images;

[0012] Extract the radiomics features corresponding to the ultrasound images in each modality respectively, and obtain a high-dimensional feature vector through splicing;

[0013] Based on the hippopotamus algorithm, optimize the parameters of the SVM model through chaotic mapping, and reduce the dimension of the high-dimensional feature vector to obtain the optimal feature subset;

[0014] Based on the parameters of the SVM model and the optimal feature subset, use the pre-constructed training data set to train the SVM model to obtain the trained SVM model.

[0015] As a preferred technical solution, the region of interest annotation includes the following steps:

[0016] For the ultrasound images in each modality, perform annotation on the premise of ensuring that the annotated area covers the same lesion site, and ensure the consistency of the region of interest annotation among the modality images.

[0017] As a preferred technical solution, the image preprocessing includes gray value normalization, image size adjustment, denoising and data augmentation processing.

[0018] As a preferred technical solution, the process of obtaining the radiomics features and the high-dimensional feature vector includes the following steps:

[0019] For the ultrasound image of each modality, extract multiple types of features to form the radiomics features corresponding to the modality;

[0020] For the radiomics features corresponding to each modality, obtain a high-dimensional feature vector through splicing in the vector dimension.

[0021] As a preferred technical solution, the process of optimizing the parameters of the SVM model through chaotic mapping based on the hippopotamus algorithm and reducing the dimension of the high-dimensional feature vector to obtain the optimal feature subset includes the following steps:

[0022] Initialize the population of the hippopotamus algorithm based on chaotic mapping. Each individual in the population includes a feature selection vector and the SVM model parameters;

[0023] Calculate the fitness function value based on the classification accuracy of the feature subset and the complexity of the SVM model parameters;

[0024] Based on the calculated fitness function value, update the individual positions in multiple stages through chaotic mapping;

[0025] Repeat the calculation of the fitness function value and the update of the individual positions to achieve multiple iterations until the preset convergence condition is reached, and output the parameters of the optimal SVM model and the feature subset.

[0026] As a preferred technical solution, the fitness function value is calculated using the following formula:

[0027]

[0028] where, Fitness(X i ) represents the fitness function value when the population is X i , Acc cv (X i ) represents the classification accuracy of the feature subset S i in cross-validation, λ 1 is the penalty coefficient for controlling the number of features, λ 2 is the penalty coefficient for controlling the complexity of the SVM parameters, Complexity(C i , γ i ) is the complexity of the penalty factors C i and the kernel parameter γ i , the penalty factors C i and the kernel parameter γ i are initialized as C i = C min + r c ×(C max - C min ), γ i = γ min + r γ ×(γ max - γ min ), r c and r γ are random numbers generated by chaotic mapping, C min , C max , γ min , γ max are the minimum and maximum values of the penalty factors and the kernel parameters respectively.

[0029] As a preferred technical solution, based on the calculated fitness function value, the multi-stage individual position update through chaotic mapping includes:

[0030] In the first stage, the position of the individual is updated using the perturbation generated by chaotic mapping, including the values of the feature selection vector, penalty factor, and kernel parameter:

[0031]

[0032] Among them, are respectively the feature selection vector, penalty factor, and kernel parameter of the i-th individual in the t-th iteration, S best , C best , γ best are respectively the feature selection vector, penalty factor, and kernel parameter of the optimal individual in the current population, and chaos is the perturbation factor obtained by iterating the Logistic mapping formula;

[0033] In the second stage, the position of the individual is updated for potential local optimal traps:

[0034]

[0035] P j = lb j + r s × (ub j - lb j )

[0036] Among them, P j is the randomly generated predator position, representing the threat source, lb j and ub j are respectively the lower and upper bounds of the search space, and r s is a random number;

[0037] In the third stage, compared with the second stage, a greater adjustment is made for the threat of severe local optimal traps, and the position of the individual is finely adjusted through local search to obtain the optimal solution:

[0038]

[0039] Among them, lb localj and ub localj are the lower and upper bounds of the local search space, and s 1 is a random number used to adjust the amplitude of position update.

[0040] As a preferred technical solution, the multi-modal ultrasonic images include two-dimensional grayscale, elastography, and blood flow ultrasonic images.

[0041] As a preferred technical solution, based on the parameters of the SVM model and the optimal feature subset, the process of training the SVM model using a pre-constructed training dataset to obtain a trained SVM model includes the following steps:

[0042] Based on the optimal feature subset, screen the corresponding features and construct a training dataset;

[0043] Based on the parameters of the SVM model, initialize the SVM model;

[0044] Use the training dataset to train the SVM model;

[0045] Evaluate the performance of the trained SVM model through multi-fold cross-validation.

[0046] Another aspect of the present invention provides a device for predicting the efficacy of NAC based on multi-modal feature fusion and hippopotamus optimization, including:

[0047] An ultrasonic image region of interest annotation and preprocessing module, which is used to obtain multi-modal ultrasonic images and perform region of interest annotation and image preprocessing;

[0048] A feature extraction and splicing module, which is used to extract the radiomics features corresponding to the ultrasonic images in each modality respectively, and obtain a high-dimensional feature vector through splicing;

[0049] An SVM model parameter and feature subset iterative optimization module, which is used to optimize the parameters of the SVM model through chaotic mapping based on the hippopotamus algorithm, and reduce the dimension of the high-dimensional feature vector to obtain an optimal feature subset;

[0050] An SVM model training module, which is used to train the SVM model using a pre-constructed training dataset based on the parameters of the SVM model and the optimal feature subset to obtain a trained SVM model;

[0051] An NAC efficacy prediction module, which is used to use the high-dimensional feature vector of the object to be predicted as the input of the trained SVM model to obtain the prediction result of whether the pathology is completely relieved after NAC treatment, and realize the efficacy prediction of NAC.

[0052] Compared with the prior art, the present invention has at least one of the following beneficial effects:

[0053] (1) Improve the prediction effect of NAC efficacy: First, the present invention obtains multi-modal ultrasound images, performs region-of-interest annotation and image preprocessing, and then extracts the radiomics features corresponding to the ultrasound images in each modality respectively, and obtains a high-dimensional feature vector through splicing. After that, based on the hippopotamus algorithm, the parameters of the SVM model are optimized through chaotic mapping, and the high-dimensional feature vector is dimensionally reduced to obtain an optimal feature subset. Finally, the SVM model is trained using a pre-constructed training data set to obtain a trained SVM model, realizing the prediction of NAC efficacy. By introducing chaotic mapping and an improved hippopotamus optimization algorithm, the model performs excellently in feature selection and SVM parameter optimization. The present invention significantly improves the accuracy and AUC of the breast cancer NAC efficacy prediction model, and can more accurately predict the chemotherapy effect.

[0054] (2) Reduce the possibility of falling into local optimum: The introduction of chaotic mapping effectively enhances the global search ability of the hippopotamus algorithm, reduces the risk of falling into local optimum, enables the model to explore potential solutions more widely in the high-dimensional feature space, thereby finding a better feature subset and SVM parameter combination, effectively reducing redundant features, improving the generalization ability of the model, reducing the overfitting phenomenon, not only enhancing the robustness of the model, but also reducing the computational complexity.

[0055] (3) Improve the performance of the SVM classifier: By simultaneously optimizing the penalty factor and kernel parameters of the SVM, the performance of the SVM classifier is further improved. The optimized SVM model performs more excellently when dealing with complex multi-modal image data, and has higher classification accuracy and stability. Description of the Drawings

[0056] Figure 1 It is a flowchart of the NAC efficacy prediction method based on multi-modal feature fusion and hippopotamus optimization in the embodiment;

[0057] Figure 2 It is a flowchart of the ultrasound image preprocessing in the embodiment;

[0058] Figure 3 It is a flowchart of extracting multi-modal features and high-throughput features in the embodiment;

[0059] Figure 4 It is a flowchart of the hippopotamus algorithm in the embodiment;

[0060] Figure 5 It is a schematic diagram of the receiver operating characteristic curve for the early prediction of breast cancer NAC efficacy by different algorithms in the embodiment;

[0061] Figure 6 It is a schematic diagram of the NAC efficacy prediction device based on multi-modal feature fusion and hippopotamus optimization in the embodiment;

[0062] Figure 7 Schematic diagram of the electronic device in the embodiment. Specific implementation manners

[0063] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0064] Embodiment 1

[0065] In view of the problems existing in the foregoing prior art, this embodiment provides a method for predicting the efficacy of NAC based on multi-modal feature fusion and hippopotamus optimization, which uses the hippopotamus optimization algorithm (HO) combined with the support vector machine (SVM) for feature dimensionality reduction and classification, and introduces a chaotic map to enhance the global search ability of the algorithm. By comprehensively using multi-modal (including GUSI, CDFI, and EI) ultrasound images before chemotherapy, the efficacy of breast cancer NAC is evaluated from multiple angles at an early stage.

[0066] See Figure 1 , this method includes the following steps:

[0067] Step S1, retrospectively collect high-quality multi-modal ultrasound images of breast cancer patients undergoing NAC before chemotherapy. Among them, the multi-modal can include (gray-scale image GUSI, blood flow image CDFI, and elastogram EI)

[0068] Step S2, preprocess the ultrasound images. This process includes manually annotating the region of interest (ROI) and image preprocessing to improve the image quality and stability and ensure the accuracy of subsequent radiomics feature extraction.

[0069] Step S3, extract the multi-modal features of each object, that is, for each type of image, extract the high-throughput radiomics features respectively. The radiomics features of the three-modal images of the same object are concatenated to obtain the high-dimensional feature vector of each object.

[0070] Step S4, combine the chaotic map to enhance the global search ability, and use the improved HO algorithm for feature dimensionality reduction. While reducing the dimensionality, optimize the kernel parameter and penalty factor of the support vector machine.

[0071] Step S5, use the optimal feature subset selected in Step S4 and the optimized SVM parameters to train the SVM model on the training data set to construct a prediction model for the efficacy of breast cancer NAC.

[0072] Specifically, step S1 may include steps S101 - S103:

[0073] In step S101, multimodal ultrasound images before NAC of 480 subjects pre - diagnosed with breast cancer were retrospectively collected, as well as the actual results of whether pathological complete remission occurred after NAC treatment, constituting 480 samples.

[0074] In step S102, ensure that images of each modality are completely preserved and of good quality. Randomly assign all samples into a training set of 358 cases and a retrospective test set of 122 cases.

[0075] In step S103, a doctor with 15 years of experience in breast ultrasound examination is responsible for controlling the quality of ultrasound images, removing some images containing artifacts, blurs, and non - diseased tissues, and selecting high - quality ultrasound images.

[0076] Specifically, refer to Figure 2 , step S2 may include steps S201 - S205:

[0077] In step S201, ensure the annotation of regions of interest (ROIs) with multimodal consistency: A sonographer with 15 years of experience in breast ultrasound examination manually annotates the ROIs to eliminate irrelevant information such as text and icons.

[0078] For different - modality ultrasound images (GUSI, CDFI, EI) of the same patient, ensure that the annotation of ROIs is consistent among the modality images, that is, try to keep the annotated regions covering the same lesion sites. However, due to differences between modalities, strict image alignment or registration is not required.

[0079] In step S202, gray - level normalization: Considering the differences in different image acquisition conditions or devices, the gray - level value ranges of ultrasound images may vary. Through gray - level normalization, adjust the gray - level values of the images to a unified range (such as 0 - 255) to eliminate the influence of device differences.

[0080] In step S203, image size adjustment: Crop the images to remove irrelevant parts and standardize all images to the same size of 256x256 to ensure the comparability of features of all images. Size normalization is achieved through bicubic interpolation.

[0081] In step S204, denoising processing: Considering the presence of noise in ultrasound images, such as salt - and - pepper noise, use Gaussian filtering for denoising to remove or reduce noise interference in the images. Gaussian filtering is a linear smoothing filtering method that removes noise by weighted averaging of each pixel in the image. The weights are determined by the Gaussian function, with the highest weight at the center point and gradually decreasing as the distance increases.

[0082] Step S205, Data Augmentation: In medical image processing, the training data is usually limited, which may cause the deep learning model to be prone to overfitting during training and difficult to exhibit good generalization ability on new data. Through data augmentation, the dataset can be artificially expanded, the robustness of the model can be increased, and the model can be more capable of dealing with the variability and noise in the images. Through data augmentation, the number of images in the training set is increased to 5 times the original data volume.

[0083] Preferably, the data augmentation process may include:

[0084] (1) Rotation: Rotate the image by a certain angle ([-15°, 15°]) to generate multiple images with different angles. By the rotation operation, the angle changes that may occur during image shooting are simulated, thereby improving the model's recognition ability for rotational invariance. Interpolation filling is performed on the rotated image to maintain the integrity of the image.

[0085] (2) Translation: Set the translation ratio of [-10%, 10%] in the horizontal or vertical direction respectively for translation to generate images at different positions. By the translation operation, the situations where the target area in the image appears at different positions are simulated, enhancing the model's robustness to target position changes. The blank areas of the translated image are processed by zero filling.

[0086] (3) Flipping: Operation description: Flip the image horizontally or vertically to generate mirror images. The flipping operation can increase the diversity of the dataset and make the model more robust to symmetry changes in the images.

[0087] (4) Scaling: Perform a scaling operation on the image according to the ratio of [0.9, 1.1], including magnification or reduction. The scaling operation can simulate the performance of the target area at different scales and help the model learn the features of targets of different sizes.

[0088] (5) Brightness Adjustment: Adjust the brightness of the image according to the adjustment range of [0.8, 1.2] to simulate the image performance under different lighting conditions. The brightness adjustment can enhance the model's robustness under different lighting conditions.

[0089] Specifically, step S3 may include steps S301 - S302:

[0090] Step S301, For the three - modality images of each object, select one image for each modality and perform feature extraction respectively. Each image obtains 866 features.

[0091] Take the two - dimensional gray - scale, elastography, and blood flow images of each patient as inputs respectively to extract radiomics features including but not limited to the following.

[0092] (1) Texture features:

[0093] Gray-Level Co-occurrence Matrix (GLCM) features, including autocorrelation, cluster prominence, cluster shade, cluster tendency, contrast, correlation, difference mean, difference entropy, difference variance, identity, information measure of difference moment, normalized information measure of difference moment, information measure of difference, information measure of correlation, inverse variance, joint mean, joint energy, joint entropy, maximum correlation coefficient, maximum probability, sum mean, sum entropy, sum of squares, etc.

[0094] For example, Gray Level Dependence Matrix (GLDM) features, including dependence entropy, dependence non-uniformity, dependence variance, normalized dependence non-uniformity, gray-level non-uniformity, gray-level variance, high gray-level emphasis, large dependence emphasis, large dependence high gray-level emphasis, large dependence low gray-level emphasis, low gray-level emphasis, small dependence emphasis, small dependence high gray-level emphasis, and small dependence low gray-level emphasis, etc.

[0095] For example, Gray Level Run Length Matrix (GLRLM) features, including gray-level non-uniformity, normalized gray-level non-uniformity, gray-level variance, high gray-level run emphasis, long run emphasis, long run high gray-level emphasis, long run low gray-level emphasis, low gray-level run emphasis, run entropy, run length non-uniformity, normalized run length non-uniformity, run percentage, run variance, short run emphasis, short run high gray-level emphasis, and short run low gray-level emphasis, etc.

[0096] For example, Gray Level Size Zone Matrix (GLSZM) features, including gray-level non-uniformity, normalized gray-level non-uniformity, gray-level variance, high gray-level zone emphasis, large area emphasis, large area high gray-level emphasis, large area low gray-level emphasis, low gray-level zone emphasis, zone size non-uniformity, normalized zone size non-uniformity, small area emphasis, small area high gray-level emphasis, small area low gray-level emphasis, zone entropy, zone percentage, and zone variance, etc.

[0097] For example, Neighboring Gray Tone Difference Matrix (NGTDM) features, including busyness, roughness, complexity, contrast, intensity, etc.

[0098] (2) Morphological features: such as shape, size, edge roughness, etc.

[0099] (3) First-order statistical features: such as mean, percentile, 10th percentile, 90th percentile, energy, entropy, interquartile range, kurtosis, maximum value, mean absolute deviation, median, minimum value, range, robust mean absolute deviation, root mean square, skewness, total energy, uniformity, and variance, etc.

[0100] (4) Wavelet features, based on the wavelet transform of the image, further extract the detailed high-throughput omics features of the lesion area at different frequencies and scales. These features together provide strong support for quantitative analysis and diagnosis.

[0101] (5) For color Doppler flow images and elastograms, extract color distribution features: such as color histogram, color moment, color gradient, etc.

[0102] Combine all the features extracted separately into a feature vector, representing the gray-scale, elasticity, and blood flow image features of the patient.

[0103] Step S302, multi-modal feature splicing.

[0104] Feature vector splicing: Splice the feature vectors extracted from the three-modal images of the same object at the vector level to form a high-dimensional vector with a total of 2598 features. This feature vector contains feature information from different modalities and can more comprehensively reflect all aspects of the lesion.

[0105] Feature matrix construction: Combine the spliced feature vectors of all patients into a feature matrix, where the rows represent the objects and the columns represent the features. This matrix will be used as the input data for subsequent dimensionality reduction and modeling.

[0106] Specifically, refer to Figure 3 , step S4 may include steps S401 - S404:

[0107] In this embodiment, the hippopotamus optimization algorithm simulates the foraging, predator avoidance, etc. behaviors of hippopotamuses in the amphibious environment, and has strong global search ability and excellent adaptability. Compared with other traditional intelligent optimization algorithms, this method shows higher computational efficiency and accuracy when dealing with high-dimensional feature data. In order to improve the classification accuracy and robustness, this method uses the HO algorithm to simultaneously reduce the dimensionality of the radiomics features and optimize the kernel parameters and penalty factors of the SVM, so as to realize the intelligent prediction of the efficacy of breast cancer NAC.

[0108] Step S401, initialize the population of the HO algorithm.

[0109] (1) Population Definition: The HO algorithm is a population-based optimization algorithm, where the search agents are represented by hippos. Each hippo represents a candidate solution to the optimization problem, and the update of its position in the search space corresponds to the values of the decision variables. Therefore, the population of the HO algorithm consists of N hippo individuals, and each individual represents a feature subset selection scheme and the corresponding SVM parameters. Each individual is represented by the vector X i =[S i , C i , γ i , where S i =[s i1 , s i2 , …, s im is the feature selection vector, and m is the dimension of the original features. s ij = 1 indicates that the j-th feature is selected, and s ij = 0 indicates that the feature is not selected. C i is the penalty factor of SVM, and γ i is the RBF kernel function parameter of SVM.

[0110] (2) Introduction of Chaotic Mapping: During the population initialization process, chaotic mapping is used to generate the initial positions of individuals. Chaotic mapping helps the hippo optimization algorithm avoid falling into local optima by generating complex and unpredictable sequences. The specific steps are as follows:

[0111] Select the initial value: A random initial value x 0 (within the interval [0, 1]) is selected for each individual.

[0112] Generate the chaotic sequence: A series of values [x 1 , x 2 , …, x n are generated by iterating the chaotic mapping formula. The chaotic mapping is generated by the Logistic mapping formula as follows:

[0113] x n+1 = r × x n × (1 - x n )

[0114] where r is set to 3.9.

[0115] Map to the search space: The values in the chaotic sequence are mapped to the range of the search space to obtain the initial positions of each individual.

[0116] (3) Population Initialization:

[0117] Initialization of the feature selection vector Si: Based on the sequence generated by chaotic mapping, each dimension of each feature vector is assigned a binary value (0 or 1) to indicate whether the feature is selected.

[0118] Penalty factor C i and kernel parameter γ i Initialization: Randomly generate within a predefined range, and the formula is as follows:

[0119] C i = C min + r c × (C max - C min ),

[0120] γ i = γ min + r γ × (γ max - γ min ),

[0121] where r c and r γ are random numbers generated by the chaotic mapping, and C min , C max , γ min , γ max are the minimum and maximum values of the penalty factor and the kernel parameter respectively.

[0122] Step S402, Definition and optimization of the fitness function:

[0123] Definition of the fitness function: The fitness function is used to evaluate the quality of each individual (i.e., each feature subset and the corresponding SVM parameters), and the specific formula is as follows:

[0124]

[0125] where: Acc cv (X i ) represents the classification accuracy of the feature subset S i in the cross-validation. λ 1 is the penalty coefficient for controlling the number of features, aiming to reduce redundant features. λ 2 is the penalty coefficient for controlling the complexity of the SVM parameters, aiming to prevent overfitting. Complexity(C i , γ i ) reflects the complexity of the penalty factor C i and the kernel parameter γ i , and is defined as their product.

[0126] Fitness evaluation: For each individual X i , according to its corresponding feature subset and SVM parameters, use 5-fold cross-validation to calculate the classification accuracy of the model. Combine the complexity of feature selection and the complexity of SVM parameters to calculate the fitness value of the individual.

[0127] Step S403, individual position update and chaotic mapping.

[0128] In each iteration of the HO algorithm, the individual position update is divided into three main stages, with different goals and update methods for each stage. To prevent the algorithm from falling into local optima and enhance the global search ability, chaotic mapping is introduced into each stage of position update as a perturbation factor.

[0129] (1) Generation of chaotic mapping: As before, a series of chaotic sequence values are generated by iterating the Logistic mapping formula. These values are used as the perturbation factor chaos in the position update process. chaos is a value extracted from the chaotic sequence, and its value range is between (0, 1). This value will be used as a multiplier in the individual position update process to introduce randomness and diversity.

[0130] (2) The first stage: Position update of the hippopotamus in the water area (exploration stage).

[0131] Extensive exploration in the early stage of the search space can find potential high-quality solutions. In each iteration, the position of the individual is updated using the perturbation generated by the chaotic mapping, including the values of the feature selection vector, penalty factor, and kernel parameter. The specific formula is as follows:

[0132]

[0133] Among them, are the feature selection vector, penalty factor, and kernel parameter of the i-th individual in the t-th iteration, respectively. S best 、C best 、γ best are the feature selection vector, penalty factor, and kernel parameter of the optimal individual in the current population, respectively. In the position update process, the introduction of chaotic mapping can increase the exploration and diversity of the search space, preventing the algorithm from falling into local optima in the early stage of the search.

[0134] (3) The second stage: The hippopotamus defends against predators (exploration stage).

[0135] When facing potential local optimum traps, the hippopotamus individuals adjust their positions to maintain their exploration ability. When a hippopotamus perceives a predator threat, it updates its position to stay safe while continuing to explore. The following formula is used for the update at this time:

[0136]

[0137] P j =lb j +r s ×(ub j -lb j )

[0138] Where: P j is the randomly generated predator position, representing the threat source. lb j and ub j are the lower and upper bounds of the search space respectively, and r s is a random number.

[0139] (4) The third stage: The hippopotamus escapes from the predator (exploitation stage).

[0140] In the exploitation stage, when encountering a serious threat, the hippopotamus individuals will make greater adjustments and finely adjust their positions through rapid local search to find the optimal solution. At this time, the following formula is used for update:

[0141]

[0142] Where: lb localj and ub localj are the lower and upper bounds of the local search space. s 1 is a random number used to further adjust the amplitude of position update.

[0143] Step S404, Iteration and Convergence Judgment of the HO Algorithm.

[0144] During the iteration of the HO algorithm, the hippopotamus individuals will continuously adjust their feature selection vector S i , the penalty factor C of SVM i and the kernel parameter γ i to find the global optimal solution. Each iteration will go through the exploration and exploitation stages, and the search ability is enhanced by introducing chaotic mapping.

[0145] (1) Iteration process.

[0146] Initial population generation: In the first generation (t = 1), the initial population is generated according to steps D1 and D2.

[0147] Fitness evaluation: For each individual X i calculate the fitness value Fitness(X i ). Select the individual with the optimal fitness as the current population's optimal X best .

[0148] Position update: In each iteration, use the formula in step D3 to update the feature selection vector, penalty factor, and kernel parameter of all individuals. The update process is divided into three stages:

[0149] The first stage (exploration stage): By approaching the current optimal solution X best and introducing chaotic perturbation, the individuals will adjust their positions within the global search space.

[0150] The second stage (defense stage): When an individual perceives a predator threat (i.e., a local optimal trap), it randomly generates the predator position Pj and updates its own position to avoid the local optimum.

[0151] The third stage (exploitation stage): The individual will perform a more refined local search, and by making adjustments within the local search space, it will search for the optimal solution.

[0152] Fitness re-evaluation: After the position update, the fitness value Fitness of all individuals is recalculated. The individual with the optimal fitness is selected again as the optimal solution X of the new generation. best 。

[0153] (2) Convergence judgment.

[0154] Set a maximum number of iterations Tmax = 300. When the number of iterations reaches this value, the algorithm automatically stops. Set a fitness convergence threshold ∈ = 0.01. If the change in the fitness value of the optimal individual in the population is less than ∈ in several consecutive iterations, the algorithm is considered to have converged. Once the convergence condition is met, the algorithm terminates and outputs the feature selection vector S of the optimal individual. best These optimal parameters will be used to train the final SVM model for predicting the efficacy of breast cancer NAC.

[0155] Specifically, see Figure 4 the flowchart of the hippopotamus algorithm, which includes the following steps:

[0156] Step1, set the number of hippopotamuses N and the total number of iterations T.

[0157] Step2, create the initial population, and set i = 1 and t = 1.

[0158] Step3, calculate the fitness function value.

[0159] Step4, update the dominant hippopotamus according to the fitness function value.

[0160] Step5, determine whether the current i is greater than N / 2. If so, perform the second-stage update and execute Step8. If not, perform the first-stage update and execute Step6.

[0161] Step6, explore in the early stage of the search space and update the individual position.

[0162] Step7, update the feature vector, penalty factor, and kernel function, i = i + 1, and execute Step5.

[0163] Step8, randomly generate the predator position.

[0164] Step 9, the hippopotamus individual updates its position to defend against predators.

[0165] Step 10, determine whether the current i is less than N. If so, i = i + 1 and execute Step 5. If not, execute Step 11.

[0166] Step 11, set i = 1.

[0167] Step 12, perform the third - stage update, and locally search to finely adjust the individual position;

[0168] Step 13, use the formula to update the candidate solution.

[0169] Step 14, determine whether the current i is less than N. If so, i = i + 1 and execute Step 12. If not, execute Step 15.

[0170] Step 15, save the best candidate solution found currently.

[0171] Step 16, determine whether t is less than T. If so, execute Step 4. If not, end the iteration.

[0172] Specifically, Step S5 includes Steps S501 - S506:

[0173] Step S501, feature extraction and screening: According to the radiomics high - throughput features of the multimodal ultrasound images extracted in Step S3, apply the optimal feature subset S best selected in Step S4 to all patient data, and screen out the optimal feature subset of each sample in the training dataset.

[0174] Step S502, SVM model training: Initialize the SVM model, and set the optimal SVM parameter C best obtained in Step D as the initial parameter of the model. Use the RBF kernel function as the kernel function of SVM.

[0175] Step S503, model training: Take the screened feature subset S best as the input feature and input it into the SVM model. Take the corresponding NAC efficacy label as the target output. Through the learning algorithm of SVM, find the hyperplane that can maximize the distance between classes, so as to divide the training dataset into different classes. The SVM model is continuously adjusted to find the optimal hyperplane to ensure the best classification of the samples in the training set.

[0176] Step S504, cross - validation: Use 5 - fold cross - validation to evaluate the performance of the model on the training dataset. Calculate and record indicators such as the average accuracy, AUC, sensitivity, and specificity of the cross - validation.

[0177] Step S505, Model Evaluation and Optimization: Evaluate the performance of the model on the training set, and record performance metrics such as the accuracy, AUC, sensitivity, and specificity of the model. If the performance on the training set is low, re-adjust the hyperparameters of the model or optimize the feature selection process.

[0178] Step S506, Model Saving: Save the finally trained SVM model as a file for subsequent prediction on new data. The saved model includes: the feature selection vector S best , the optimized SVM parameter C best- and γ best , as well as the trained SVM classifier.

[0179] To verify the effectiveness of this method, input the multi-modal ultrasound images of the independent test set into the trained SVM model for testing and evaluate the prediction efficacy. Compare the performances of different algorithms to verify the advantages of the improved HO algorithm. Specifically, it includes steps S601 - S605.

[0180] Step S601, Independent Test Set Data Preparation: Perform the same preprocessing on the data of each patient in the test set as on the training set, including extracting radiomics high-throughput features. According to the optimal feature subset S best selected in step S4, screen out the corresponding features in the test set.

[0181] Step S602, Model Loading: Load the optimal SVM model trained in step S5, which includes the feature selection vector S best , the penalty factor C best _, and the kernel parameter γ best .

[0182] Step S603, Prediction Using the SVM Model: Input the feature subset screened out from the independent test set into the trained SVM model. The SVM model predicts each sample in the test set according to the input feature set and outputs the corresponding prediction results (the predicted treatment effect is "pCR" or "non-pCR").

[0183] Step S604, Loading Models of Other Optimization Algorithms: In addition to the improved HO-SVM model provided in this embodiment, it is also necessary to load SVM models trained based on other optimization algorithms, including: MPA-SVM (Marine Predators Algorithm), GWO-SVM (Grey Wolf Optimization Algorithm), PO-SVM (Parrot Optimization Algorithm), RIME-SVM (Frost Ice Algorithm), and the original HO-SVM (HO algorithm without chaotic mapping)

[0184] Step S605, evaluate the model performance: Calculate the performance metrics of the SVM model on the test set, including accuracy, area under the receiver operating characteristic curve (AUC), sensitivity, and specificity. See Table 1 and Figure 5 as the test results, in Figure 5 , the abscissa (1 - Specificity) represents the false positive rate, and the smaller the value, the fewer false alarms of the model. The ordinate (Sensitivity) represents the true positive rate, and the larger the value, the higher the proportion of positive samples that the model can correctly identify.

[0185] Table 1 Performance of the test set in predicting pCR

[0186]

[0187] According to Table 1 and Figure 5 , the AUC of the improved HO algorithm reached 0.8533, significantly higher than other algorithms. This indicates that the improved HO algorithm shows stronger discriminative ability when distinguishing between chemotherapy pCR and non-pCR patients. In contrast, the worst-performing in terms of AUC is MPA-SVM, only 0.7535, while the AUC of the original HO algorithm is 0.8122, proving that the introduction of the chaotic mapping enhances the effect of global search. The performances of PO-SVM, RIME-SVM, and the original HO-SVM are relatively close but still lower than the improved HO algorithm, which shows that through feature dimensionality reduction and parameter optimization, the improved HO algorithm can find more representative classification features and more appropriate SVM hyperparameters.

[0188] In summary, the present method addresses the problem that the method for predicting based on a single-stage single-modal single ultrasound image can only extract information from a single dimension and has limited predictive value. By combining multi-modal ultrasound images, a novel feature optimization and classification prediction method is provided. First, by introducing a chaotic mapping and an improved hippopotamus optimization algorithm, the model performs excellently in feature selection and SVM parameter optimization. Compared with traditional optimization methods, the present method significantly improves the accuracy and AUC of the breast cancer NAC efficacy prediction model, and can more accurately predict the chemotherapy effect. Second, the introduction of the chaotic mapping effectively enhances the global search ability of the HO algorithm and reduces the risk of falling into local optima. This enables the model to more extensively explore potential solutions in the high-dimensional feature space, thereby finding a better feature subset and SVM parameter combination. This can effectively reduce redundant features, improve the generalization ability of the model, and reduce overfitting. It not only improves the robustness of the model but also reduces the computational complexity. Finally, the present method further improves the performance of the SVM classifier by simultaneously optimizing the penalty factor and kernel parameter of the SVM. The optimized SVM model performs more excellently when dealing with complex multi-modal image data and has higher classification accuracy and stability.

[0189] Verified by experiments, the present method shows significant advantages in the early prediction task of NAC efficacy. Compared with the single-branch network method, this method has made significant improvements in prediction accuracy and can provide a non-invasive tool for the early prediction of individualized NAC efficacy.

[0190] Embodiment 2

[0191] Based on Embodiment 1, the present embodiment provides a NAC efficacy prediction device based on multi-modal feature fusion and hippopotamus optimization. Refer to Figure 6 , which mainly includes:

[0192] (1) Ultrasound image region of interest annotation and preprocessing module, which is used to obtain multi-modal ultrasound images and perform region of interest annotation and image preprocessing. This module is used to implement the functions of steps S1 - S2 in Embodiment 1.

[0193] (2) Feature extraction and splicing module, which is used to extract the radiomics features corresponding to the ultrasound images in each modality respectively and obtain a high-dimensional feature vector through splicing. This module is used to implement the function of step S3 in Embodiment 1

[0194] (3) SVM model parameter and feature subset iterative optimization module, which is used to optimize the parameters of the SVM model based on the hippopotamus algorithm through chaotic mapping and reduce the high-dimensional feature vector to obtain the optimal feature subset. This module is used to implement the function of step S4 in Embodiment 1.

[0195] (4) SVM model training module, which is used to train the SVM model based on the parameters of the SVM model and the optimal feature subset by using the pre-constructed training data set, and obtain the trained SVM model. This module is used to implement the function of step S5 in Embodiment 1.

[0196] (5) NAC efficacy prediction module, which is used to take the high-dimensional feature vector of the object to be predicted as the input of the trained SVM model, and obtain the prediction result of whether the pathology is completely relieved after NAC treatment, so as to realize the efficacy prediction of NAC.

[0197] Embodiment 3

[0198] On the basis of the foregoing embodiments, this embodiment provides an electronic device, including: one or more processors and a memory. The memory stores one or more programs, and the one or more programs include instructions for executing the NAC efficacy prediction method based on multi-modal feature fusion and Hippo optimization as described in Embodiment 1.

[0199] As Figure 7 described, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory. Of course, it may also include other hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the above Figure 1 described method. Of course, in addition to the software implementation method, the present invention does not exclude other implementation methods, such as logic devices or a combination of software and hardware. That is to say, the execution subject of the following processing flow is not limited to each logic unit, and can also be hardware or a logic device.

[0200] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of, for example, read-only memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.

[0201] Computer-readable media include both permanent and non-permanent, removable and non-removable media and can store information by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transitory medium that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0202] As described above, the above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A NAC efficacy prediction method based on multimodal feature fusion and Hippo optimization, characterized in that: The high-dimensional feature vector of the object to be predicted is used as the input of the trained SVM model to obtain the prediction result of whether the pathology is completely relieved after NAC treatment, so as to achieve the prediction of NAC efficacy for non-therapeutic purposes, wherein the training process of the SVM model includes the following steps: Acquire multimodal ultrasound images; Performing region of interest marking and image preprocessing on the multimodal ultrasound image; The imaging features corresponding to the ultrasound images in each modality are extracted respectively, and a high-dimensional feature vector is obtained by splicing; Based on the Hippo algorithm, the parameters of the SVM model are optimized through chaotic mapping, and the high-dimensional feature vector is reduced to obtain the optimal feature subset; Based on the parameters of the SVM model and the optimal feature subset, the SVM model is trained using a pre-constructed training data set to obtain a trained SVM model.

2. According to claim 1, a NAC efficacy prediction method based on multimodal feature fusion and Hippo optimization is characterized in that: The region of interest marking comprises the following steps: For ultrasound images of each modality, annotation is performed on the premise that the annotated area covers the same lesion site, ensuring the consistency of the annotation of the region of interest between images of each modality.

3. The NAC efficacy prediction method based on multimodal feature fusion and Hippo optimization according to claim 1 is characterized in that: The image preprocessing includes gray value standardization, image size adjustment, denoising and data enhancement processing.

4. The NAC efficacy prediction method based on multimodal feature fusion and Hippo optimization according to claim 1 is characterized in that: The process of obtaining radiomics features and high-dimensional feature vectors includes the following steps: For each modality of ultrasound image, multiple types of features are extracted to form the imaging omics features corresponding to the modality; For the radiomics features corresponding to each modality, a high-dimensional feature vector is obtained by splicing in the vector dimension.

5. The NAC efficacy prediction method based on multimodal feature fusion and Hippo optimization according to claim 1 is characterized in that: Based on the Hippo algorithm, the process of optimizing the parameters of the SVM model through chaotic mapping and reducing the dimension of the high-dimensional feature vector to obtain the optimal feature subset includes the following steps: Initialize the Hippo algorithm population based on the chaotic map, and each individual in the population includes a feature selection vector and SVM model parameters; Calculate the fitness function value based on the classification accuracy of feature subsets and the complexity of SVM model parameters; Based on the calculated fitness function value, the individual positions are updated in multiple stages through chaotic mapping; The fitness function value calculation and individual position update are repeated for multiple iterations until the preset convergence conditions are reached, and the optimal parameters and feature subsets of the SVM model are output.

6. The NAC efficacy prediction method based on multimodal feature fusion and Hippo optimization according to claim 5 is characterized in that: The fitness function value is calculated using the following formula: Among them, Fitness(X i ) indicates that the population is X i The fitness function value when Acc cv (X i ) represents the feature subset S i The classification accuracy in cross validation, λ1 is the penalty coefficient for controlling the number of features, λ2 is the penalty coefficient for controlling the complexity of SVM parameters, Complexity (C i , γ i ) is the penalty factor C i and the kernel parameter γ i The complexity of the penalty factor C i and the kernel parameter γ i Initialize to C i =C min +r c ×(C max -C min ), γ i =γ min +r γ ×(γ max -γ min ), r c and r γ is the random number generated by the chaotic map, C min , C max , γ min , γ max are the minimum and maximum values ​​of the penalty factor and kernel parameter, respectively.

7. The NAC efficacy prediction method based on multimodal feature fusion and Hippo optimization according to claim 5, characterized in that: Based on the calculated fitness function value, the multi-stage individual position update through chaotic mapping includes: In the first stage, the perturbations generated by the chaotic map are used to update the positions of the individuals, including the values ​​of the feature selection vector, penalty factor, and kernel parameters: in, are the feature selection vector, penalty factor and kernel parameter of the i-th individual in the t-th iteration, s best 、c best , γ best are the feature selection vector, penalty factor and kernel parameter of the best individual in the current population, respectively. Chaos is the disturbance factor obtained by iterating the Logistic mapping formula. In the second stage, the individual positions are updated for potential local optimal traps: P j =lb j +r s ×(ub j -lb j ) Among them, P j is a randomly generated predator position, indicating the source of threat, lb j andub j are the lower and upper bounds of the search space, r s is a random number; In the third stage, compared with the second stage, a greater adjustment is made to the serious threat of local optimal traps. The positions of individuals are fine-tuned through local search to search for the optimal solution: Among them, lb localj and lb localj are the lower and upper bounds of the local search space, and s1 is a random number used to adjust the amplitude of the position update.

8. The NAC efficacy prediction method based on multimodal feature fusion and Hippo optimization according to claim 1, characterized in that: The multimodal ultrasound images include two-dimensional grayscale, elasticity and blood flow ultrasound images.

9. The NAC efficacy prediction method based on multimodal feature fusion and Hippo optimization according to claim 1, characterized in that: Based on the parameters of the SVM model and the optimal feature subset, the SVM model is trained using a pre-constructed training data set to obtain a trained SVM model, comprising the following steps: Based on the optimal feature subset, select corresponding features and construct a training data set; Initializing the SVM model based on the parameters of the SVM model; Using the training data set to train the SVM model; The performance of the trained SVM model was evaluated by multi-fold cross validation.

10. A NAC efficacy prediction device based on multimodal feature fusion and Hippo optimization, characterized in that: include: Ultrasound image region of interest annotation and preprocessing module, used to acquire multimodal ultrasound images and perform region of interest annotation and image preprocessing; The feature extraction and splicing module is used to extract the imaging features corresponding to the ultrasound images in each modality and obtain the high-dimensional feature vector by splicing; SVM model parameter and feature subset iterative optimization module, which is used to optimize the parameters of the SVM model through chaotic mapping based on the Hippo algorithm, and reduce the dimension of the high-dimensional feature vector to obtain the optimal feature subset; An SVM model training module is used to train the SVM model based on the parameters of the SVM model and the optimal feature subset using a pre-constructed training data set to obtain a trained SVM model; The NAC efficacy prediction module is used to use the high-dimensional feature vector of the object to be predicted as the input of the trained SVM model to obtain the prediction result of whether the pathology is completely relieved after NAC treatment, thereby realizing the efficacy prediction of NAC.

Images