Early prediction method for cancer chemotherapy efficacy based on dual-channel convolutional neural network

Abstract

The present invention discloses an early prediction method for the efficacy of cancer chemotherapy based on a dual-channel convolutional neural network. The pre-chemotherapy image data and the image data after the first-stage chemotherapy are respectively input into two channels for layer-by-layer convolutional operations. Feature fusion is performed between the two channels, and the features output by the last layer of the two channels are weighted and fused to obtain the output result. Each channel has 9 convolutional layers, and four feature fusions and three poolings are performed. The present invention takes into account the connection between data at different chemotherapy stages, can effectively utilize the connection features between data. In addition, the present invention adds a channel weight analysis strategy, guides the model to be trained by introducing prior knowledge, and more fully considers the connection between data at different chemotherapy stages. The prediction task can be completed only by using the ultrasound image data of the first-stage neoadjuvant chemotherapy, and the accuracy is high and the speed is fast.

Description

Technical Field

[0001] The present invention relates to the field of computers, and proposes a dual-channel neural network algorithm for predicting the efficacy of neoadjuvant chemotherapy for cancer. Background Art

[0002] Neoadjuvant chemotherapy refers to systemic chemotherapy performed before local treatment methods (such as surgery or radiotherapy), aiming to shrink the tumor mass and kill invisible metastatic cells at an early stage to facilitate subsequent treatments such as surgery and radiotherapy.

[0003] Neoadjuvant chemotherapy for breast cancer is a systemic drug treatment method before local surgical treatment or radiotherapy, targeting patients with locally advanced breast cancer (LABC). Patients are expected to achieve a pathologic complete response (pCR) after neoadjuvant chemotherapy, which is of great significance for the next step of the patient's treatment. However, a considerable number of patients do not have a pathologic complete response to this treatment method. Therefore, there is an urgent need to establish a method for predicting the efficacy of neoadjuvant chemotherapy to predict whether a patient can reach pCR after a series of treatments, so as to provide timely advice for the next step of their treatment.

[0004] The research status of the efficacy evaluation of neoadjuvant chemotherapy for breast cancer at home and abroad can be roughly divided into three categories. First, methods based on gene-related characteristics and pathological factors, which are quite subjective and have low prediction accuracy; second, machine learning methods based on imaging characteristics and clinical palpation indicators, which are essentially feature engineering work, consuming a lot of manpower and having insufficient accuracy; finally, recently emerging research based on deep learning, which is time-saving and labor-saving and can achieve a certain accuracy rate, but ignores the characteristic laws of data at each stage of neoadjuvant chemotherapy, resulting in low prediction accuracy. Therefore, exploring an auxiliary diagnosis model that can predict the efficacy of neoadjuvant chemotherapy with high accuracy remains a challenge. Summary of the Invention

[0005] The object of the present invention: Aiming at the problem of low accuracy of the current prediction algorithm, a dual-channel neural network algorithm for predicting the efficacy of neoadjuvant chemotherapy with high precision is established by introducing the feature sharing strategy and weight analysis strategy in the dual-branch network. This algorithm uses the ultrasonic image data of patients before and after the first stage of neoadjuvant chemotherapy as the research material, and can effectively complete the task of early predicting the effect of neoadjuvant chemotherapy with high accuracy.

[0006] To achieve the above object, the concept of the present invention is as follows: First, frame cutting operation is performed on the original ultrasound video data of neoadjuvant chemotherapy, and ultrasound images with different shapes and clear boundaries are selected; then preprocessing operation is performed on the selected ultrasound images, and this process includes cropping, grayscale conversion, noise reduction, and data augmentation of the region of interest (ROI) of the image; finally, a deep learning model is constructed based on the processed data set by using a two-channel neural network algorithm to achieve the purpose of predicting the efficacy of neoadjuvant chemotherapy.

[0007] The technical solution of the present invention is as follows:

[0008] An early prediction method for the efficacy of cancer chemotherapy based on a two-channel convolutional neural network: The image data before chemotherapy and the image data after the first-stage chemotherapy are respectively input into two channels for layer-by-layer convolutional operations, feature fusion is performed between the two channels, and the features output by the last layer of the two channels are weighted and fused to obtain the output result.

[0009] Each channel has 9 convolutional layers.

[0010] Four times of feature fusion and four times of pooling are performed.

[0011] When performing weighted fusion, the weight of the input channel of the image data before chemotherapy is 0.2, and the weight of the input channel of the image data after the first-stage chemotherapy is 0.8.

[0012] Preprocessing is performed before the image data is input, including cropping the ROI, grayscale conversion, and noise reduction processing.

[0013] Frame cutting operation is performed on the original ultrasound video data of neoadjuvant chemotherapy to obtain image data.

[0014] The specific steps include:

[0015] A. Video frame cutting operation: Frame cutting process is performed on the neoadjuvant chemotherapy ultrasound video, and some frames of images with different shapes and clear boundaries are selected for supporting processing;

[0016] A1 Video frame cutting operation, in which a video with a certain time length is cut at a fixed frame interval into an indefinite number of M ultrasound images, and then N ultrasound images with different lesion shapes and clear boundaries are selected as the sample data at this stage.

[0017] A2 Frame image supporting processing: According to the 2N frame image data of the two chemotherapy stages, one image is selected from the N1 set of the data before chemotherapy, and then one ultrasound image is randomly selected from the N2 set of the data after the first-stage chemotherapy to be paired and used as a group of inputs for the next step. The paired data has the same type of label, that is, pathological complete remission and incomplete remission.

[0018] B. Data preprocessing: Select and crop the ROI of the ultrasonic image for the supporting frame ultrasonic image data; then grayscale the ROI area to convert the three-channel color image to grayscale; finally, use the median filtering method to denoise the grayscale image.

[0019] B1 ROI cropping, crop the single-frame ultrasonic image to obtain the ROI image with lesions.

[0020] B2 Grayscale the tumor ROI obtained in B1 to make it a single-channel image.

[0021] B3 Use the median filtering method to denoise the tumor ROI after B2 processing.

[0022] C. Data calculation: Establish a two-channel neural network for predicting the efficacy of neoadjuvant chemotherapy by combining the feature sharing strategy and weight analysis strategy between channel branch networks, and input the supporting image data into the neural network.

[0023] Use the supporting image data with known labels for the training and testing of the neural network, and use the trained neural network to predict the labels of the supporting images.

[0024] Based on the study of the imaging features of tumors in ultrasonic images and the deep learning method for image recognition, this technology proposes a two-channel neural network algorithm for predicting the efficacy of neoadjuvant chemotherapy. This method uses the imaging data before and after the first stage of neoadjuvant chemotherapy to complete the prediction task. The algorithm is assisted in training by adopting the sharing strategy and weight analysis strategy of the channel branch network, which can effectively improve the accuracy of efficacy prediction. Compared with the prior art, the present invention has the following outstanding substantive features and significant advantages:

[0025] 1. The previous research methods for neoadjuvant chemotherapy were based on the first two stages of neoadjuvant treatment. The present invention can complete the prediction task only by using the ultrasonic imaging data of the first stage of neoadjuvant chemotherapy, and has high accuracy and fast speed.

[0026] 2. The previous research methods for neoadjuvant chemotherapy were based on imaging data such as CT, MRI, and PET. The present invention uses ultrasonic imaging data, fully considering the practical significance of the research materials.

[0027] 3. The present invention considers the connection between data in different chemotherapy stages, can effectively utilize the connection features between data. In addition, the present invention adds a channel weight analysis strategy, and guides the model to train by introducing prior knowledge, which more fully considers the connection between data in different chemotherapy stages. Description of the Drawings

[0028] Figure 1It is an example of ultrasound images before and after the first stage of neoadjuvant chemotherapy for breast cancer. Pre-NAC is the image data before chemotherapy, and NAC1 is the image data after the first stage of chemotherapy.

[0029] Figure 2 It is a flowchart of step A. First, the original ultrasound video data is frame-cut, and then tumor ultrasound images with different shapes and clear boundaries are selected and processed accordingly.

[0030] Figure 3 It is a flowchart of step B. First, Figure 2 the paired ultrasound images obtained through step A in are cropped, the ROI region is selected, and then the ROI region is grayscale and median-filtered for noise reduction to obtain the preprocessed image data.

[0031] Figure 4 It is a dual-channel neural network algorithm framework diagram for predicting the efficacy of neoadjuvant chemotherapy for breast cancer in the present invention.

[0032] Figure 5 It is Figure 4 a framework diagram of the feature sharing strategy proposed in step C in .

[0033] Figure 6 It is Figure 4 a framework diagram of the feature weight analysis strategy proposed in step C in .

[0034] Figure 7 It lists the performance comparison of the present invention under different numbers of convolutional layers and feature sharing methods.

[0035] Figure 8 It lists the performance comparison of the present invention under different feature connection methods.

[0036] Figure 9 It lists the performance comparison of the present invention under different data augmentation strategies.

[0037] Figure 10 It lists the performance comparison of the present invention with the algorithms proposed in the latest research since 2020.

[0038] Figure 11 It lists the performance comparison of the present invention based on data from different chemotherapy stages. Detailed implementation manners

[0039] Example 1: The concept of the present invention is as follows: First, frame extraction is performed on the original ultrasound video data of adjuvant chemotherapy, and ultrasound images with different shapes and clear boundaries are selected; then, preprocessing operations are performed on the selected ultrasound images, which include cropping, grayscale conversion, noise reduction, and data enhancement of the region of interest (ROI) of the image; finally, a deep learning model is constructed using a dual-channel neural network algorithm based on the processed dataset to achieve the purpose of predicting the efficacy of neoadjuvant chemotherapy. The training and testing of the neural network are carried out using the supporting image data with known efficacy, and the trained neural network is used to predict the efficacy of the supporting images.

[0040] The specific steps include:

[0041] A. Video frame extraction operation: Frame extraction is performed on the ultrasound video of neoadjuvant chemotherapy, and some frame images with different shapes and clear boundaries are selected for supporting processing;

[0042] A1 Video frame extraction operation, in which a video of a certain time length is cut at a fixed frame interval, cut into an indefinite number of M ultrasound images, and then N ultrasound images with different lesion shapes and clear boundaries are selected as the sample data at this stage.

[0043] A2 Frame image supporting processing: Based on the 2N frame image data of two chemotherapy stages, one image is selected from the N1 set of pre-chemotherapy data, and then one ultrasound image is randomly selected from the N2 set of post-chemotherapy data of the first stage to be matched and used as a group of inputs for the next operation. The supporting data has the same type of label, that is, pathological complete remission and incomplete remission.

[0044] B. Data preprocessing: ROI selection and cropping of ultrasound images are performed on the supporting frame ultrasound image data; then the ROI area is grayscale-converted to convert the three-channel color image into a grayscale image; finally, the grayscale image is denoised using the median filtering method;

[0045] B1 ROI cropping, cropping a single-frame ultrasound image to crop out the ROI image with a lesion;

[0046] B2 The tumor ROI obtained in B1 is grayscale-converted to make it a single-channel image;

[0047] B3 The median filtering method is used to denoise the tumor ROI processed in B2.

[0048] C. Data calculation: A dual-channel neural network for predicting the efficacy of neoadjuvant chemotherapy is established by combining the feature sharing strategy and weight analysis strategy between channel branch networks, and the supporting image data is input into the neural network.

[0049] An early prediction method for the efficacy of cancer chemotherapy based on a dual-channel convolutional neural network inputs the pre-chemotherapy image data and the image data after the first-stage chemotherapy into two channels respectively for layer-by-layer convolutional operations, performs feature fusion between the two channels, and performs weighted fusion on the features output by the last layer of the two channels to obtain the output result.

[0050] Each channel has 9 convolutional layers.

[0051] Four times of feature fusion and four times of pooling are performed.

[0052] When performing weighted fusion, the weight of the input channel for pre-chemotherapy image data is 0.2, and the weight of the input channel for image data after the first-stage chemotherapy is 0.8.

[0053] Preprocessing is performed before inputting the image data, including cropping the ROI, grayscale conversion, and noise reduction processing.

[0054] Frame extraction is performed on the original ultrasound video data of neoadjuvant chemotherapy to obtain image data.

[0055] Example 2:

[0056] The preferred embodiments of the present invention are further described in detail below with reference to the accompanying drawings. In this embodiment, the specific configuration of the server for running the experiment is as follows: the CPU is Intel Xeon Silver 4110, the graphics card is two Nvidia GeForce RTX2080Ti GPUs, and the ROM is 64GB. In terms of model establishment, both the dual-channel neural network algorithm and the comparison model are implemented based on the open-source deep learning tool PyTorch 1.7.0, and the software environment of the experimental platform is Python 3.7 version. In terms of experimental settings, the experiment selects Adam as the optimization algorithm, the batch size is set to 8, the initial learning rate is set to 0.005, and the number of iterations is set to 128. In terms of the loss function, the cross-entropy loss function is used. In terms of performance evaluation, the experiment evaluates the prediction classification performance of the method through performance parameters such as accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), F1-score, and area under the receiver operating characteristic curve (AUC). The larger these seven performance indicators are, the better the prediction classification effect of the method, and vice versa.

[0057] In this embodiment, breast cancer is used as the research object, and this method is also applicable to other cancers. Neoadjuvant chemotherapy for breast cancer is a systemic drug treatment method before local surgical treatment or radiotherapy, targeting patients with locally advanced breast cancer (LABC). After neoadjuvant chemotherapy, patients are expected to achieve a pathologic complete response (pCR). The dataset used was collected from a cooperative hospital, and 752 sets of ultrasound images of 72 patients were used. According to a 4:1 ratio, 600 sets were used for the training set and 152 sets were used for the test set. Among all the paired images, there were 300 sets of pCR ultrasound images (30 patients), 240 sets for training and 60 sets for testing; there were 452 sets of pR ultrasound images (42 patients), 360 sets for training and 92 sets for testing. The collection of the dataset was operated by experienced professional doctors. Specific ultrasound image examples are as Figure 1 shown.

[0058] The dual-channel neural network algorithm for predicting the efficacy of neoadjuvant chemotherapy for breast cancer of the present invention, as Figures 2 - 6 shown, includes the following steps:

[0059] A. Since the collected ultrasound data is video data, in order to be able to input it into the neural network, a series of processing operations are first performed on the video format data. The detailed steps are as Figure 2 shown. The specific steps include:

[0060] A1. Video frame cutting operation. In this operation, a video of a certain time length is cut at a fixed frame interval into an indefinite number of M ultrasound images, and then N ultrasound images with different lesion morphologies and clear boundaries are selected as the sample data at this stage. The processing code for this stage is based on the CV package of Python.

[0061] A2. Frame image matching processing: According to the 2N frame image data of the two chemotherapy stages, one image is selected from the N1 set in Pre-NAC, and then one ultrasound image is randomly selected from the N2 set in NAC1 to be matched and used as a group of inputs for the next operation. The paired data has the same type of label, that is, pathologic complete response and incomplete response.

[0062] B. Perform preprocessing operations on the paired data: crop the ROI, grayscale and noise reduction processing. The specific process is as Figure 3 shown.

[0063] The specific steps include:

[0064] B1. ROI cropping: The single-frame ultrasound image processed by A2 contains some useless or private information, such as the ultrasound instrument model, patient's personal information, etc. These information are of no use to the algorithm. Therefore, it is necessary to crop the single-frame ultrasound image to crop out the ROI image with the lesion, and the pixel size of this image is 445 pixels × 445 pixels;

[0065] B2. Grayscale processing of the tumor ROI obtained in B1: Since the original ultrasound image is an RGB three-channel image, it will consume a lot of training time. Therefore, grayscale processing is performed to make it a single-channel image;

[0066] B3. Denoising the tumor ROI processed in B2 by using the median filtering method. The multiplicative speckle noise in the ultrasound image is relatively serious, so median filtering can be used for denoising. Median filtering belongs to a non-linear filtering method. Its general content is as follows: First, define a filter F with an odd window size jk , and then arrange the pixels in the filter in ascending order to obtain a filtering queue {F jk}, and use the median of all pixel values in {F jk} to replace the pixel value at the center position G(i, y) of the filter. Its formula can be expressed as:

[0067] G(i, y) = Mid{F jk} #(1)

[0068] Median filtering can ensure the preservation of the edge characteristics of the image while overcoming noise, and can play an effective role in denoising.

[0069] C. Establish a prediction framework: Combine the analysis of the number of dual-channel convolutional layers, the feature sharing strategy and the weight connection strategy to establish a new neoadjuvant chemotherapy efficacy prediction framework. The specific framework of the model is as Figure 4 shown: The specific steps include:

[0070] C1. The analysis of the number of dual-channel convolutional layers is actually the main body architecture of the model determined according to the task. If the model is too deep, it will lead to a decrease in accuracy due to overfitting. If the model is too shallow, it will lead to insufficient feature extraction and a decrease in accuracy. Therefore, in the present invention, a series of dual-channel convolutional layers are explored, and finally nine-layer dual-channel convolution is selected as the main framework of the model.

[0071] C2. On the basis of the exploration in C1, determine the feature sharing strategy. In the present invention, a feature fusion method is proposed as the way of feature sharing. Specifically, it is as Figure 5 shown. First, the network starts from the input layer, and is expressed by the formula:

[0072] C0 = X #(2)

[0073] C′0 = Y#(3)

[0074] Among them, X represents the input of Pre-NAC data, and Y represents the input of NAC1 data. Then, C0 and C′0 are respectively input into their respective convolutional layers to extract features through convolutional kernels. Finally, feature maps C1 and C′1 are generated. The formula is expressed as:

[0075] C i = σ i (ω i * C i-1 + b i )#(4)

[0076] C′ i = σ′ i (ω′ i * C′ i-1 + b′ i )#(5)

[0077] Among them, C i , C′ i represent the feature maps of the i-th layer, and the value range of i is {1, 3, 5, 7, 8}. σ i , σ′ i both represent the ReLU activation function, ω i , ω′ i are respectively the network weights of the i-th layer of the two channels, b i , b′ i are both the network biases of the i-th layer, and * represents the convolution operation. C i-1 , C′ i-1 are respectively used as the inputs of the next layer C i , C′ i .

[0078] C j = σ j (ω j *(C j-1 + C′ j-1 )+ b j )#(6)

[0079] C′ j = σ′ j (ω′ j *(C′ j-1 + C j-1 )+ b′ j )#(7)

[0080] Among them, C j , C′ j represent the feature maps of the j-th layer, and the value range of j is {2, 4, 6, 9}. Cj-1 , C' j-1 serve as the inputs for the next layer C j , C' j respectively.

[0081] After passing through two convolutional layers, C2 and C'2 can be obtained and are respectively put into the downsampling layer to reduce the dimension of the feature map. The downsampling layer is represented by the formula:

[0082] C k = maxpooling(C k ) #(8)

[0083] C' k = maxpooling(C' k ) #(9)

[0084] where C k , C' k represent the feature maps of the k-th layer, and the value range of k is {2, 4, 7, 9}.

[0085] During the learning process, the branch network fuses the features from different channels, which can, to a certain extent, emphasize the connection between features and ensure the sufficiency of learning. Feature fusion is essentially an operation of adding the values of the feature maps from the two branch networks, which can, to a certain extent, reflect the differences between the feature maps in different branches, thus emphasizing the discrimination between data in different chemotherapy stages.

[0086] C3. On the basis of the exploration of C2, determine the feature connection strategy. In the present invention, a way of using weight connection as feature fusion is proposed. Specifically, as Figure 6 shown, the feature maps from different branch networks are multiplied by different weights to highlight the importance of different channels. Finally, the updated features are added together to obtain the final feature vector F(Z) that fuses the two channels. This process is represented by the formula:

[0087] F(Z) = (α * C9 + β * C'9) #(10)

[0088] where α is the weight coefficient of the Pre-NAC channel and β is the weight coefficient of the NAC1 channel.

[0089] Finally, through two fully connected layers, the mapping from high dimension to low dimension is completed, and then the prediction task is completed.

[0090] Refer to Figure 7, which shows the performance comparison under different combinations of the number of two-channel convolutional layers in step C1 and the feature sharing strategy in step C2. It can be seen that the model can achieve better performance when using feature fusion, and the accuracy, sensitivity, NPV, and F1-score are all higher than those of the feature concatenation method and the CNN-9 model without feature sharing.

[0091] Refer to Figure 8 , which shows the performance comparison results under different feature connection methods in step C3. When the feature weight of the Pre-NAC channel is 0.2 and the feature weight of the NAC1 channel is 0.8, the model has the best effect and a relatively high accuracy. Therefore, in the present invention, the weight concatenation strategy is adopted for the channel feature connection method, and the experimental results also show that the best effect can be achieved by using this method. In addition, the setting of the weights to 0.2 and 0.8 is interpretable. The contribution of the chemotherapy data in the NAC1 stage to the prediction results of patients is significantly greater than that in the Pre-NAC stage.

[0092] Refer to Figure 9 , which shows the performance comparison under different data augmentation strategies. The present invention does not adopt any data augmentation method, but the effect is the best. The interpretable reasons are as follows: First, when compared with the data augmentation method of geometric transformation, it can be seen that not augmenting is much higher than the performance under this augmentation strategy. This is because the data sample volume is already unbalanced, and augmenting both types of samples will exacerbate the data imbalance to a greater extent, resulting in a decrease in the model's performance. The Mixup augmentation strategy is also too low for the same reason. In addition, the Mixup strategy may not be suitable for the augmentation of medical datasets. According to its principle, Mixup will disrupt the relationship between lesions and the surrounding areas of lesions, as well as between lesions and non-lesion areas in the same image, causing the model to learn a lot of incorrect information. Finally, it is the upsampling of the minority class samples, which means enhancing the data volume of the samples with the pathological result of pCR to be the same as that of the pR samples, and the experimental results are also not as good as the performance of the model without data augmentation. The possible reason is that when the prediction model learns the data distribution among pCR samples, it learns redundant features, resulting in a decrease in the model's performance.

[0093] Refer to Figure 10 , which shows the result comparison between the prediction algorithm of the present invention and the algorithms proposed in the latest research since 2020. The experimental results show that the prediction method proposed in this paper is superior to the methods proposed in the latest research in terms of accuracy, sensitivity, NPV, F1-score, and AUC value indicators.

[0094] Refer to Figure 11, which shows the comparison of the prediction results of the data of the present invention in different chemotherapy stages. It can be seen from the Pre-NAC and NAC1 chemotherapy data that the effect of using a single-channel network to predict single-stage data is not as good as that of the present algorithm using multi-stage data. In addition, it can also be seen that when using single-stage data, the performance of the model trained under NAC1 chemotherapy data is higher than that of the model under Pre-NAC chemotherapy data, which also reflects the necessity of weight analysis.

[0095] In summary, compared with the similar representative methods, the algorithm for predicting the efficacy of neoadjuvant chemotherapy for breast cancer of the present invention not only considers the necessity of feature sharing in the training process of the branch network, but also considers the problem of weights during feature splicing. Its overall prediction accuracy is higher and it has higher stability. It can analyze the patient's situation more accurately and comprehensively and determine whether the patient can achieve pathological complete remission.

[0096] This article is described in combination with the accompanying drawings of the specification and specific embodiments only to help understand the method and core idea of the present invention. The method described in the present invention is not limited to the embodiments described in the specific implementation manners. Other implementation manners obtained by those skilled in the art based on the method and idea of the present invention also belong to the scope of the technical innovation of the present invention. The content of this specification should not be construed as a limitation on the present invention.

Claims

1. An early prediction method for the efficacy of cancer chemotherapy based on a dual-channel convolutional neural network, characterized in that: The image data before chemotherapy and the image data after the first - stage chemotherapy are respectively input into two channels for layer - by - layer convolution operations. Feature fusion is performed between the two channels, and the features output by the last layer of the two channels are weighted and fused to obtain the output result. The specific steps include: A. Video frame - cutting operation: Perform frame - cutting processing on the neoadjuvant chemotherapy ultrasound video, and select some frame images with different shapes and clear boundaries for supporting processing, including: A1 Video frame - cutting operation. In this operation, a video with a certain time length is cut at a fixed frame interval into an indefinite number of M ultrasound images, and then N ultrasound images with different lesion shapes and clear boundaries are selected as the sample data for this stage; A2 Frame - image supporting processing: According to the 2N frame - image data of the two chemotherapy stages, select one image from the N1 set in the pre - chemotherapy data, and then randomly select one ultrasound image from the N2 set in the data after the first - stage chemotherapy to match it. These are used as a set of inputs for the next step. The supporting data has the same type of label, namely pathological complete remission and incomplete remission; B. Data pre - processing: Select and crop the ROI of the ultrasound image for the supporting frame ultrasound image data; then gray - scale the ROI area to convert the three - channel color image into a gray - scale image; finally, use the median filtering method to denoise the gray - scale image; C. Data calculation: Establish a two - channel neural network for predicting the efficacy of neoadjuvant chemotherapy by combining the feature - sharing strategy and the weight - analysis strategy between the channel - branch networks, and input the supporting image data into the neural network.

2. The early prediction method for cancer chemotherapy efficacy based on a dual-channel convolutional neural network according to claim 1, characterized in that: Each channel has 9 convolutional layers.

3. The early prediction method for cancer chemotherapy efficacy based on a dual-channel convolutional neural network according to claim 2, wherein: Perform four times of feature fusion and three times of pooling.

4. The early prediction method for cancer chemotherapy efficacy based on a dual-channel convolutional neural network according to claim 2, characterized in that: When performing weighted fusion, the weight of the input channel for the pre - chemotherapy image data is 0.2, and the weight of the input channel for the image data after the first - stage chemotherapy is 0.

8.

5. The early prediction method for cancer chemotherapy efficacy based on a dual-channel convolutional neural network according to any one of claims 1-4, characterized in that Step B includes: B1 ROI cropping, crop the single - frame ultrasound image to obtain the ROI image with the lesion; B2 Gray - scale the tumor ROI obtained in B1 to make it a single - channel image; B3 Use the median filtering method to denoise the tumor ROI processed in B2.

6. The early prediction method for cancer chemotherapy efficacy based on a dual-channel convolutional neural network according to claim 5, characterized in that: Use the supporting image data with known labels for the training and testing of the neural network, and use the trained neural network to predict the labels of the supporting images.

Images