Classification network training method, image classification method and related equipment

Abstract

The present application discloses a training method for a classification network, an image classification method, an electronic device, and a computer-readable storage medium. The training method includes: obtaining a training set, the training set includes multiple sample image sequences, each sample image sequence includes multiple sample images; using a classification network to extract a first feature of the sample image sequence, using a segmentation network to process the sample image sequence to obtain a processing result; using a classification network to obtain an attention weight corresponding to the first feature based on the processing result; using a classification network to multiply the first feature and the attention weight to obtain a multiplied feature; using a classification network to classify the sample image sequence based on the multiplied feature to obtain a first classification result, the first classification result is used to represent the category of the airway; adjusting the parameters of the classification network based on the first classification result. In this way, the accuracy of airway classification can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a classification network training method, an image classification method, an electronic device, and a computer-readable storage medium. Background Art

[0002] Tracheal intubation involves inserting a specialized endotracheal tube through the patient's oral or nasal cavity and into the trachea via the glottis. This technique provides optimal conditions for airway (respiratory) patency, ventilation and oxygenation, suctioning, and preventing aspiration, making it a crucial measure for rescuing patients with respiratory dysfunction. Tracheas vary in size, making intubation more difficult or challenging. Therefore, to successfully perform intubation, the appropriate intubation method must be selected based on the appropriate difficulty.

[0003] Existing technology relies solely on medical personnel's experience to classify the patient's airway by examining the nasal cavity, teeth, mouth opening, neck mobility, and throat, thereby determining the difficulty of tracheal intubation. However, since the area below the throat and into the lungs cannot be observed, this method has significant blind spots, resulting in low airway classification accuracy. Summary of the Invention

[0004] The present application provides a classification network training method, an image classification method, an electronic device, and a computer-readable storage medium, which can solve the problem of low accuracy of existing manual airway classification.

[0005] To solve the above technical problems, a technical solution adopted in this application is: providing a training method for a classification network. The method includes: obtaining a training set, the training set including multiple sample image sequences, each sample image sequence including multiple sample images, each sample image corresponding to a different position of the airway of the same living body, and each sample image including at least part of the airway and its related tissues; using a classification network to extract a first feature of the sample image sequence, using a segmentation network to process the sample image sequence to obtain a processing result, the segmentation network is trained on the training set, and during the training of the segmentation network, the segmentation network is used to perform semantic segmentation on the airway and airway-related tissue corresponding to the sample image sequence; using the classification network to obtain an attention weight corresponding to the first feature based on the processing result; using the classification network to multiply the first feature and the attention weight to obtain a multiplied feature; using the classification network to classify the sample image sequence based on the multiplied feature to obtain a first classification result, the first classification result being used to represent the category of the airway; and adjusting the parameters of the classification network based on the classification result.

[0006] In order to solve the above technical problems, another technical solution adopted in the present application is: to provide an image classification method, the method comprising: obtaining a target image sequence, the target image sequence comprising multiple target images, each target image corresponding to a different position of the target airway, and each sample image comprising at least part of the target airway and its related tissues; inputting the target image sequence into a classification network to obtain a classification result of the target image sequence, the classification result of the target image sequence being used to represent the category of the target airway; wherein, the classification network is trained using the aforementioned method.

[0007] To solve the above technical problems, another technical solution adopted in this application is: to provide an electronic device, which includes a processor and a memory connected to the processor, wherein the memory stores program instructions; the processor is used to execute the program instructions stored in the memory to implement the above method.

[0008] In order to solve the above technical problems, another technical solution adopted in this application is: providing a computer-readable storage medium storing program instructions, which can implement the above method when executed.

[0009] Through the above method, in the process of training the classification network of the present application, the classification network does not directly classify based on the first feature, but obtains the attention weight based on the processing result, and multiplies the attention weight with the first feature to obtain the multiplied feature, and then classifies based on the multiplied feature. Due to the multiplication of the first feature and the attention weight, the classification network can increase its attention to the airway classification-related features in the first feature (the part corresponding to the airway and airway-related tissues) and reduce its attention to the airway classification-irrelevant features (other parts) in the first feature, so that the first classification result obtained can be more accurate. Therefore, the present application can improve the training effect of the classification network, so that the classification network can have a higher accuracy in airway classification during the application process. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 1 is a flow chart of a first embodiment of a training method for a classification network of the present application;

[0011] Figure 2 This is a structural diagram of the classification network and segmentation network of this application;

[0012] Figure 3 This is a schematic diagram of the result of the coding block of this application;

[0013] Figure 4 This is a schematic diagram of the implementation principle of the recalibration layer of this application;

[0014] Figure 5 This is a schematic diagram of the implementation principle of the AFF module of this application;

[0015] Figure 6 This is a schematic diagram of the implementation principle of the MS-CAM of this application;

[0016] Figure 7 1 is a flow chart of the second embodiment of the training method of the classification network of the present application;

[0017] Figure 8 yes Figure 7 Specific process diagram of S21;

[0018] Figure 9 1 is a flow chart of the third embodiment of the training method of the classification network of the present application;

[0019] Figure 10 1 is a flow chart of the fourth embodiment of the image classification method of the present application;

[0020] Figure 11 This is a structural diagram of an embodiment of an electronic device of the present application;

[0021] Figure 12 It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0023] The terms "first," "second," and "third" in this application are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, a feature specified as "first," "second," or "third" may explicitly or implicitly include at least one of the features. In the description of this application, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically specified.

[0024] Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments unless there is a conflict.

[0025] Before introducing the method provided by this application, we first explain the training modes of the classification network and the segmentation network. There are many training modes, and two implementation methods are listed below:

[0026] The first method involves training the segmentation network on the training set (which will be covered later and not detailed here). After the segmentation network has achieved the desired training results, a joint model of the trained segmentation network and classification network is trained. This joint training process essentially trains the classification network, with the segmentation network serving as a supporting model.

[0027] The second method, considering the limited amount of airway-related data, aims to improve training effectiveness. Prior to the first training method, the segmentation and classification networks are trained separately on a public dataset. Then, the segmentation and classification networks are jointly trained on the public dataset. Furthermore, after these training steps, the trained joint model is validated and tested using a validation set and a test set, respectively. The training, validation, and test sets are split from the original dataset. For example, the split ratio can be 6:2:2.

[0028] Figure 1 It is a flow chart of the first embodiment of the training method of the classification network of the present application. It should be noted that if there are substantially the same results, this embodiment does not use Figure 1 The process sequence shown is limited. Figure 1 As shown, this embodiment may include:

[0029] S11: Get the training set.

[0030] The training set includes multiple sample image sequences, each sample image sequence includes multiple sample images, each sample image corresponds to a different position of the airway of the same living body, and each sample image includes at least part of the airway and related tissues.

[0031] The airway-related tissue may be the tongue root, fat, and other tissues related to the airway in the sample image.

[0032] The sample image sequence is labeled with an airway category label to identify the true category of the airway. In addition, the sample image sequence can also be labeled with a semantic category label to identify the semantic category of each pixel (e.g., airway, tongue base, fat, and others).

[0033] Methods for acquiring a sample image sequence include, but are not limited to, CT scanning. In a CT scanning acquisition method, the sample image sequence is obtained by sequentially performing CT scans from the head to the lungs of a living subject, and the sample image sequence is a CT image sequence. Each image includes a cross-section of the airway and airway-related tissue located on the cross-section.

[0034] S12: Using the classification network to extract the first feature of the sample image sequence, and using the segmentation network to process the sample image sequence to obtain a processing result.

[0035] The segmentation network is trained on the training set, and in the process of training the segmentation network, the segmentation network is used to perform semantic segmentation of the airway and airway-related tissues corresponding to the sample image sequence.

[0036] Semantic segmentation involves pixel-level classification of sample images, grouping pixels in the sample image sequence that belong to the same semantic category. For example, if the semantic categories include airway, tongue base, fat, and other, then during the training of the segmentation network, the segmentation results for the sample image sequence are the probabilities of each pixel in the sample image sequence belonging to airway, tongue base, fat, and other, respectively.

[0037] It can be understood that since the segmentation network is used to perform semantic segmentation on the airway and airway-related tissues corresponding to the sample image sequence during the training process of the segmentation network, the processing results of the sample image sequence by the trained segmentation network cover the semantic category information of the pixel points of the sample image sequence, that is, it can indirectly reflect which pixel points in the sample image sequence correspond to the airway part, which pixel points correspond to the airway-related tissue part, and which pixel points correspond to other parts.

[0038] Optionally, the processing result is the second feature of the sample image sequence. A segmentation network can be used to extract features from the sample image sequence to obtain the second feature of the sample image sequence.

[0039] Optionally, the processing result is a first segmentation result of the sample image sequence. A segmentation network may be used to perform a first semantic segmentation of the airway and airway-related tissues on the sample image sequence to obtain the first segmentation result.

[0040] S13: Using the classification network to obtain the attention weight corresponding to the first feature based on the processing result.

[0041] The classification network can be used to obtain a convolution kernel that is suitable for the processing result, and the classification network can be used to convolve the processing result based on the convolution kernel to obtain the attention weight.

[0042] S14: Use the classification network to multiply the first feature and the attention weight to obtain a multiplied feature.

[0043] Multiplying the first feature by the attention weight causes the classification network to focus more on features related to airway classification (the portion corresponding to the airway and airway-related tissues) and less on features irrelevant to airway classification (the rest of the first feature). Subsequent classification using this multiplied feature results in more accurate classification results.

[0044] The implementation process of S13-S14 above can be called context adaptive convolution. Figure 2 The implementation process of S13-S14 is explained with an example. Figure 2 As shown in the figure, when the processing result is the second feature, and the second feature is the output of DecoderBlock (refer to the following description, which is not repeated here), the classification network can be used to obtain the convolution kernel (CaC Kernels) corresponding to the processing result (output of DecoderBlock), and the classification network can be used to convolve the processing result based on CaCKernels to obtain the attention weights (Weighting factors). The classification network is used to multiply the processing result and the Weighting factors to obtain the multiplied features (Out feature maps).

[0045] Specifically, two convolutions can be performed on the second feature to generate features Q (B, C, H, W) and feature K (B, S^2, H, W); the corresponding points of Q and K are multiplied and summed, and then batch normalization is performed to obtain CaCKernels. CaC Kernels are convolved with the second feature using multiple dilation coefficients (for example, 1, 2, 3) to obtain multiple corresponding different results; finally, the multiple different results are added to obtain Weighting factors; the Weighting factors are multiplied with the first feature to obtain the multiplied feature.

[0046] S15: Using a classification network to classify the sample image sequence based on the multiplied features, to obtain a first classification result.

[0047] The first classification result is used to indicate the category of the airway.

[0048] The first classification result may be an airway category probability, and the category of the airway may be determined by the relationship between the airway category probability and a threshold.

[0049] S16: Adjust parameters of the classification network based on the first classification result.

[0050] Based on the difference between the first classification result and the airway category label, a loss function (eg, BCEloss) of the classification network can be obtained, and parameters of the classification network can be adjusted based on the loss function.

[0051] S17: Repeat the above process until the preset conditions are met.

[0052] The preset conditions can be that the loss function converges, the number of training times / time reaches a threshold, the training effect meets expectations, etc.

[0053] Through the implementation of this embodiment, in the process of training the classification network in this application, the classification network does not directly classify based on the first feature, but instead obtains an attention weight based on the processing result, multiplies the attention weight by the first feature to obtain a multiplied feature, and then classifies based on the multiplied feature. Since the first feature and the attention weight are multiplied, the classification network can increase its attention to the airway classification-related features in the first feature (the part corresponding to the airway and airway-related tissues) and reduce its attention to the airway classification-irrelevant features (other parts) in the first feature, so that the first classification result can be more accurate. Therefore, the present application can improve the training effect of the classification network, thereby achieving a higher accuracy in airway classification during the application process of the classification network.

[0054] The following describes the training phase of the classification network and the structures of the classification network and the segmentation network.

[0055] If the processing result is the second feature, the segmentation network may include at least one encoding block and at least one decoding block. The sample image sequence may be processed sequentially using the at least one encoding block and the at least one decoding block to obtain the second feature.

[0056] If the processing result is the first segmentation result, the segmentation network may further include a semantic segmentation block. The segmentation network may sequentially process the sample image sequence using at least one encoding block and at least one decoding block to obtain features for semantic segmentation; and use the semantic segmentation block to perform semantic segmentation of the airway and airway-related tissues on the features for semantic segmentation to obtain the first segmentation result.

[0057] In addition, to further improve the effectiveness of the segmentation network and enable it to better assist in the training of the classification network, the segmentation network can also include an AFF module. The AFF module is used to perform weighted processing on the output of the current decoding block and the output of the corresponding encoding block, and to concatenate the weighted processing result with the output of the current decoding block to obtain a concatenated result, which serves as the input for the next block of the current decoding block.

[0058] The classification network may include at least one encoding block, an attention block, and a classification block. The encoding block of the classification network has the same structure as the encoding block of the segmentation network. The classification network may use the at least one encoding block to extract a first feature of the sample image sequence, use the attention module to adjust the first feature based on the processing result to change the attention paid to the first feature, and use the classification block to classify the adjusted first feature to obtain a first classification result.

[0059] In the training architecture that uses the above-mentioned segmentation network to assist the above-mentioned classification network, the first processing result of the segmentation network is the output of the first decoding block (the second feature) or the output of the semantic segmentation block (the first segmentation result), that is, the output of the first decoding block or the output of the semantic segmentation block of the segmentation network can be used to assist the training of the classification network.

[0060] Combine Figure 2 For example, Figure 2 This is a structural diagram of the classification network and the segmentation network.

[0061] like Figure 2 As shown in the figure, the segmentation network includes four sequentially connected encoding blocks (Encoderblock1-4), four decoding blocks (Decoderblock1-4), a semantic segmentation block (Segmentation), and AFF modules 1-4. The input of AFF module 1 is connected to Encoderblock4 and Decoderblock1, and its output is connected to Decoderblock2. The input of AFF module 2 is connected to Encoderblock3 and Decoderblock2, and its output is connected to Decoderblock3. The input of AFF module 3 is connected to Encoderblock2 and Decoderblock3, and its output is connected to Decoderblock4. The input of AFF module 4 is connected to Encoderblock1 and Decoderblock4, and its output is connected to Segmentation. The classification network includes four sequentially connected encoding blocks (Encoderblock1-4), an attention block (Attention), and a classification block (Classification). In the segmentation network, Decoderblock1 is connected to Attention.

[0062] Therefore, the classification network can obtain the first feature through Encoderblock1~4, and the segmentation network can obtain the second feature through Encoderblock1~4 and Decoderblock1. The Attention of the classification network processes the first and second features and then sends them to Classification for classification to obtain the airway category.

[0063] The following describes the encoding block and AFF module:

[0064] Each encoder block can include at least one encoding sub-block, a perturbation layer, a recalibration layer and a downsampling layer. Each encoding sub-block includes a cascaded three-dimensional convolution layer, an instance normalization layer and an activation layer. Figure 3For example, each encoding block includes two encoding sub-blocks, a perturbation layer (Disout), a recalibration layer (Feature Realization), and downsampling (maxpooling (not shown)). An encoding sub-block includes a three-dimensional convolution layer (3D conv), an instance normalization layer (InstanceNorml3d), and an activation layer (PRelu). The channel data of the convolution kernels of the two 3D conv layers gradually increases with the feature extraction stage (n).

[0065] The structure of the decoding block Decoderblock is similar to that of Encoderblock. The difference from Encoderblock is that it includes an upsampling layer instead of a downsampling layer, and the number of channels of its convolution kernel is equal to the number of channels of the convolution and encoder block.

[0066] The perturbation layer is used to perturb the features to enhance the generalization of the model. The recalibration layer is used to filter information. Figure 4 The implementation of the recalibration layer is explained. , through Z (.) along the depth, height, and width of the three dimensions to integrate the spatial mapping :

[0067]

[0068]

[0069]

[0070] Secondly, channel reorganization is performed on the spatial map to obtain the channel description operator :

[0071]

[0072] Finally, the channel description operator and the features obtained by the perturbation layer Multiply to get the result of the recalibration layer:

[0073]

[0074] Where B is the broadcast mechanism, are the learnable parameters, is the activation function, It is a 1*1*1 3D convolution layer. It is a point-by-point multiplication operation.

[0075] Combine Figure 5-6The AFF module is described by taking the example of AFF module 1 performing weighted processing on output X of encoder block 4 and output Y of decoder block 1.

[0076] like Figure 5 As shown, AFF module 2 first adds X and Y point by point and then inputs them into MS-CAM block for processing. Figure 6 As shown in the figure, MS-CAM consists of two parts, namely the local attention part and the global attention part. In the local attention part, the result of adding X and Y is sequentially processed with point-wise convolution (Point-wise Conv) dimensionality reduction, nonlinear transformation (ReLU), and point-wise convolution (Point-wise Conv) dimensionality increase to suppress the information irrelevant to the classification in the result of adding X and Y and strengthen the information relevant to the classification. In the global attention part, unlike the local attention part, before performing point-wise convolution (Point-wise Conv) dimensionality reduction, the result of adding X and Y is subjected to global average pooling (GlobalAvgPooling). The MS-CAM block adds the results of the local attention part and the global attention part, and obtains the result after Sigmoid, i.e., the weight AFF module 2 is based on Weight X and Y to get the weighted result :

[0077] .

[0078] The classification block Classification consists of a 3DConv layer with a convolution kernel of 1*1*1 and a Sigmoid layer.

[0079] The semantic segmentation block Segmentation consists of a 3DConv layer with a convolution kernel of 1*1*1 and a Softmax layer.

[0080] Figure 7 This is a flow chart of the second embodiment of the training method of the classification network of the present application. It should be noted that if there are substantially the same results, this embodiment does not use Figure 7 The process sequence shown is limited. Figure 7 As shown, based on the above structure, if the above processing result is the first segmentation result, then the above S12 using the segmentation network to process the sample image sequence to obtain the processing result may include the following sub-steps:

[0081] S21: sequentially processing the sample image sequence using at least one encoding block and at least one decoding block to obtain features for semantic segmentation.

[0082] To simplify the description, the following description is made based on the case where the segmentation network includes a first encoding block and a second encoding block, a first decoding block and a second decoding block, a first AFF module and a second AFF module. Figure 8 , S21 may include the following sub-steps:

[0083] S211: Encode the sample image sequence using a first encoding block to obtain a first encoding result.

[0084] S212: Encode the first encoding result using the second encoding block to obtain a second encoding result.

[0085] S213: Decode the second encoding result using the first decoding block to obtain a first decoding result.

[0086] S214: Using the first AFF module, weight the first decoding result and the second encoding result to obtain a first weighted result.

[0087] S215: Using a second decoding block, decode the concatenation result of the first weighted result and the first decoding result to obtain a second decoding result.

[0088] S216: Using the second AFF module, weight the second decoding result and the first encoding result to obtain a second weighted result.

[0089] A concatenation result of the second weighted result and the second decoding result is a feature for semantic segmentation.

[0090] S22: Using the semantic segmentation block, perform semantic segmentation of the airway and airway-related tissues on the features used for semantic segmentation to obtain a first segmentation result.

[0091] For other detailed descriptions of this embodiment, please refer to the previous embodiments and will not be repeated here.

[0092] Based on the above structure, the following describes the training process of the segmentation network on the training set before using the segmentation network to process the sample image sequence to obtain the processing results in the above S12. During the stage of training the segmentation network alone, the segmentation network includes a semantic segmentation block.

[0093] Figure 9 This is a flow chart of the third embodiment of the training method of the classification network of the present application. It should be noted that if there are substantially the same results, this embodiment does not use Figure 9 The process sequence shown is limited. Figure 9 As shown, training the segmentation network on the training set can include the following sub-steps:

[0094] S31: Using a segmentation network to perform semantic segmentation of the airway and airway-related tissues on the sample image sequence to obtain a second segmentation result.

[0095] S32: Adjust parameters of the segmentation network based on the second segmentation result.

[0096] The classification network trained by the above training method can be put into practical application.

[0097] Figure 10 It is a flowchart of the fourth embodiment of the image classification method of the present application. It should be noted that if there are substantially the same results, this embodiment does not use Figure 10 The process sequence shown is limited. Figure 10 As shown, this embodiment may include:

[0098] S41: Acquire a target image sequence.

[0099] The target image sequence includes a plurality of target images, each target image corresponds to a different position of the target airway, and each target image includes at least a portion of the target airway and related tissues.

[0100] S42: Input the target image sequence into the classification network to obtain the classification result of the target image sequence.

[0101] The classification result of the target image sequence is used to represent the category of the target's airway.

[0102] The classification network can be trained based on the aforementioned training method.

[0103] For a detailed description of this embodiment, please refer to the previous embodiment and will not be repeated here.

[0104] Through the implementation of this embodiment, the present application can classify the target's airway based on the target's airway-related image sequence through a classification network to determine the category of the target's airway.

[0105] Figure 11 This is a schematic diagram of the structure of an embodiment of the electronic device of the present application. Figure 11 As shown, the electronic device may include a processor 51 and a memory 52 coupled to the processor 51 .

[0106] The memory 52 stores program instructions for implementing any of the above-described method embodiments; the processor 51 is configured to execute the program instructions stored in the memory 52 to implement the steps of the above-described method embodiments. The processor 51 may also be referred to as a CPU (Central Processing Unit). The processor 51 may be an integrated circuit chip with signal processing capabilities. The processor 51 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. A general-purpose processor may be a microprocessor, or the processor 51 may be any conventional processor.

[0107] Figure 12 This is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of the present application. Figure 12 As shown, the computer-readable storage medium 60 of the embodiment of the present application stores program instructions 61, which, when executed, implement the method provided in the above embodiment of the present application. The program instructions 61 can be stored in the computer-readable storage medium 60 in the form of a program file in the form of a software product, so that a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) can execute all or part of the steps of the method of each embodiment of the present application. The aforementioned computer-readable storage medium 60 includes: various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or terminal devices such as a computer, server, mobile phone, and tablet.

[0108] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0109] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. The above is only an implementation method of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the content of the description and drawings of this application, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A training method for a classification network, characterized in that: include: Acquiring a training set, the training set comprising a plurality of sample image sequences, each of the sample image sequences comprising a plurality of sample images, each of the sample images corresponding to a different position of the airway of a living subject, and each of the sample images comprising at least a portion of the airway and related tissues; Extracting a first feature of the sample image sequence using the classification network, and processing the sample image sequence using a segmentation network to obtain a processing result, wherein the segmentation network is trained on the training set, and during the training of the segmentation network, the segmentation network is used to perform semantic segmentation on the airway and airway-related tissue corresponding to the sample image sequence, and the processing result is the first segmentation result of the sample image sequence or the second feature of the sample image sequence; Obtaining, using the classification network, an attention weight corresponding to the first feature based on the processing result; Multiplying the first feature and the attention weight using the classification network to obtain a multiplied feature; Using a classification network to classify the sample image sequence based on the multiplied features to obtain a first classification result, where the first classification result is used to represent the category of the airway; Parameters of the classification network are adjusted based on the first classification result.

2. The training method according to claim 1, characterized in that The obtaining, using the classification network based on the processing result, an attention weight corresponding to the first feature includes: Using the classification network to obtain a convolution kernel suitable for the processing result; The classification network is used to convolve the processing result based on the convolution kernel to obtain the attention weight.

3. The training method according to claim 1, characterized in that The segmentation network includes at least one encoding block and at least one decoding block.

4. The training method according to claim 3, characterized in that The step of acquiring the second feature includes: The sample image sequence is processed in sequence using the at least one encoding block and the at least one decoding block to obtain the second feature.

5. The training method according to claim 3, characterized in that: The segmentation network further includes a semantic segmentation block, and the step of obtaining the first segmentation result includes: sequentially processing the sample image sequence using the at least one encoding block and the at least one decoding block to obtain features for semantic segmentation; The semantic segmentation block is used to perform semantic segmentation on the features used for semantic segmentation to obtain the first segmentation result.

6. The training method according to claim 5, characterized in that The at least one encoding block includes a first encoding block and a second encoding block, the at least one decoding block includes a first decoding block and a second decoding block, the segmentation network further includes a first AFF module and a second AFF module, and the sequentially using the at least one encoding block, the at least one decoding block, and the semantic segmentation to process the sample image sequence to obtain features for semantic segmentation includes: Encoding the sample image sequence using the first encoding block to obtain a first encoding result; Encoding the first encoding result using the second encoding block to obtain a second encoding result; Decoding the second encoding result using the first decoding block to obtain a first decoding result; weighting the first decoding result and the second encoding result using the first AFF module to obtain a first weighted result; Decoding a concatenation of the first weighted result and the first decoding result using the second decoding block to obtain a second decoding result; The second decoding result and the first encoding result are weighted by the second AFF module to obtain a second weighted result, and a concatenation result of the second weighted result and the second decoding result is the feature for semantic segmentation.

7. The training method according to claim 1, characterized in that Before the sample image sequence is processed by the segmentation network to obtain a processing result, the following steps are included for training the segmentation network: performing semantic segmentation of the airway and the airway-related tissue on the sample image sequence using the segmentation network to obtain a second segmentation result; Parameters of the segmentation network are adjusted based on the second segmentation result.

8. The training method according to claim 1, characterized in that: The classification network includes at least one encoding block, and extracting the first feature of the sample image sequence using the classification network includes: A first feature of the sample image sequence is extracted using the at least one coding block.

9. The training method according to claim 8, characterized in that The encoding block includes at least one encoding sub-block, a perturbation layer, a recalibration layer, and a downsampling layer, the encoding sub-block includes a three-dimensional convolution layer, an instance normalization layer, and an activation layer, and extracting the first feature of the sample image sequence using the at least one encoding block includes: The sample image sequence is processed in sequence using the at least one coding sub-block, the perturbation layer, the recalibration layer, and the downsampling layer to obtain the first feature.

10. The training method according to claim 1, characterized in that: The sample image sequence is a CT sample image sequence, and / or the airway-related tissue includes tongue root and fat.

11. An image classification method, characterized in that: include: Acquire a target image sequence, wherein the target image sequence includes a plurality of target images, each target image corresponds to a different position of the target airway, and each sample image includes at least a portion of the target airway and related tissues; Inputting the target image sequence into a classification network to obtain a classification result of the target image sequence, wherein the classification result of the target image sequence is used to represent the category of the airway of the target; Wherein, the classification network is trained using any one of the methods in claims 1-10.

12. An electronic device, characterized in that: comprising a processor and a memory connected to the processor, wherein: The memory stores program instructions; The processor is configured to execute the program instructions stored in the memory to implement the method according to any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that The storage medium stores program instructions, and when the program instructions are executed, the method according to any one of claims 1 to 11 is implemented.

Images