Method, apparatus and device for determining bronchial tuberculosis typing

Abstract

The present application discloses a method, device and equipment for determining the type of bronchial tuberculosis, which relates to the field of artificial intelligence-assisted diagnosis. The method includes: obtaining a data set; the data set is an image sample under bronchoscopy; constructing a bronchial tuberculosis diagnosis model; the bronchial tuberculosis diagnosis model is a bronchial tuberculosis diagnosis model based on the ResNet34 framework and incorporating multi-head self-attention and depthwise separable convolution; training the bronchial tuberculosis diagnosis model based on the data set; inputting the image of the user under bronchoscopy into the trained bronchial tuberculosis diagnosis model to obtain the type of bronchial tuberculosis. The present application can realize the intelligent diagnosis of bronchial tuberculosis through an artificial intelligence-assisted diagnosis system, and can effectively reduce the misdiagnosis and missed diagnosis of bronchial tuberculosis.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence-assisted diagnosis, and particularly to a method, device and equipment for determining the type of bronchial tuberculosis. Background Art

[0002] Tuberculosis is one of the world's largest infectious disease "killers". Currently, more than 60% of patients have serious complications such as atelectasis, bronchial stenosis, pulmonary function damage, and pulmonary destruction, and even need to undergo lobectomy to be cured. This is mainly because bronchial tuberculosis has no specific clinical symptoms and imaging manifestations, and bronchoscopy requires experienced doctors to give accurate diagnoses. Therefore, bronchial tuberculosis is often misdiagnosed and missed, resulting in the loss of the best treatment opportunity. Therefore, if an intelligent diagnosis of bronchial tuberculosis can be achieved through an artificial intelligence-assisted diagnosis system, it can effectively reduce the misdiagnosis and missed diagnosis of bronchial tuberculosis, detect and treat it early, and reduce the risk of tuberculosis transmission. Summary of the Invention

[0003] The purpose of the present application is to provide a method, device and equipment for determining the type of bronchial tuberculosis, which can achieve the intelligent diagnosis of bronchial tuberculosis through an artificial intelligence-assisted diagnosis system, and can effectively reduce the misdiagnosis and missed diagnosis of bronchial tuberculosis.

[0004] To achieve the above purpose, the present application provides the following solutions:

[0005] In a first aspect, the present application provides a method for determining the type of bronchial tuberculosis, including:

[0006] Obtain a data set; the data set is an image sample under bronchoscopy;

[0007] Construct a bronchial tuberculosis diagnosis model; the bronchial tuberculosis diagnosis model is a bronchial tuberculosis diagnosis model based on the ResNet34 framework and incorporating multi-head self-attention and depthwise separable convolution;

[0008] Train the bronchial tuberculosis diagnosis model based on the data set;

[0009] Input the image of the user under bronchoscopy into the trained bronchial tuberculosis diagnosis model to obtain the type of bronchial tuberculosis.

[0010] Optionally, training the bronchial tuberculosis diagnosis model based on the data set specifically includes the following steps:

[0011] Input the image sample under bronchoscopy into the bronchial tuberculosis diagnosis model to obtain the model output;

[0012] Use the cross-loss function to calculate the difference between the output of the model and the true label to obtain the loss;

[0013] Calculate the gradient of the loss with respect to the parameters of the bronchial tuberculosis diagnosis model, and propagate the gradient of the loss from the output layer to the input layer through the chain rule;

[0014] Use an optimizer to update the parameters of the bronchial tuberculosis diagnosis model according to the gradient of the loss, and finally obtain the trained bronchial tuberculosis diagnosis model.

[0015] Optionally, the bronchial tuberculosis diagnosis model specifically includes:

[0016] A 7×7 convolutional layer, a pooling layer, a first residual module group, a second residual module group, a third residual module group, a fourth residual module group, a global average pooling layer, and a fully connected layer;

[0017] The 7×7 convolutional layer, the pooling layer, the first residual module group, the second residual module group, the third residual module group, the fourth residual module group, the global average pooling layer, and the fully connected layer are connected in sequence.

[0018] Optionally, the first residual module group includes a first ordinary residual block, a second ordinary residual block, and a third ordinary residual block. The first ordinary residual block, the second ordinary residual block, and the third ordinary residual block each include two 3×3 convolutional layers, and the first ordinary residual block, the second ordinary residual block, and the third ordinary residual block are connected in sequence;

[0019] The second residual module group includes: a first depthwise separable convolutional residual block, a fourth ordinary residual block, a fifth ordinary residual block, and a sixth ordinary residual block; the fourth ordinary residual block, the fifth ordinary residual block, and the sixth ordinary residual block each include two 3×3 convolutional layers;

[0020] The first depthwise separable convolutional residual block, the fourth ordinary residual block, the fifth ordinary residual block, and the sixth ordinary residual block are connected in sequence;

[0021] The third residual module group includes: a second depthwise separable convolutional residual block, a seventh ordinary residual block, an eighth ordinary residual block, a ninth ordinary residual block, a tenth ordinary residual block, and an eleventh ordinary residual block; the seventh ordinary residual block, the eighth ordinary residual block, the ninth ordinary residual block, the tenth ordinary residual block, and the eleventh ordinary residual block each include two 3×3 convolutional layers;

[0022] The second depthwise separable convolutional residual block, the seventh ordinary residual block, the eighth ordinary residual block, the ninth ordinary residual block, the tenth ordinary residual block, and the eleventh ordinary residual block are connected in sequence;

[0023] The fourth residual module group includes: a third depthwise separable convolutional residual block, a first multi-head self-attention mechanism residual block, and a second multi-head self-attention mechanism residual block;

[0024] The third depthwise separable convolutional residual block, the first multi-head self-attention mechanism residual block, and the second multi-head self-attention mechanism residual block are connected in sequence.

[0025] Optionally, the first ordinary residual block, the second ordinary residual block, the third ordinary residual block, the fourth ordinary residual block, the fifth ordinary residual block, the sixth ordinary residual block, the seventh ordinary residual block, the eighth ordinary residual block, the ninth ordinary residual block, the tenth ordinary residual block, and the eleventh ordinary residual block all adopt the following calculation formula:

[0026] y = F(x, {W i}) + W 1x1 *x

[0027] where y represents the output, F(x, {W i}) = ReLU(BN(W2 * ReLU(BN(W1 * x)))), x represents the input, BN represents batch normalization operation, ReLU represents non-linear calculation, W i where i = 1, 2, W1 represents the first convolution operation and its weight, W2 represents the second convolution operation and its weight, W 1×1 represents the 1*1 convolution operation and its weight.

[0028] Optionally, the first depthwise separable convolutional residual block, the second depthwise separable convolutional residual block, and the third depthwise separable convolutional residual block all adopt the following calculation formula:

[0029] y = F(x, {W i}) + W pw *(W dw *x)

[0030] where W pw represents, W dw represents.

[0031] Optionally, the first multi-head self-attention mechanism residual block and the second multi-head self-attention mechanism residual block all adopt the following calculation formula:

[0032] y = ReLU(BN(W 3×3 *MHSA(X))) + W 1×1 *x

[0033] where MHSA(X) = Concat(O1, O2,..., O h )W o , W 3×3 represents the 3*3 convolution operation and its weight, Concat represents the merging function.

[0034] Optionally, the fully connected layer specifically adopts the following formula:

[0035]

[0036] where p i represents the probability of the i-th category, zi represents the i-th element of the output z of the linear transformation, and e represents the natural constant.

[0037] In a second aspect, the present application provides a device for determining the typing of bronchial tuberculosis, including:

[0038] A dataset acquisition module for acquiring a dataset; the dataset is an image sample under bronchoscopy;

[0039] A bronchial tuberculosis diagnosis model construction module for constructing a bronchial tuberculosis diagnosis model; the bronchial tuberculosis diagnosis model is a bronchial tuberculosis diagnosis model based on the ResNet34 framework and incorporating multi-head self-attention and depthwise separable convolution;

[0040] A training module for training the bronchial tuberculosis diagnosis model based on the dataset;

[0041] A typing determination module for bronchial tuberculosis, which inputs the image of the user under bronchoscopy into the trained bronchial tuberculosis diagnosis model to obtain the typing of bronchial tuberculosis.

[0042] In a third aspect, the present application provides a computer device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor executes the computer program to implement the steps of the method for determining the typing of bronchial tuberculosis described in any one of the above.

[0043] According to the specific embodiments provided by the present application, the following technical effects are disclosed:

[0044] The present application provides a method, apparatus, and device for determining the type of bronchial tuberculosis. By introducing depthwise separable convolution, a large amount of computation in traditional convolution operations is split into two smaller computation steps: depthwise convolution and pointwise convolution. This reduces the number of model parameters and the amount of computation in the convolution operation, improves the classification performance of the model, accelerates the training speed of the model, and effectively reduces the computational burden of the model. In the second and third residual blocks of the fourth residual block group in the ResNet34 network, the second convolution is replaced with a multi-head self-attention mechanism, enabling the model to simultaneously focus on the global and local features of bronchial images, improving the accuracy of the model, and the accuracy rate can reach nearly 90%. In addition, the use of a dual USB foot pedal ensures the simultaneous use of the artificial intelligence-assisted diagnosis system for bronchial tuberculosis and the hospital bronchoscope reporting system without mutual interference. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0046] Figure 1 It is a schematic flowchart of a method for determining the type of bronchial tuberculosis provided by an embodiment of the present application;

[0047] Figure 2 It is a schematic diagram of the interface of the artificial intelligence-assisted diagnosis system for bronchial tuberculosis according to an embodiment of the present application;

[0048] Figure 3 It is a schematic working principle diagram of the artificial intelligence-assisted diagnosis system for bronchial tuberculosis according to an embodiment of the present application;

[0049] Figure 4 It is a structural diagram of a bronchial tuberculosis diagnosis model based on the ResNet34 framework incorporating multi-head self-attention and depthwise separable convolution according to an embodiment of the present application;

[0050] Figure 5 It is a schematic diagram of the structure of a depthwise separable convolution residual block according to an embodiment of the present application;

[0051] Figure 6 It is a schematic diagram of the structure of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0052] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0053] To make the above objects, features, and advantages of the present application more obvious and understandable, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0054] The artificial intelligence-assisted diagnosis system for bronchial tuberculosis is an intelligent diagnosis system that can be used in conjunction with a bronchoscope to identify bronchial tuberculosis and give typing prompts. Doctors only need to open the software and then click the "Open Video" button. At this time, the images of the hospital's endoscope workstation will be synchronized in real time in the video detection area in the upper left corner of the software. When a doctor performs a bronchoscope examination on a patient and discovers a suspected bronchial tuberculosis lesion, stepping on the foot pedal can collect the current picture and send it to the hospital's PACS system and the artificial intelligence-assisted diagnosis system for bronchial tuberculosis at the same time. When the artificial intelligence-assisted diagnosis system for bronchial tuberculosis detects the signal transmitted by the medical foot pedal, it will intercept the current video frame, display the intercepted image in the EBTB image diagnosis area in the upper right corner, automatically perform intelligent diagnosis on the image, and display positive judgments and typing prompts in the diagnosis result display area on the right side of the screen, such as Figure 2 shown. It helps doctors in diagnosis and reduces the workload of doctors.

[0055] See Figure 3 , which is the working principle diagram of the artificial intelligence-assisted diagnosis system for bronchial tuberculosis. The endoscope video is connected to the high-definition data acquisition card through an HDMI high-definition data cable. When the foot pedal is stepped on, the current frame image of the video is collected and saved in the memory of the workstation. The trained auxiliary diagnosis model for bronchial tuberculosis reads the currently collected image and automatically gives whether it is bronchial tuberculosis. If it is, it will automatically give typing prompts. If it is not bronchial tuberculosis, it will prompt type 0.

[0056] The system uses a dual-USB foot pedal. The control signal of the foot pedal controls the high-definition data acquisition card of the auxiliary diagnosis system on the one hand and the original PACS image acquisition of the hospital on the other hand. The hospital's PACS system is a system used by a hospital for bronchial data acquisition and diagnosed by doctors. It and the artificial intelligence-assisted diagnosis system for bronchial tuberculosis in the present application are two independent systems without mutual interference.

[0057] Figure 1 is a schematic flowchart of a method for determining the type of bronchial tuberculosis provided by an embodiment of the present application. As Figure 1 shown, the method includes:

[0058] Step 101: Obtain a dataset; the dataset is an image sample under a bronchoscope.

[0059] Specifically, clinical data was collected in the hospital to establish an image sample database with more than 20,000 bronchoscopic images.

[0060] Then, the above data was preprocessed, including scaling, normalization, and data augmentation, so that the number of various types of bronchial tuberculosis pictures in the database was balanced to ensure the training effect.

[0061] Step 102: Construct a bronchial tuberculosis diagnosis model; the bronchial tuberculosis diagnosis model is a bronchial tuberculosis diagnosis model based on the ResNet34 framework and incorporating multi-head self-attention and depthwise separable convolution.

[0062] See Figure 4 , Figure 4 For the bronchial tuberculosis diagnosis model structure based on the ResNet34 framework incorporating multi-head self-attention and depthwise separable convolution, it specifically includes:

[0063] 7×7 convolutional layer, pooling layer, first residual module group, second residual module group, third residual module group, fourth residual module group, global average pooling layer, and fully connected layer;

[0064] Among them, the 7×7 convolutional layer, pooling layer, first residual module group, second residual module group, third residual module group, fourth residual module group, global average pooling layer, and fully connected layer are connected in sequence.

[0065] The first residual module group includes a first ordinary residual block, a second ordinary residual block, and a third ordinary residual block. The first ordinary residual block, the second ordinary residual block, and the third ordinary residual block all include two 3×3 convolutional layers, and the first ordinary residual block, the second ordinary residual block, and the third ordinary residual block are connected in sequence;

[0066] The second residual module group includes: a first depthwise separable convolutional residual block, a fourth ordinary residual block, a fifth ordinary residual block, and a sixth ordinary residual block; the fourth ordinary residual block, the fifth ordinary residual block, and the sixth ordinary residual block all include two 3×3 convolutional layers;

[0067] The first depthwise separable convolutional residual block, the fourth ordinary residual block, the fifth ordinary residual block, and the sixth ordinary residual block are connected in sequence;

[0068] The third residual module group includes: a second depthwise separable convolutional residual block, a seventh ordinary residual block, an eighth ordinary residual block, a ninth ordinary residual block, a tenth ordinary residual block, and an eleventh ordinary residual block; the seventh ordinary residual block, the eighth ordinary residual block, the ninth ordinary residual block, the tenth ordinary residual block, and the eleventh ordinary residual block each include two 3×3 convolutional layers;

[0069] The second depthwise separable convolutional residual block, the seventh ordinary residual block, the eighth ordinary residual block, the ninth ordinary residual block, the tenth ordinary residual block, and the eleventh ordinary residual block are connected in sequence;

[0070] The fourth residual module group includes: a third depthwise separable convolutional residual block, a first multi-head self-attention mechanism residual block, and a second multi-head self-attention mechanism residual block;

[0071] The third depthwise separable convolutional residual block, the first multi-head self-attention mechanism residual block, and the second multi-head self-attention mechanism residual block are connected in sequence.

[0072] See Figure 5 , where the structures of the first depthwise separable convolutional residual block, the second depthwise separable convolutional residual block, and the third depthwise separable convolutional residual block all include:

[0073] A DW conv layer, a BN layer, a ReLU layer, and a PW conv layer connected in sequence.

[0074] Among them, the first ordinary residual block, the second ordinary residual block, the third ordinary residual block, the fourth ordinary residual block, the fifth ordinary residual block, the sixth ordinary residual block, the seventh ordinary residual block, the eighth ordinary residual block, the ninth ordinary residual block, the tenth ordinary residual block, and the eleventh ordinary residual block all adopt the following calculation formula:

[0075] y = F(x,{W i}) + W 1×1 *x(1)

[0076] Among them, y represents the output, F(x,{W i}) = ReLU(BN(W2*ReLU(BN(W1*x)))), x represents the input, BN represents batch normalization operation, ReLU represents non-linear calculation, ReLU(x) = max(0,x). Max is the maximum value operation, W i where i = 1, 2, W1 represents the first convolutional operation and weight, W2 represents the second convolutional operation and weight, W 1×1 represents the 1*1 convolutional operation and weight.

[0077] The first depthwise separable convolutional residual block, the second depthwise separable convolutional residual block, and the third depthwise separable convolutional residual block all adopt the following calculation formula:

[0078] y = F(x, {W i}) + W pw *(W dw *x) (2)

[0079] where, W pw denotes, W dw denotes.

[0080] The first multi-head self-attention mechanism residual block and the second multi-head self-attention mechanism residual block both adopt the following calculation formula:

[0081] y = ReLU(BN(W 3×3 *MHSA(X))) + W 1×1 *x (3)

[0082] where, MHSA(X) = Concat(O1, O2,..., O h )W o , W 3×3 denotes W 1×1 denotes the 3*3 convolutional operation and weight, Concat represents the merging function that merges the outputs of self-attention calculated in parallel by multiple heads on different sub-controls, MHSA(X) = Convat(O1, O2,..., O h )W o , O i = Attention(O1, O1, O1).

[0083] The fully connected layer specifically adopts the following formula:

[0084]

[0085] where, p i denotes the probability of the i-th category, zi denotes the i-th element of the output z of the linear transformation, e denotes the natural constant, z = Wh + b, h is the input feature vector with a dimension of 512, W is the weight matrix of the fully connected layer with a dimension of 4×512, b is the bias vector of the fully connected layer with a dimension of 4, and is converted into a probability distribution through the Softmax function: p = Softmax(z).

[0086] Step 103: Train the bronchial tuberculosis diagnosis model based on the dataset.

[0087] The specific training process is as follows:

[0088] Input the image sample under bronchoscope into the bronchial tuberculosis diagnosis model to obtain the model output;

[0089] Use the cross - entropy loss function to calculate the difference between the output of the model and the true label to obtain the loss;

[0090] Calculate the gradient of the loss with respect to the parameters of the bronchial tuberculosis diagnosis model, and propagate the gradient of the loss from the output layer to the input layer through the chain rule;

[0091] Use the optimizer to update the parameters of the bronchial tuberculosis diagnosis model according to the gradient of the loss, and finally obtain the trained bronchial tuberculosis diagnosis model.

[0092] The model training parameter configuration is shown in Table 1:

[0093] Table 1 Model training parameter configuration

[0094]

[0095] Step 104: Input the image of the user under bronchoscope into the trained bronchial tuberculosis diagnosis model to obtain the classification of bronchial tuberculosis.

[0096] Specifically, the internal processing process of the bronchial tuberculosis diagnosis model is as follows:

[0097] ① The image information of 224 * 224 pixels first undergoes conv(7, 7, cin = 3, cout = 64, padding = 3, stride = 2) convolution operation through a 7x7 convolutional layer. Through the convolution operation, the initial features of the image can be extracted in a larger field of view, reducing the image size by half and increasing the depth of the feature map, obtaining a feature matrix of 64 * 112 * 112.

[0098] ② Input the 64 * 112 * 112 feature matrix obtained in step ① into the pooling layer (pool layer) for conv(3, 3, cin = 3, cout = 64, padding = 1, stride = 2) convolution operation, reducing the size by half, obtaining a feature matrix of 64 * 56 * 56. The role of the pooling layer is to highlight significant features and reduce the computational amount of subsequent convolutional layers, thereby improving the overall computational efficiency of the network.

[0099] ③ Input the 64*56*56 feature matrix obtained in step ② into Residual Module Group 1. This module group contains 3 Residual Blocks, and each block has two 3x3 convolutional layers. The input and output of each residual block are added together to form a residual connection. The specific operation of each residual block is shown in Formula 1. After the operation of this module, a 64*56*56 feature matrix is obtained. The main function of this module is to further extract and enhance features, and at the same time reduce the problem of gradient disappearance through residual connections.

[0100] ④ Input the feature matrix obtained in step ③ into Residual Module Group 2. This module contains 4 residual blocks, and each block is still composed of two 3x3 convolutional layers. The first residual block is a residual block with depthwise separable convolution, and the calculation method is shown in Formula 3. The remaining 3 residual blocks are ordinary residual blocks, and the calculation method is shown in Formula 1. After Residual Module Group 2, a 128*28*28 feature matrix is output. Compared with Residual Module Group 1, the feature extraction in this stage is more in-depth, and more complex features are learned through more residual blocks.

[0101] ⑤ Input the feature matrix obtained in step ④ into Residual Module Group 3. This module contains 3 residual blocks, and each block is still composed of two 3x3 convolutional layers. The first residual block is a residual block with depthwise separable convolution, and the second and third residual blocks are calculated as shown in Formula 2. The remaining 5 residual blocks are ordinary residual blocks, and the calculation method is shown in Formula 1. After Residual Module Group 3, a 256*14*14 feature matrix is output. Residual Module Group 3 is the core part of feature extraction, and high-level features are extracted through a large number of residual blocks.

[0102] ⑥ Input the image obtained in step ⑤ into Residual Module Group 4. This module contains 3 residual blocks. The first residual block is a residual block with depthwise separable convolution, and the calculation method is shown in Formula 2. The remaining 2 residual blocks are residual blocks with multi-head self-attention mechanism (MHSA), that is, the second convolutional layer of them introduces MHSA, and the calculation method is shown in Formula 3. Introducing MHSA is to capture the relationships and context information between distant features, enhance the global nature of feature representation, and improve the expressive ability of features. After Residual Module Group 4, a 512*7*7 feature matrix is output. Residual Module Group 4 integrates and classifies the previously extracted features, enabling the model to simultaneously focus on the global and local features of the image and improve the accuracy of the model.

[0103] ⑦ Input the feature matrix obtained in step ⑥ into the global average pooling layer (avg pool) to output a 512*1*1 feature vector. The global average pooling layer extracts global features through dimensionality reduction, replaces the fully connected layer, smooths the feature map, and promotes the interpretability of the feature map.

[0104] ⑧Based on the feature vectors output in step ⑦, the classification task is achieved through a fully connected layer, and finally the bronchial tuberculosis pictures are typed.

[0105] EBTB is divided into 6 types according to its manifestations under bronchoscopy during the disease progression: type Ⅰ (inflammatory infiltration type), type Ⅱ (ulcero-necrotic type), type Ⅲ (granulation proliferation type), type Ⅳ (scar stenosis type), type V (softening of the bronchial wall type), type Ⅵ (lymph node fistula type).

[0106] Based on the same inventive concept, the embodiment of the present application also provides a bronchial tuberculosis typing determination device for implementing the bronchial tuberculosis typing determination method involved above. The implementation solution provided by this device to solve the problem is similar to the implementation solution recorded in the above method. Therefore, the specific limitations in one or more embodiments of the bronchial tuberculosis typing determination device provided below can refer to the limitations on the bronchial tuberculosis typing determination method in the above text, and will not be elaborated here.

[0107] In an exemplary embodiment, a bronchial tuberculosis typing determination device is provided, including:

[0108] A dataset acquisition module, configured to acquire a dataset; the dataset is an image sample under bronchoscopy.

[0109] A bronchial tuberculosis diagnosis model construction module, configured to construct a bronchial tuberculosis diagnosis model; the bronchial tuberculosis diagnosis model is a bronchial tuberculosis diagnosis model based on the ResNet34 framework and incorporating multi-head self-attention and depthwise separable convolution.

[0110] A training module, configured to train the bronchial tuberculosis diagnosis model based on the dataset.

[0111] A bronchial tuberculosis typing determination module, configured to input the image of the user under bronchoscopy into the trained bronchial tuberculosis diagnosis model to obtain the typing of bronchial tuberculosis.

[0112] In an exemplary embodiment, a computer device is provided. This computer device can be a server or a terminal, and its internal structure diagram can be as Figure 6As shown in the figure. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data for determining the typing of bronchial tuberculosis. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a method for determining the typing of bronchial tuberculosis.

[0113] Those skilled in the art can understand that Figure 6 the structure shown in the figure is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0114] In an exemplary embodiment, a computer device is further provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0115] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0116] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0117] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0118] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include Read-Only Memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0119] The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0120] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0121] Specific examples are used in this article to elaborate on the principles and implementation manners of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A method for determining the typing of bronchial tuberculosis, characterized in that: The method for determining the typing of bronchial tuberculosis comprises: Acquire a data set; the data set is an image sample under bronchial endoscopy; Constructing a bronchial tuberculosis diagnosis model; the bronchial tuberculosis diagnosis model is a bronchial tuberculosis diagnosis model based on a ResNet34 framework and incorporating multi-head self-attention and deep separable convolution; Training the bronchial tuberculosis diagnosis model based on the data set; Input the user's bronchoscope image into the trained bronchial tuberculosis diagnosis model to obtain the classification of bronchial tuberculosis; The bronchial tuberculosis diagnostic model specifically includes: 7×7 convolutional layer, pooling layer, first residual module group, second residual module group, third residual module group, fourth residual module group, global average pooling layer and fully connected layer; The 7×7 convolutional layer, the pooling layer, the first residual module group, the second residual module group, the third residual module group, the fourth residual module group, the global average pooling layer and the fully connected layer are connected in sequence; The first residual module group includes a first common residual block, a second common residual block and a third common residual block, the first common residual block, the second common residual block and the third common residual block each include two 3×3 convolutional layers, and the first common residual block, the second common residual block and the third common residual block are connected in sequence; The second residual module group includes: a first depth-separable convolution residual block, a fourth common residual block, a fifth common residual block and a sixth common residual block; the fourth common residual block, the fifth common residual block and the sixth common residual block each include two 3×3 convolution layers; The first depthwise separable convolution residual block, the fourth common residual block, the fifth common residual block and the sixth common residual block are connected in sequence; The third residual module group includes: a second depth-separable convolution residual block, a seventh common residual block, an eighth common residual block, a ninth common residual block, a tenth common residual block and an eleventh common residual block; the seventh common residual block, the eighth common residual block, the ninth common residual block, the tenth common residual block and the eleventh common residual block each include two 3×3 convolutional layers; The second depthwise separable convolution residual block, the seventh common residual block, the eighth common residual block, the ninth common residual block, the tenth common residual block and the eleventh common residual block are connected in sequence; The fourth residual module group includes: a third depth-separable convolution residual block, a first multi-head self-attention mechanism residual block, and a second multi-head self-attention mechanism residual block; The third depth-separable convolution residual block, the first multi-head self-attention mechanism residual block and the second multi-head self-attention mechanism residual block are connected in sequence.

2. The method for determining bronchial tuberculosis typing according to claim 1, characterized in that: Training the bronchial tuberculosis diagnosis model based on the data set specifically includes the following steps: Inputting the image samples under bronchial endoscopy into the bronchial tuberculosis diagnosis model to obtain the model output; Use the cross loss function to calculate the difference between the output of the model and the true label to obtain the loss; Calculating the gradient of the loss with respect to the parameters of the bronchial tuberculosis diagnosis model, and propagating the gradient of the loss from the output layer to the input layer by the chain rule; The optimizer is used to update the parameters of the bronchial tuberculosis diagnosis model according to the gradient of the loss, and finally the trained bronchial tuberculosis diagnosis model is obtained.

3. The method for determining bronchial tuberculosis typing according to claim 1, characterized in that: The first common residual block, the second common residual block, the third common residual block, the fourth common residual block, the fifth common residual block, the sixth common residual block, the seventh common residual block, the eighth common residual block, the ninth common residual block, the tenth common residual block and the eleventh common residual block all adopt the following calculation formula: y=F(x,{W i })+W 1×1 *x Among them, y represents the output, F(x,{W i })=ReLU(BN(W2*ReLU(BN(W1*x)))), where x represents input, BN represents batch normalization operation, ReLU represents nonlinear calculation, and W i Where i = 1, 2, W1 represents the first convolution operation and weight, W2 represents the second convolution operation and weight, and W 1×1 Represents 1*1 convolution operation and weight.

4. The method for determining bronchial tuberculosis typing according to claim 3, characterized in that: The first depth-separable convolution residual block, the second depth-separable convolution residual block, and the third depth-separable convolution residual block all use the following calculation formula: y=F(x,{W i })+W pw *(W dw *x) Among them, W pw Represents channel-by-channel convolution operations and weights, W dw Represents point-by-point convolution operations and weights.

5. The method for determining bronchial tuberculosis typing according to claim 3, characterized in that: The first multi-head self-attention mechanism residual block and the second multi-head self-attention mechanism residual block both use the following calculation formula: y=ReLU(BN(W 3×3 *MHSA(X)))+W 1×1 *x Among them, MHSA(X)=Concat(O1,O2,…,O h )W o , W 3×3 It represents the 3*3 convolution operation and weight, and Concat represents the consolidation function.

6. The method for determining bronchial tuberculosis typing according to claim 1, characterized in that: The fully connected layer specifically adopts the following formula: Among them, p i represents the probability of the i-th category, z i represents the i-th element of the linear transformation output z, and e represents a natural constant.

7. A device for determining the typing of bronchial tuberculosis, applied to the method for determining the typing of bronchial tuberculosis according to any one of claims 1 to 6, characterized in that: The bronchial tuberculosis typing determination device comprises: A data set acquisition module, used to acquire a data set; the data set is an image sample under bronchial endoscopy; A bronchial combined diagnosis model construction module is used to construct a bronchial tuberculosis diagnosis model; the bronchial tuberculosis diagnosis model is a bronchial tuberculosis diagnosis model based on the ResNet34 framework and incorporating multi-head self-attention and deep separable convolution; A training module, used for training the bronchial tuberculosis diagnosis model based on the data set; The bronchial classification determination module is used to input the user's image under the bronchoscope into the trained bronchial tuberculosis diagnosis model to obtain the classification of bronchial tuberculosis.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method for determining the typing of bronchial tuberculosis according to any one of claims 1 to 6.

Images