Bronchial tuberculosis typing determination method, device and equipment

Abstract

The invention discloses a bronchial tuberculosis typing determination method, device and equipment, and relates to the field of artificial intelligence auxiliary diagnosis, and the method comprises the steps: obtaining a data set; the data set is an image sample under a bronchus endoscope; constructing a bronchial tuberculosis diagnosis model; the bronchial tuberculosis diagnosis model is a bronchial tuberculosis diagnosis model based on a ResNet34 framework and integrated with multi-head self-attention and depth separable convolution; training the bronchial tuberculosis diagnosis model based on the data set; and inputting the image of the user under the bronchoscope into the trained bronchial tuberculosis diagnosis model to obtain the type of the bronchial tuberculosis, so that intelligent diagnosis of the bronchial tuberculosis can be realized through an artificial intelligence auxiliary diagnosis system, and misdiagnosis and missed diagnosis of the bronchial tuberculosis can be effectively reduced.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence-assisted diagnosis, and in particular to a method, device and equipment for determining the typing of bronchial tuberculosis. Background Art

[0002] Tuberculosis is one of the world's largest infectious disease "killers", causing tens of millions of people to become ill each year, and more than 3,500 people die every day from this preventable and curable disease. At present, more than 60% of patients have severe complications such as atelectasis, bronchial stenosis, lung function damage, and lung damage, and even need lobectomy to be cured. This is mainly because bronchial tuberculosis has no specific clinical symptoms and imaging manifestations, and bronchoscopy requires experienced doctors to give an accurate diagnosis. Therefore, bronchial tuberculosis is often misdiagnosed and missed, missing the best time for treatment. Therefore, if intelligent diagnosis of bronchial tuberculosis is achieved through an artificial intelligence-assisted diagnosis system, it can effectively reduce the misdiagnosis and missed diagnosis of bronchial tuberculosis, early detection and early treatment, and reduce the risk of tuberculosis transmission. Summary of the invention

[0003] The purpose of this application is to provide a method, device and equipment for determining the typing of bronchial tuberculosis, which can realize intelligent diagnosis of bronchial tuberculosis through an artificial intelligence-assisted diagnosis system, and can effectively reduce misdiagnosis and missed diagnosis of bronchial tuberculosis.

[0004] To achieve the above objectives, this application provides the following solutions:

[0005] In a first aspect, the present application provides a method for determining the typing of bronchial tuberculosis, comprising:

[0006] Acquire a data set; the data set is an image sample under bronchial endoscopy;

[0007] Constructing a bronchial tuberculosis diagnosis model; the bronchial tuberculosis diagnosis model is a bronchial tuberculosis diagnosis model based on a ResNet34 framework and incorporating multi-head self-attention and deep separable convolution;

[0008] Training the bronchial tuberculosis diagnosis model based on the data set;

[0009] The user's images under bronchoscope are input into the trained bronchial tuberculosis diagnosis model to obtain the classification of bronchial tuberculosis.

[0010] Optionally, training the bronchial tuberculosis diagnosis model based on the data set specifically comprises the following steps:

[0011] Inputting the image samples under bronchial endoscopy into the bronchial tuberculosis diagnosis model to obtain the model output;

[0012] Use the cross loss function to calculate the difference between the output of the model and the true label to obtain the loss;

[0013] Calculating the gradient of the loss with respect to the parameters of the bronchial tuberculosis diagnosis model, and propagating the gradient of the loss from the output layer to the input layer by the chain rule;

[0014] The optimizer is used to update the parameters of the bronchial tuberculosis diagnosis model according to the gradient of the loss, and finally the trained bronchial tuberculosis diagnosis model is obtained.

[0015] Optionally, the bronchial tuberculosis diagnostic model specifically includes:

[0016] 7×7 convolutional layer, pooling layer, first residual module group, second residual module group, third residual module group, fourth residual module group, global average pooling layer and fully connected layer;

[0017] The 7×7 convolutional layer, the pooling layer, the first residual module group, the second residual module group, the third residual module group, the fourth residual module group, the global average pooling layer and the fully connected layer are connected in sequence.

[0018] Optionally, the first residual module group includes a first common residual block, a second common residual block and a third common residual block, the first common residual block, the second common residual block and the third common residual block each include two 3×3 convolutional layers, and the first common residual block, the second common residual block and the third common residual block are connected in sequence;

[0019] The second residual module group includes: a first depth-separable convolution residual block, a fourth common residual block, a fifth common residual block and a sixth common residual block; the fourth common residual block, the fifth common residual block and the sixth common residual block each include two 3×3 convolution layers;

[0020] The first depthwise separable convolution residual block, the fourth common residual block, the fifth common residual block and the sixth common residual block are connected in sequence;

[0021] The third residual module group includes: a second depth-separable convolution residual block, a seventh common residual block, an eighth common residual block, a ninth common residual block, a tenth common residual block and an eleventh common residual block; the seventh common residual block, the eighth common residual block, the ninth common residual block, the tenth common residual block and the eleventh common residual block each include two 3×3 convolutional layers;

[0022] The second depthwise separable convolution residual block, the seventh common residual block, the eighth common residual block, the ninth common residual block, the tenth common residual block and the eleventh common residual block are connected in sequence;

[0023] The fourth residual module group includes: a third depth-separable convolution residual block, a first multi-head self-attention mechanism residual block, and a second multi-head self-attention mechanism residual block;

[0024] The third depth-separable convolution residual block, the first multi-head self-attention mechanism residual block and the second multi-head self-attention mechanism residual block are connected in sequence.

[0025] Optionally, the first common residual block, the second common residual block, the third common residual block, the fourth common residual block, the fifth common residual block, the sixth common residual block, the seventh common residual block, the eighth common residual block, the ninth common residual block, the tenth common residual block and the eleventh common residual block all adopt the following calculation formula:

[0026] y=F(x,{W i})+W 1x1 *x

[0027] Among them, y represents the output, F(x,{W i})=ReLU(BN(W2*ReLU(BN(W1*x)))), where x represents input, BN represents batch normalization operation, ReLU represents nonlinear calculation, and W i Where i = 1, 2, W1 represents the first convolution operation and weight, W2 represents the second convolution operation and weight, and W 1×1 Represents 1*1 convolution operation and weight.

[0028] Optionally, the first depth-separable convolution residual block, the second depth-separable convolution residual block, and the third depth-separable convolution residual block all use the following calculation formula:

[0029] y=F(x,{W i})+W pw *(W dw *x)

[0030] Among them, W pw Indicates that W dw express.

[0031] Optionally, the first multi-head self-attention mechanism residual block and the second multi-head self-attention mechanism residual block both use the following calculation formula:

[0032] y = ReLU(BN(W 3×3 *MHSA(X)))+W 1×1 *x

[0033] Where, MHSA(X)=Concat(O1, O2, ..., O h )W o , W 3×3It represents the 3*3 convolution operation and weight, and Concat represents the merging function.

[0034] Optionally, the fully connected layer specifically adopts the following formula:

[0035]

[0036] Among them, p i represents the probability of the i-th category, zi represents the i-th element of the linear transformation output z, and e represents a natural constant.

[0037] In a second aspect, the present application provides a device for determining the typing of bronchial tuberculosis, comprising:

[0038] A data set acquisition module, used to acquire a data set; the data set is an image sample under bronchial endoscopy;

[0039] A bronchial combined diagnosis model construction module is used to construct a bronchial tuberculosis diagnosis model; the bronchial tuberculosis diagnosis model is a bronchial tuberculosis diagnosis model based on the ResNet34 framework and incorporating multi-head self-attention and deep separable convolution;

[0040] A training module, used for training the bronchial tuberculosis diagnosis model based on the data set;

[0041] The bronchial classification determination module is used to input the user's image under the bronchoscope into the trained bronchial tuberculosis diagnosis model to obtain the classification of bronchial tuberculosis.

[0042] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above-described methods for determining the typing of bronchial tuberculosis.

[0043] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0044] The present application provides a method, device and equipment for determining the typing of bronchial tuberculosis. By introducing deep separable convolution, a large amount of calculation in traditional convolution operation is split into two smaller calculation steps: depthwise convolution and pointwise convolution, which reduces the number of model parameters and the amount of calculation of convolution operation, improves the classification performance of the model, accelerates the training speed of the model, and effectively reduces the calculation burden of the model; the second convolution in the second and third residual blocks in the residual module group 4 in the ResNet34 network is replaced with a multi-head self-attention mechanism, so that the model can pay attention to the global and local features of the bronchial image at the same time, improves the accuracy of the model, and the accuracy rate can reach nearly 90%; in addition, the use of dual USB pedals ensures the simultaneous use of the bronchial tuberculosis artificial intelligence-assisted diagnosis system and the hospital bronchoscopy reporting system without affecting each other. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 A schematic diagram of a flow chart of a method for determining bronchial tuberculosis typing provided in one embodiment of the present application;

[0047] Figure 2 This is a schematic diagram of the interface of the artificial intelligence-assisted diagnosis system for bronchial tuberculosis according to one embodiment of the present application;

[0048] Figure 3 This is a working principle diagram of the artificial intelligence-assisted diagnosis system for bronchial tuberculosis in one embodiment of the present application;

[0049] Figure 4 This is a structural diagram of a bronchial tuberculosis diagnosis model based on a ResNet34 framework that incorporates multi-head self-attention and deep separable convolution in an embodiment of the present application;

[0050] Figure 5 This is a schematic diagram of the structure of a depth-separable convolution residual block according to an embodiment of the present application;

[0051] Figure 6 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0053] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0054] The artificial intelligence-assisted diagnosis system for bronchial tuberculosis is an intelligent diagnosis system that can be used in conjunction with a bronchoscope to identify bronchial tuberculosis and provide typing prompts. The doctor only needs to open the software and click the "Open Video" button. At this time, the video detection area in the upper left corner of the software will synchronize the hospital's endoscopy workstation image in real time. When the doctor performs a bronchoscopic examination on the patient and finds a suspected lesion of bronchial tuberculosis, he can step on it with his foot to capture the current image and send it to the hospital's PACS system and the artificial intelligence-assisted diagnosis system for bronchial tuberculosis at the same time. When the artificial intelligence-assisted diagnosis system for bronchial tuberculosis detects a signal transmitted by the medical foot pedal, it will capture the current video frame and display the captured image in the EBTB image diagnosis area in the upper right corner. It will then automatically perform an intelligent diagnosis on the image and display a positive judgment and typing prompt in the diagnosis result display area on the right side of the screen, such as Figure 2 As shown. Help doctors diagnose and reduce their workload.

[0055] See also Figure 3 , is the working principle diagram of the artificial intelligence-assisted diagnosis system for bronchial tuberculosis. The endoscopic video is connected to the high-definition data acquisition card through the HDMI high-definition data cable. When the pedal is pressed, the current frame of the video is captured and saved in the memory of the workstation. The trained bronchial tuberculosis auxiliary diagnosis model reads the currently captured image and automatically gives whether it is bronchial tuberculosis. If it is, it automatically gives a typing prompt. If it is not bronchial tuberculosis, it prompts type 0.

[0056] The system uses dual USB pedals. The control signal of the pedals controls the high-definition data acquisition card of the auxiliary diagnosis system on the one hand, and controls the original PACS image acquisition of the hospital on the other hand. The hospital PACS system is a system used by the hospital for bronchial data collection and diagnosis reports by doctors. The artificial intelligence-assisted diagnosis system for bronchial tuberculosis in this application is two independent systems without mutual interference.

[0057] Figure 1 A schematic diagram of a method for determining bronchial tuberculosis typing provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method includes:

[0058] Step 101: Acquire a data set; the data set is image samples under bronchial endoscopy.

[0059] Specifically, clinical data were collected in the hospital and a database with more than 20,000 bronchoscopic image samples was established.

[0060] Then the above data are preprocessed, including scaling, normalization, and data enhancement, so that the number of various types of bronchial tuberculosis images in the database is balanced to ensure the training effect.

[0061] Step 102: construct a bronchial tuberculosis diagnosis model; the bronchial tuberculosis diagnosis model is a bronchial tuberculosis diagnosis model based on the ResNet34 framework and incorporating multi-head self-attention and deep separable convolution.

[0062] See also Figure 4 , Figure 4 The structure of the bronchial tuberculosis diagnosis model is based on the ResNet34 framework and incorporates multi-head self-attention and deep separable convolution. The specific structure includes:

[0063] 7×7 convolutional layer, pooling layer, first residual module group, second residual module group, third residual module group, fourth residual module group, global average pooling layer and fully connected layer;

[0064] Among them, the 7×7 convolution layer, the pooling layer, the first residual module group, the second residual module group, the third residual module group, the fourth residual module group, the global average pooling layer and the fully connected layer are connected in sequence.

[0065] The first residual module group includes a first common residual block, a second common residual block and a third common residual block, wherein the first common residual block, the second common residual block and the third common residual block each include two 3×3 convolutional layers, and the first common residual block, the second common residual block and the third common residual block are connected in sequence;

[0066] The second residual module group includes: a first depthwise separable convolution residual block, a fourth common residual block, a fifth common residual block and a sixth common residual block; the fourth common residual block, the fifth common residual block and the sixth common residual block each include two 3×3 convolution layers;

[0067] The first depth-separable convolution residual block, the fourth common residual block, the fifth common residual block and the sixth common residual block are connected in sequence;

[0068] The third residual module group includes: a second depth-separable convolution residual block, a seventh common residual block, an eighth common residual block, a ninth common residual block, a tenth common residual block and an eleventh common residual block; the seventh common residual block, the eighth common residual block, the ninth common residual block, the tenth common residual block and the eleventh common residual block each include two 3×3 convolutional layers;

[0069] The second depth-separable convolution residual block, the seventh common residual block, the eighth common residual block, the ninth common residual block, the tenth common residual block and the eleventh common residual block are connected in sequence;

[0070] The fourth residual module group includes: a third depth-separable convolution residual block, a first multi-head self-attention mechanism residual block, and a second multi-head self-attention mechanism residual block;

[0071] The third depthwise separable convolution residual block, the first multi-head self-attention mechanism residual block, and the second multi-head self-attention mechanism residual block are connected in sequence.

[0072] See also Figure 5 , wherein the structures of the first depth-separable convolution residual block, the second depth-separable convolution residual block and the third depth-separable convolution residual block all include:

[0073] The DW conv layer, BN layer, ReLU layer, and PW conv layer are connected in sequence.

[0074] Among them, the first common residual block, the second common residual block, the third common residual block, the fourth common residual block, the fifth common residual block, the sixth common residual block, the seventh common residual block, the eighth common residual block, the ninth common residual block, the tenth common residual block and the eleventh common residual block all use the following calculation formula:

[0075] y=F(x,{W i})+W 1×1 *x(1)

[0076] Among them, y represents the output, F(x,{W i})=ReLU(BN(W2*ReLU(BN(W1*x)))), x represents input, BN represents batch normalization operation, ReLU represents nonlinear calculation, ReLU(x)=max(0,x). Max is the maximum value operation, W i Where i = 1, 2, W1 represents the first convolution operation and weight, W2 represents the second convolution operation and weight, and W 1×1 Represents 1*1 convolution operation and weight.

[0077] The first depth-separable convolution residual block, the second depth-separable convolution residual block, and the third depth-separable convolution residual block all use the following calculation formula:

[0078] y=F(x,{W i})+W pw *(W dw *x) (2)

[0079] Among them, W pw Indicates that W dw express.

[0080] The first multi-head self-attention mechanism residual block and the second multi-head self-attention mechanism residual block both use the following calculation formula:

[0081] y = ReLU(BN(W 3×3 *MHSA(X)))+W 1×1 *x (3)

[0082] Where, MHSA(X)=Concat(O1, O2, ..., O h )W o , W 3×3 W 1×1 represents the 3*3 convolution operation and weight, Concat represents the merging function, which merges the outputs of self-attention calculated in parallel by multiple heads on different sub-controls, MHSA(X)=Convat(O1, O2, ..., O h )W o , O i =Attention(O1, O1, O1).

[0083] The fully connected layer specifically uses the following formula:

[0084]

[0085] Among them, p i Represents the probability of the i-th category, zi represents the i-th element of the linear transformation output z, e represents a natural constant, z=Wh+b, h is the input feature vector with a dimension of 512, W is the weight matrix of the fully connected layer with a dimension of 4×512, b is the bias vector of the fully connected layer with a dimension of 4, and is converted to a probability distribution through the Softmax function: p=Softmax(z).

[0086] Step 103: training the bronchial tuberculosis diagnosis model based on the data set.

[0087] The specific training process is as follows:

[0088] Inputting the image samples under bronchial endoscopy into the bronchial tuberculosis diagnosis model to obtain the model output;

[0089] Use the cross loss function to calculate the difference between the output of the model and the true label to obtain the loss;

[0090] Calculating the gradient of the loss with respect to the parameters of the bronchial tuberculosis diagnosis model, and propagating the gradient of the loss from the output layer to the input layer by the chain rule;

[0091] The optimizer is used to update the parameters of the bronchial tuberculosis diagnosis model according to the gradient of the loss, and finally the trained bronchial tuberculosis diagnosis model is obtained.

[0092] The model training parameter configuration is shown in Table 1:

[0093] Table 1 Model training parameter configuration

[0094]

[0095] Step 104: Input the image of the user under the bronchoscope into the trained bronchial tuberculosis diagnosis model to obtain the classification of bronchial tuberculosis.

[0096] Specifically, the internal processing of the bronchial tuberculosis diagnosis model is as follows:

[0097] ① The image information of 224*224 pixels is first passed through a 7x7 convolutional layer for conv(7,7,cin=3,cout=64,padding=3,stride=2) convolution operation. The convolution operation can extract preliminary features of the image in a larger field of view, reduce the image size by half and increase the depth of the feature map, resulting in a 64*112*112 feature matrix.

[0098] ② Input the 64*112*112 feature matrix obtained in step ① into the pooling layer for conv(3,3,cin=3,cout=64,padding=1,stride=2) convolution operation, reducing the size by half to obtain a 64*56*56 feature matrix. The role of the pooling layer is to highlight significant features and reduce the amount of calculation of subsequent convolutional layers, thereby improving the overall computational efficiency of the network.

[0099] ③ Input the 64*56*56 feature matrix obtained in step ② into residual module group 1. This module group contains 3 residual blocks, each of which has two 3x3 convolutional layers. The input and output of each residual block are added to form a residual connection. The specific operation of each residual block is shown in formula 1. After the operation of this module, a 64*56*56 feature matrix is ​​obtained. The main function of this module is to further extract and enhance features, while reducing the gradient vanishing problem through residual connections.

[0100] ④ The feature matrix obtained according to step ③ is input to residual module group 2, which contains 4 residual blocks. Each block is still composed of two 3x3 convolution layers. The first residual block is a residual block with depthwise separable convolution, and the calculation method is shown in formula 3. The remaining three residual blocks are ordinary residual blocks, and the calculation method is shown in formula 1. After residual module group 2, a 128*28*28 feature matrix is ​​output. Compared with residual module group 1, the feature extraction at this stage is more in-depth, and more complex features are learned through more residual blocks.

[0101] ⑤The feature matrix obtained according to step ④ is input to residual module group 3. This module contains 3 residual blocks, each of which is still composed of two 3x3 convolutional layers, of which the first residual block is a residual block with depthwise separable convolution, the second and third residual blocks are with calculation methods as shown in formula 2, and the remaining 5 residual blocks are ordinary residual blocks, and the calculation method is shown in formula 1. After residual module group 3, a 256*14*14 feature matrix is ​​output. Residual module group 3 is the core part of feature extraction, and high-level features are extracted through a large number of residual blocks.

[0102] ⑥The image obtained according to step ⑤ is input to residual module group 4. This module contains 3 residual blocks, of which the first residual block is a residual block with depthwise separable convolution, and the calculation method is shown in formula 2. The other two residual blocks are residual blocks with multi-head self-attention mechanism (MHSA), that is, their second convolution layer introduces MHSA, and the calculation method is shown in formula 3. The introduction of MHSA is to capture the relationship and contextual information between long-distance features, enhance the globality of feature representation and improve the expressiveness of features. After residual module group 4, a 512*7*7 feature matrix is ​​output. Residual module group 4 integrates and classifies the previously extracted features, so that the model can pay attention to the global and local features of the image at the same time, improving the accuracy of the model.

[0103] ⑦ According to the feature matrix obtained in step ⑥, it is input into the global average pooling layer (avg pool), and the output is a 512*1*1 feature vector. The global average pooling layer extracts global features by reducing the dimension, replaces the fully connected layer, smoothes the feature map, and promotes the interpretability of the feature map.

[0104] ⑧According to the feature vector outputted in step ⑦, the classification task is realized through the fully connected layer, and finally the bronchial tuberculosis images are classified.

[0105] EBTB is divided into six types according to the manifestations under bronchoscopy during the course of the disease: type I (inflammatory infiltration type), type II (ulcer necrosis type), type III (granulation proliferation type), type IV (scar stenosis type), type V (tube wall softening type), and type VI (lymph node fistula type).

[0106] Based on the same inventive concept, the embodiment of the present application also provides a bronchial tuberculosis typing determination device for implementing the above-mentioned bronchial tuberculosis typing determination method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations of one or more bronchial tuberculosis typing determination device embodiments provided below can refer to the limitations of the bronchial tuberculosis typing determination method above, and will not be repeated here.

[0107] In an exemplary embodiment, a device for determining the typing of bronchial tuberculosis is provided, comprising:

[0108] The data set acquisition module is used to acquire a data set; the data set is an image sample under bronchial endoscopy.

[0109] The bronchial combined diagnosis model construction module is used to construct a bronchial tuberculosis diagnosis model; the bronchial tuberculosis diagnosis model is a bronchial tuberculosis diagnosis model based on the ResNet34 framework and incorporating multi-head self-attention and deep separable convolution.

[0110] A training module is used to train the bronchial tuberculosis diagnosis model based on the data set.

[0111] The bronchial classification determination module is used to input the user's image under the bronchoscope into the trained bronchial tuberculosis diagnosis model to obtain the classification of bronchial tuberculosis.

[0112] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 6As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store bronchial tuberculosis typing determination data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for determining the typing of bronchial tuberculosis is implemented.

[0113] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0114] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0115] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0116] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0117] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0118] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0119] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0120] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0121] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for determining the typing of bronchial tuberculosis, characterized in that: The method for determining the typing of bronchial tuberculosis comprises: Acquire a data set; the data set is an image sample under bronchial endoscopy; Constructing a bronchial tuberculosis diagnosis model; the bronchial tuberculosis diagnosis model is a bronchial tuberculosis diagnosis model based on a ResNet34 framework and incorporating multi-head self-attention and deep separable convolution; Training the bronchial tuberculosis diagnosis model based on the data set; The user's images under bronchoscope are input into the trained bronchial tuberculosis diagnosis model to obtain the classification of bronchial tuberculosis.

2. The method for determining bronchial tuberculosis typing according to claim 1, characterized in that: Training the bronchial tuberculosis diagnosis model based on the data set specifically includes the following steps: Inputting the image samples under bronchial endoscopy into the bronchial tuberculosis diagnosis model to obtain the model output; Use the cross loss function to calculate the difference between the output of the model and the true label to obtain the loss; Calculating the gradient of the loss with respect to the parameters of the bronchial tuberculosis diagnosis model, and propagating the gradient of the loss from the output layer to the input layer by the chain rule; The optimizer is used to update the parameters of the bronchial tuberculosis diagnosis model according to the gradient of the loss, and finally the trained bronchial tuberculosis diagnosis model is obtained.

3. The method for determining bronchial tuberculosis typing according to claim 1, characterized in that: The bronchial tuberculosis diagnostic model specifically includes: 7×7 convolutional layer, pooling layer, first residual module group, second residual module group, third residual module group, fourth residual module group, global average pooling layer and fully connected layer; The 7×7 convolutional layer, the pooling layer, the first residual module group, the second residual module group, the third residual module group, the fourth residual module group, the global average pooling layer and the fully connected layer are connected in sequence.

4. The method for determining bronchial tuberculosis typing according to claim 3, characterized in that: The first residual module group includes a first common residual block, a second common residual block and a third common residual block, the first common residual block, the second common residual block and the third common residual block each include two 3×3 convolutional layers, and the first common residual block, the second common residual block and the third common residual block are connected in sequence; The second residual module group includes: a first depth-separable convolution residual block, a fourth common residual block, a fifth common residual block and a sixth common residual block; the fourth common residual block, the fifth common residual block and the sixth common residual block each include two 3×3 convolution layers; The first depthwise separable convolution residual block, the fourth common residual block, the fifth common residual block and the sixth common residual block are connected in sequence; The third residual module group includes: a second depth-separable convolution residual block, a seventh common residual block, an eighth common residual block, a ninth common residual block, a tenth common residual block and an eleventh common residual block; the seventh common residual block, the eighth common residual block, the ninth common residual block, the tenth common residual block and the eleventh common residual block each include two 3×3 convolutional layers; The second depthwise separable convolution residual block, the seventh common residual block, the eighth common residual block, the ninth common residual block, the tenth common residual block and the eleventh common residual block are connected in sequence; The fourth residual module group includes: a third depth-separable convolution residual block, a first multi-head self-attention mechanism residual block, and a second multi-head self-attention mechanism residual block; The third depth-separable convolution residual block, the first multi-head self-attention mechanism residual block and the second multi-head self-attention mechanism residual block are connected in sequence.

5. The method for determining bronchial tuberculosis typing according to claim 4, characterized in that: The first common residual block, the second common residual block, the third common residual block, the fourth common residual block, the fifth common residual block, the sixth common residual block, the seventh common residual block, the eighth common residual block, the ninth common residual block, the tenth common residual block and the eleventh common residual block all adopt the following calculation formula: y=F(x,{W i })+W 1×1 *x Among them, y represents the output, F(x,{W i })=ReLU(BN(W2*ReLU(BN(W1*x)))), where x represents input, BN represents batch normalization operation, ReLU represents nonlinear calculation, and W i Where i = 1, 2, W1 represents the first convolution operation and weight, W2 represents the second convolution operation and weight, and W 1×1 Represents 1*1 convolution operation and weight.

6. The method for determining bronchial tuberculosis typing according to claim 5, characterized in that: The first depth-separable convolution residual block, the second depth-separable convolution residual block, and the third depth-separable convolution residual block all use the following calculation formula: y=F(x,{W i })+W pw *(W dw *x) Among them, W pw Represents channel-by-channel convolution operations and weights, W dw Represents point-by-point convolution operations and weights.

7. The method for determining bronchial tuberculosis typing according to claim 5, characterized in that: The first multi-head self-attention mechanism residual block and the second multi-head self-attention mechanism residual block both use the following calculation formula: y=ReLU(BN(W 3×3 *MHSA(X)))+W 1×1 *x Among them, MHSA(X)=Concat(O1,O2,…,O h )W o , W 3×3 It represents the 3*3 convolution operation and weight, and Concat represents the merging function.

8. The method for determining bronchial tuberculosis typing according to claim 3, characterized in that: The fully connected layer specifically adopts the following formula: Among them, p i represents the probability of the i-th category, z i represents the i-th element of the linear transformation output z, and e represents a natural constant.

9. A device for determining the typing of bronchial tuberculosis, characterized in that: The bronchial tuberculosis typing determination device comprises: A data set acquisition module, used to acquire a data set; the data set is an image sample under bronchial endoscopy; A bronchial combined diagnosis model construction module is used to construct a bronchial tuberculosis diagnosis model; the bronchial tuberculosis diagnosis model is a bronchial tuberculosis diagnosis model based on the ResNet34 framework and incorporating multi-head self-attention and deep separable convolution; A training module, used for training the bronchial tuberculosis diagnosis model based on the data set; The bronchial classification determination module is used to input the user's image under the bronchoscope into the trained bronchial tuberculosis diagnosis model to obtain the classification of bronchial tuberculosis.

10. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method for determining the typing of bronchial tuberculosis according to any one of claims 1 to 8.

Images