Bronchial tuberculosis auxiliary determination method, device, equipment, medium and product

Abstract

The invention discloses a bronchial tuberculosis auxiliary determination method, device and equipment, a medium and a product, and relates to the field of bronchial tuberculosis auxiliary diagnosis, and the method comprises the steps: constructing an image sample data set; constructing a bronchial tuberculosis diagnosis model integrated with multi-head self-attention, wherein the bronchial tuberculosis diagnosis model comprises a first residual module, a second residual module, a third residual module and a fourth residual module; the first residual module comprises a plurality of common residual blocks; the second residual group comprises a dotted line residual block and a plurality of common residual blocks; the third residual error module comprises a dotted line residual error block, a plurality of common residual error blocks and a residual error block with a multi-head self-attention mechanism; the fourth residual error module comprises a dotted line residual error block, a common residual error block and a residual error block with a multi-head self-attention mechanism; and training the bronchial tuberculosis diagnosis model integrated with multi-head self-attention, and obtaining the type of bronchial tuberculosis based on the trained model. According to the invention, the diagnosis precision of bronchial tuberculosis can be greatly improved.

Description

Technical Field

[0001] The present application relates to the field of auxiliary diagnosis of bronchial tuberculosis, and in particular to an auxiliary determination method, device, equipment, medium and product for bronchial tuberculosis. Background Art

[0002] Tuberculosis is one of the world's largest infectious disease "killers", causing tens of millions of people to become ill each year, and more than 3,500 people die every day from this preventable and curable disease. On November 7, 2023, the WHO's "2023 Global Tuberculosis Report" pointed out that in 2022, my country's new cases of tuberculosis were 748,000 (95% CI: 634,000-872,000), ranking third in the world, of which about 250,000 new cases of bronchial tuberculosis were reported each year. They are mainly distributed in underdeveloped areas in the central and western regions.

[0003] In my country, more than 60% of patients experience severe complications such as atelectasis, bronchial stenosis, lung function damage, and lung destruction, and even need lobectomy to be cured. This is mainly because bronchial tuberculosis has no specific clinical symptoms and imaging manifestations, and mainly relies on bronchoscopic examinations. It requires experienced doctors to give an accurate diagnosis. Therefore, bronchial tuberculosis is often misdiagnosed and missed in underdeveloped areas, missing the best time for treatment. Therefore, if the bronchoscopic images uploaded by patients can be intelligently diagnosed and accurate classification suggestions can be given through an artificial intelligence-based cloud service diagnosis platform, patients in underdeveloped areas or areas lacking experienced doctors can also easily get an accurate diagnosis, effectively reducing the misdiagnosis and missed diagnosis of bronchial tuberculosis, early detection and early treatment, and reducing the risk of tuberculosis transmission.

[0004] Therefore, how to improve the diagnostic accuracy of tuberculosis in patients in underdeveloped areas has become an urgent problem to be solved in this field. Summary of the invention

[0005] The purpose of this application is to provide a method, device, equipment, medium and product for auxiliary determination of bronchial tuberculosis, which can greatly improve the diagnostic accuracy of tuberculosis in underdeveloped areas and reduce missed diagnoses and misdiagnoses.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In a first aspect, the present application provides a method for assisting in the determination of bronchial tuberculosis, comprising:

[0008] Build a sample image dataset;

[0009] Construct a bronchial tuberculosis diagnosis model incorporating multi-head self-attention; the bronchial tuberculosis diagnosis model incorporating multi-head self-attention includes: a first residual module, a second residual module, a third residual module and a fourth residual module; wherein the first residual module includes a plurality of ordinary residual blocks; the second residual group includes a dotted residual block and a plurality of ordinary residual blocks; the third residual module includes: a dotted residual block, a plurality of ordinary residual blocks and a residual block with a multi-head self-attention mechanism; the fourth residual module includes: a dotted residual block, a common residual block and a residual block with a multi-head self-attention mechanism;

[0010] Training the bronchial tuberculosis diagnosis model incorporating multi-head self-attention based on the image sample data set;

[0011] The user's bronchial endoscope image to be processed is input into the trained bronchial tuberculosis diagnosis model with multi-head self-attention to obtain the classification of bronchial tuberculosis.

[0012] Optionally, the bronchial tuberculosis diagnosis model incorporating multi-head self-attention further includes:

[0013] 7×7 convolutional layer, pooling layer, global average pooling layer and fully connected layer;

[0014] The 7×7 convolutional layer, the pooling layer, the first residual module, the second residual module, the third residual module, the fourth residual module, the global average pooling layer and the fully connected layer are connected in sequence.

[0015] Optionally, the first residual module includes a first common residual block, a second common residual block and a third common residual block;

[0016] The first common residual block, the second common residual block and the third common residual block each include two 3×3 convolutional layers;

[0017] The first common residual block, the second common residual block and the third common residual block are connected in sequence;

[0018] The second residual module includes: a first dotted line residual block, a fourth normal residual block, a fifth normal residual block and a sixth normal residual block;

[0019] The first dotted line residual block, the fourth normal residual block, the fifth normal residual block and the sixth normal residual block each include two 3×3 convolutional layers;

[0020] The first dotted line residual block, the fourth normal residual block, the fifth normal residual block and the sixth normal residual block are connected in sequence;

[0021] The third residual module includes: a second dotted line residual block, a seventh ordinary residual block, an eighth ordinary residual block, a ninth ordinary residual block, a tenth ordinary residual block and a first residual block with a multi-head self-attention mechanism;

[0022] The second dotted line residual block, the seventh ordinary residual block, the eighth ordinary residual block, the ninth ordinary residual block and the tenth ordinary residual block each include two 3×3 convolutional layers; the first residual block with multi-head self-attention mechanism includes a 3×3 convolutional layer and a convolutional layer integrated with the multi-head self-attention mechanism;

[0023] The fourth residual module includes: a third dotted line residual block, an eleventh ordinary residual block and a second residual block with a multi-head self-attention mechanism;

[0024] The third dotted line residual block and the eleventh ordinary residual block both include two 3×3 convolutional layers, and the second residual block with multi-head self-attention mechanism includes a 3×3 convolutional layer and a convolutional layer integrated with the multi-head self-attention mechanism.

[0025] Optionally, training the bronchial tuberculosis diagnosis model incorporating multi-head self-attention based on the image sample data set specifically comprises the following steps:

[0026] Preprocessing the image sample data set;

[0027] Initializing the weights of the bronchial tuberculosis diagnosis model incorporating multi-head self-attention;

[0028] Setting parameters of the bronchial tuberculosis diagnosis model incorporating multi-head self-attention; the parameters include: model input size, training cycle, batch size, optimizer, loss function, basic learning rate, and weight decay rate;

[0029] The actual output of the bronchial tuberculosis diagnosis model incorporating multi-head self-attention is calculated through forward propagation;

[0030] Construct loss function;

[0031] Calculate a loss value based on the loss function and the actual output;

[0032] Calculating the gradient of the loss value relative to the parameters of the bronchial tuberculosis diagnosis model incorporating multi-head self-attention by back-propagation, and propagating the gradient of the loss from the output layer to the input layer by the chain rule;

[0033] The optimizer is used to update the parameters of the bronchial tuberculosis diagnosis model incorporating multi-head self-attention according to the gradient until the bronchial tuberculosis diagnosis model incorporating multi-head self-attention converges.

[0034] Optionally, the loss function is expressed as follows:

[0035] L Total =L WCE +α×L red ;

[0036] Among them, L Total is the total loss, L WCE is the weighted classification loss, L red It is a color feature loss designed based on doctors’ expertise in diagnosing bronchial tuberculosis, and α is a trainable weight parameter.

[0037] Optionally, the calculation of the gradient of the loss value relative to the parameters of the bronchial tuberculosis diagnosis model incorporating multi-head self-attention by back propagation specifically adopts the following formula:

[0038]

[0039] Among them, θ is the parameter of the model, is the gradient of the classification loss with respect to the parameters, is the gradient of the color loss with respect to the parameter.

[0040] In a second aspect, the present application provides a device for auxiliary determination of bronchial tuberculosis, the device for auxiliary determination of bronchial tuberculosis comprising:

[0041] A dataset construction module, used to construct an image sample dataset;

[0042] A diagnostic model construction module, used to construct a bronchial tuberculosis diagnostic model incorporating multi-head self-attention; the bronchial tuberculosis diagnostic model incorporating multi-head self-attention includes: a first residual module, a second residual module, a third residual module and a fourth residual module; wherein the first residual module includes a plurality of ordinary residual blocks; the second residual group includes a dotted residual block and a plurality of ordinary residual blocks; the third residual module includes: a dotted residual block, a plurality of ordinary residual blocks and a residual block with a multi-head self-attention mechanism; the fourth residual module includes: a dotted residual block, a common residual block and a residual block with a multi-head self-attention mechanism;

[0043] A training module, used for training the bronchial tuberculosis diagnosis model incorporating multi-head self-attention based on the image sample data set;

[0044] The bronchial tuberculosis classification determination module is used to input the user's bronchial endoscope image to be processed into a trained bronchial tuberculosis diagnosis model that incorporates multi-head self-attention to obtain the classification of bronchial tuberculosis.

[0045] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any one of the above-mentioned methods for auxiliary determination of bronchial tuberculosis.

[0046] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-mentioned methods for auxiliary determination of bronchial tuberculosis.

[0047] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of any of the above-mentioned methods for auxiliary determination of bronchial tuberculosis.

[0048] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0049] The present application provides a method, device, equipment, medium and product for auxiliary determination of bronchial tuberculosis. The second convolution in the last residual block of the third residual module and the fourth residual module in the ResNet34 network is replaced with a multi-head self-attention mechanism, so that the model can pay attention to the global and local features of the bronchial image at the same time, greatly improving the accuracy and generalization ability of the model; and the attention mechanism is added to the last residual block of the third residual module and the fourth residual module. Compared with adding the attention mechanism only in the fourth residual module, the addition of the global features of the middle layer can realize the global modeling of the middle and high-level features. On the one hand, it can capture richer global feature information, which can further improve the accuracy of the model. On the other hand, the model generalization ability may be stronger; in addition, the doctor's diagnostic expertise is integrated into the model training, the corresponding loss function is designed, and then combined with the weighted classification loss function for joint optimization. In this way, the model can learn to focus on key areas during the training process, and combine the features automatically extracted by the deep learning model to improve the accuracy of classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0051] Figure 1 This is a flow chart of a method for auxiliary determination of bronchial tuberculosis in one embodiment of the present application;

[0052] Figure 2This is a schematic diagram of the structure of a bronchial tuberculosis diagnosis model incorporating multi-head self-attention in one embodiment of the present application;

[0053] Figure 3 This is a functional diagram of the website of the bronchial tuberculosis auxiliary diagnosis cloud service platform in one embodiment of the present application;

[0054] Figure 4 This is a schematic diagram of the workflow of the bronchial tuberculosis auxiliary diagnosis cloud service platform system in one embodiment of the present application;

[0055] Figure 5 This is a schematic diagram of the diagnosis interface of the bronchial tuberculosis artificial intelligence-assisted diagnosis cloud service platform in one embodiment of the present application;

[0056] Figure 6 This is a schematic diagram of the historical record interface of the bronchial tuberculosis artificial intelligence-assisted diagnosis cloud service platform in one embodiment of the present application;

[0057] Figure 7 This is a schematic diagram of the popular science interface of the bronchial tuberculosis artificial intelligence-assisted diagnosis cloud service platform in one embodiment of the present application;

[0058] Figure 8 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0060] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0061] Figure 1 This is a flow chart of a method for assisting in determining bronchial tuberculosis in one embodiment of the present application, see Figure 1 , the method in this application includes:

[0062] Step 101: Build an image sample dataset.

[0063] Specifically, this application has established a database of more than 20,000 bronchoscopic image samples, all of which are annotated by director-level doctors to ensure accuracy.

[0064] Step 102: Construct a bronchial tuberculosis diagnosis model incorporating multi-head self-attention; the bronchial tuberculosis diagnosis model incorporating multi-head self-attention includes: a first residual module, a second residual module, a third residual module and a fourth residual module; wherein the first residual module includes multiple ordinary residual blocks; the second residual group includes a dotted residual block and multiple ordinary residual blocks; the third residual module includes: a dotted residual block, multiple ordinary residual blocks and a residual block with a multi-head self-attention mechanism; the fourth residual module includes: a dotted residual block, an ordinary residual block and a residual block with a multi-head self-attention mechanism.

[0065] This application proposes a new bronchial tuberculosis diagnosis model based on the ResNet34 framework and the multi-head self-attention mechanism. This model is based on the ResNet34 basic model framework and the multi-head self-attention mechanism (Multi-Head Self-Attention, MHSA). Figure 2 As shown in Figure 2. MHSA is a multi-head self-attention mechanism.

[0066] The ResNet34 model is a shallow network. The number of parameters in a shallow network is not very large, the cost of running a training program is not very high, and the trained parameter file is relatively small, making it suitable for placement on a cloud service platform.

[0067] Combination Figure 2 , the model specifically includes:

[0068] 7×7 convolution layer, pooling layer, first residual module, second residual module, third residual module, fourth residual module, global average pooling layer and fully connected layer; wherein, 7×7 convolution layer, pooling layer, first residual module, second residual module, third residual module, fourth residual module, global average pooling layer and fully connected layer are connected in sequence.

[0069] Specifically, the first residual module includes a first common residual block, a second common residual block and a third common residual block;

[0070] The first common residual block, the second common residual block and the third common residual block each include two 3×3 convolutional layers;

[0071] The first common residual block, the second common residual block and the third common residual block are connected in sequence;

[0072] The second residual module includes: a first dotted line residual block, a fourth normal residual block, a fifth normal residual block and a sixth normal residual block;

[0073] The first dotted line residual block, the fourth normal residual block, the fifth normal residual block and the sixth normal residual block each include two 3×3 convolutional layers;

[0074] The first dotted line residual block, the fourth normal residual block, the fifth normal residual block and the sixth normal residual block are connected in sequence;

[0075] The third residual module includes: a second dotted line residual block, a seventh ordinary residual block, an eighth ordinary residual block, a ninth ordinary residual block, a tenth ordinary residual block and a first residual block with a multi-head self-attention mechanism;

[0076] The second dotted line residual block, the seventh ordinary residual block, the eighth ordinary residual block, the ninth ordinary residual block and the tenth ordinary residual block each include two 3×3 convolutional layers; the first residual block with multi-head self-attention mechanism includes a 3×3 convolutional layer and a convolutional layer integrated with the multi-head self-attention mechanism;

[0077] The fourth residual module includes: a third dotted line residual block, an eleventh ordinary residual block and a second residual block with a multi-head self-attention mechanism;

[0078] The third dotted line residual block and the eleventh ordinary residual block both include two 3×3 convolutional layers, and the second residual block with multi-head self-attention mechanism includes a 3×3 convolutional layer and a convolutional layer integrated with the multi-head self-attention mechanism.

[0079] Step 103: training the bronchial tuberculosis diagnosis model incorporating multi-head self-attention based on the image sample dataset.

[0080] The specific training process is as follows:

[0081] A database of more than 20,000 bronchoscopic image samples was established, and all images were annotated by director-level doctors to ensure accuracy. The model training process is as follows:

[0082] (1) Data preparation

[0083] Collect clinical data in hospitals: Construct a high-quality EBTB medical image dataset.

[0084] Data preprocessing: Preprocess the data to ensure the training effect.

[0085] (2) Model initialization

[0086] Definition Figure 2 The structure of the bronchial tuberculosis diagnosis model based on the ResNet34 framework and multi-head self-attention is shown.

[0087] Initialize Weights: Initialize the weights of the model.

[0088] (3) Training Model

[0089] Set the model training parameters according to Table 1 below. The model automatically calculates forward propagation, calculates loss, performs back propagation, and updates parameters.

[0090] Table 1 Model training parameter configuration

[0091]

[0092] 1) Forward propagation: The input image goes through a series of convolution, batch normalization, activation and pooling layers, and finally gets the output through the fully connected layer.

[0093] 2) Calculation loss: The color feature loss designed based on doctors’ expertise in diagnosing bronchial tuberculosis is combined with the weighted cross entropy loss to construct a comprehensive loss function. This design not only ensures that the model maintains efficient classification capabilities, but also focuses on the key color feature areas in the image that are closely related to type I disease characteristics.

[0094] The expression of the loss function is as follows:

[0095] L Total =L WCE +α×L red ;

[0096] Among them, L Total is the total loss, L WCE is the weighted classification loss, and the cross entropy loss is used to measure the difference between the classification result output by the model and the actual label. red It is a color feature loss designed based on doctors’ expertise in diagnosing bronchial tuberculosis. α is a trainable weight parameter used to balance the contribution of classification loss and color loss to the total loss.

[0097] In each training iteration, the model calculates the output through forward propagation and uses the total loss function to calculate the loss value. Next, the gradient of the loss function relative to the model parameters is calculated through backpropagation. In this process, the gradient of the color loss affects the way the model parameters are updated. The calculation method is as follows:

[0098]

[0099] Where: θ is the parameter of the model, is the gradient of the classification loss with respect to the parameters, is the gradient of the color loss with respect to the parameter.

[0100] The model adjusts its parameters based on the above gradients so that the total loss function value gradually decreases.

[0101] 3) Back propagation: Calculate the gradient of the loss with respect to the model parameters. Through the chain rule, the gradient of the loss is propagated from the output layer to the input layer.

[0102] 4) Parameter update: Use the optimizer (SGD) to update the parameters according to the calculated gradient. The optimizer will adjust the parameters of each layer to reduce the loss. This process will be repeated for multiple training cycles (epochs) until the model converges.

[0103] After the model training is completed, an optimized model can be obtained for auxiliary diagnosis of bronchial tuberculosis.

[0104] Step 104: Input the user's bronchial endoscope image to be processed into the trained bronchial tuberculosis diagnosis model incorporating multi-head self-attention to obtain the classification of bronchial tuberculosis.

[0105] Specifically, the internal data processing process of the bronchial tuberculosis diagnosis model incorporating multi-head self-attention is as follows:

[0106] ① The pixel 224*224 image information is first passed through a 7x7 convolutional layer for conv(7,7,cin=3,cout=64,padding=3,stride=2) convolution operation, which can extract preliminary features of the image in a larger field of view, reduce the image size by half and increase the depth of the feature map, obtaining a 64*112*112 feature matrix.

[0107] ② Input the feature matrix obtained in step ① into the pooling layer for conv(3,3,cin=3,cout=64,padding=1,stride=2) convolution operation, reducing the size by half to obtain a 64*56*56 feature matrix. The role of the pooling layer is to highlight significant features and reduce the amount of calculation of subsequent convolutional layers, thereby improving the overall computational efficiency of the network.

[0108] ③ Input the feature matrix obtained in step ② to the first residual module. This module contains 3 residual blocks, each of which has two 3x3 convolutional layers. The input and output of each residual block are added to form a residual connection. The specific operation of each residual block is shown in the following formula. After the operation of this module, a 64*56*56 feature matrix is ​​obtained. The main function of this module is to further extract and enhance features, while reducing the gradient vanishing problem through residual connections.

[0109] The calculation formula of the ordinary residual block is as follows:

[0110] y=F(x,{W i})+W 1×1 *x

[0111] Among them, y represents the output, F(x,{W i})=ReLU(BN(W2*ReLU(BN(W1*x)))), x represents input, BN represents batch normalization operation, which is a regularization technique; ReLU represents nonlinear calculation, ReLU(x)=max(0,x). Max is the maximum value operation; W i Where i = 1, 2, W1 represents the first convolution operation and weight, W2 represents the second convolution operation and weight, and W 1×1 Represents 1*1 convolution operation and weight.

[0112] ④ According to the feature matrix obtained in step ③, it is input to the second residual module, which contains 4 residual blocks. Each block is still composed of two 3x3 convolutional layers. The first residual block is the dotted residual block, and the calculation method is shown in the following formula. The remaining 3 residual blocks are ordinary residual blocks. The calculation method refers to the formula in step ③. The second residual module outputs a 128*28*28 feature matrix. Compared with the first residual module, the feature extraction at this stage is more in-depth, and more complex features are learned through more residual blocks.

[0113] The calculation formula of the dotted residual block is as follows:

[0114] y=F(x,{W i})+W 1×1 *x

[0115] ⑤ Input the feature matrix obtained in step ④ to the third residual module. This module contains 3 residual blocks, each of which is still composed of two 3x3 convolutional layers. The first residual block is a dotted residual block. The calculation method is shown in the formula in step ⑤. The next 4 residual blocks are ordinary residual blocks. The calculation method is shown in the formula in step ③. The last residual block is a residual block with a multi-head self-attention mechanism (MHSA), that is, its second convolutional layer introduces MHSA, and the calculation method is shown in Formula 2. MHSA is introduced to capture the relationship and contextual information between long-distance features, enhance the globality of feature representation, and improve the expressiveness of features. After the third residual module, a 256*14*14 feature matrix is ​​output. The third residual module is the core part of feature extraction, and high-level features are extracted through a large number of residual blocks.

[0116] The calculation formula of the residual block with multi-head self-attention mechanism (MHSA) is as follows:

[0117] y = ReLU(BN(W 3×3 *MHSA(X)))+W 1×1 *x

[0118] Among them, MHSA(X)=Concat(O1,O2,…,O h)W o , W 3×3 W 1×1 represents the 3*3 convolution operation and weight, Concat represents the merging function, which merges the outputs of self-attention calculated in parallel by multiple heads on different sub-controls, MHSA(X)=Concat(O1,O2,…,O h )W o , O i =Attention(O1,O1,O1).

[0119] ⑥The image obtained according to step ⑤ is input to the fourth residual module. This module contains 3 residual blocks, of which the first residual block is a dotted residual block, and the calculation method refers to the formula in step ④. The next residual block is an ordinary residual block, and the calculation method refers to the formula in step ①. The last residual block is a residual block with a multi-head self-attention mechanism (MHSA), that is, its second convolutional layer introduces MHSA, and the calculation method refers to the formula in step ⑤. Finally, a 512*7*7 feature matrix is ​​output. The fourth residual module integrates and classifies the previously extracted features, so that the model can pay attention to the global and local features of the image at the same time, thereby improving the accuracy of the model.

[0120] ⑦ According to the feature matrix obtained in step ⑥, it is input into the global average pooling layer (avg pool) and outputs a 512*1*1 feature vector. The global average pooling layer extracts global features by reducing the dimension, replaces the fully connected layer, smoothes the feature map, and promotes the interpretability of the feature map.

[0121] ⑧According to the feature vector output in step ⑦, the classification task is realized through the fully connected layer, and finally the bronchial tuberculosis image is classified. The specific calculation formula of the fully connected layer is:

[0122] 1) Linear transformation z = Wh + b where:

[0123] Among them, h is the input feature vector with a dimension of 512; W is the weight matrix of the fully connected layer with a dimension of 4×512; b is the bias vector of the fully connected layer with a dimension of 4.

[0124] 2) Convert to probability distribution through Softmax function: p = Softmax(z)

[0125] The formula of the Softmax function is:

[0126]

[0127] Among them, p i is the probability of the ith category, z i is the i-th element of the linear transformation output z.

[0128] In order to further demonstrate the diagnostic accuracy of the model in this application, we conducted experiments using the EBTB image database constructed in this application. The dataset was divided into training set, validation set, and test set in a ratio of 8:1:1.

[0129] The experimental results show that:

[0130] The overall classification accuracy of the model in the prior art is 81.04%;

[0131] The overall classification accuracy of the model in this application is 88.06%, which shows that the diagnostic accuracy is greatly improved.

[0132] The present application also provides a cloud service platform, which is mainly composed of a cloud server and a domain name. After binding the domain name to the IP of the cloud server and setting the port number, the program on the cloud server can be bound to the domain name. The web page logic program app.py file is run on the cloud server to connect the jump logic between each web page and realize the interaction between the web page and the program under the domain name.

[0133] In the program, open the web page to open the homepage interface, and then click the login button to enter the login interface. When the account and password entered are consistent with those in the database, you can enter the artificial intelligence assisted diagnosis cloud service platform.

[0134] When the user uploads a bronchoscopic image on the web page, the program will call the trained bronchial tuberculosis auxiliary diagnosis model to read the current image. After the model calculation, it will determine whether it is bronchial tuberculosis. If it is, it will automatically give a typing prompt. If it is not bronchial tuberculosis, it will prompt "No bronchial tuberculosis". The image, diagnosis results and typing results will be saved in the "history record".

[0135] Figure 3 This is a functional diagram of the bronchial tuberculosis auxiliary diagnosis cloud service platform website in one embodiment of the present application, see Figure 3 , the website function diagram mainly revolves around the many functions of the user center. The starting point of the flow chart is "Login", which means that the user needs to log in to the platform first. If you already have an account, enter the account and password to log in, and the system will match and verify with the information in the database; if you have not registered yet, you will be guided to the registration page to complete the information filling. After successful registration, the information is encrypted and stored in the database through the AES algorithm. Afterwards, the user will enter the "User Center" interface. This interface provides entrances to multiple functional modules such as account information, user feedback, question consultation, tuberculosis popularization, historical records, usage services and instructions for use.

[0136] Within the user center, the process is divided into multiple functions. First, users can perform "account management", including changing passwords, binding mobile phones, and other operations. If users have any questions during use, they can ask questions through the "question" module and get help. In addition, the platform also provides a "popular science" module to convey medical knowledge and prevention methods related to bronchial tuberculosis to users.

[0137] In the core part of the user center, there is a prominent "diagnosis service" module. This module is the core function of the entire platform, which provides auxiliary diagnosis services for bronchial tuberculosis. Users can upload their medical records, test results and other information here, and the platform will use advanced algorithms and models to process and analyze the data, and then give diagnostic suggestions or predictions.

[0138] In addition to diagnostic services, the User Center also provides "User Feedback" and "History" modules. Users can use the "User Feedback" module to provide feedback on their experience or make suggestions to the platform to help it continuously optimize and improve. The "History" module records the user's past operations and behaviors, making it convenient for users to view and review them at any time.

[0139] In addition, in the face of user data security in the bronchial tuberculosis auxiliary diagnosis cloud service platform system, the back-end application ensures that users need to authenticate when logging in and using the service. Here, symmetric encryption technology is selected to ensure the security of user data, and the AES-CBC mode is used to further ensure the secure storage and transmission of user data.

[0140] Database AES encryption implementation

[0141] (1) Key and Initialization Vector (IV) Management

[0142] Key generation: The system uses a safe and reliable random number generator to generate a 256-bit AES key. This key is kept strictly confidential and is only used during encryption and decryption.

[0143] Initialization vector: To enhance the security of encryption, the system also generates an initialization vector (IV) with the same size as the encryption block. The IV is unique in each encryption process and is different from the key. It is used for group chaining in CBC mode to ensure that even the same plaintext data will not produce the same ciphertext in two encryption processes.

[0144] (2) Data encryption process

[0145] Plaintext data preparation: When user data needs to be stored or transmitted, the system first ensures that the data exists in plaintext. This includes sensitive information such as user personal information and diagnosis results.

[0146] AES encryption: The system uses AES-CBC mode and the generated key and IV to encrypt plaintext data. The encryption process involves multiple rounds of iterative operations, including byte substitution, row shift, column confusion, and round key addition, which ultimately converts plaintext data into ciphertext data.

[0147] Ciphertext storage: The encrypted ciphertext together with IV is securely stored in the database of the cloud service platform.

[0148] (3) Data decryption process

[0149] Ciphertext data retrieval: When accessing user data, the system first retrieves the ciphertext data and IV from the database.

[0150] AES decryption: Using the same key, AES-CBC mode and IV as used for encryption, the system decrypts the ciphertext data. The decryption process is the opposite of the encryption process, including steps such as reverse addition of round keys, reverse column confusion, reverse row shifting and reverse byte substitution, and finally restores the ciphertext data to the original plaintext data.

[0151] Use of plaintext data: The decrypted plaintext data can be used for subsequent processing, analysis or display to the user.

[0152] In conclusion, by using the AES algorithm for data encryption, the bronchial tuberculosis auxiliary diagnosis cloud service platform system can ensure the security of user data during storage and transmission.

[0153] See also Figure 4 , is a schematic diagram of the workflow of the bronchial tuberculosis auxiliary diagnosis cloud service platform system, which includes:

[0154] (1) Users can open any browser in an area with Internet access, enter the domain name of the EBTB network diagnosis platform (www.ebtbpredict.asia), and press Enter to enter the login page of the platform.

[0155] (2) Enter your account and password and click Login. If the account and password are entered incorrectly, the platform will pop up a corresponding error box to prompt the user that the account or password is incorrect and ask the user to re-enter the information. If the account and password are entered correctly, the platform will enter the diagnosis page of the platform and then click the corresponding function button in the upper left corner of the diagnosis page to select a local EBTB image. The platform will then perform a preliminary screening of the EBTB image selected by the user. If the image selected is not in JPG format, a warning box will pop up on the web page and prompt you to select an EBTB image in the correct format. The platform will then display the uploaded EBTB image in the correct format in the image box on the web page and pre-process the uploaded image on the server background to ensure that the image passed in meets the input requirements of the platform's diagnosis model.

[0156] (3) Click the EBTB lesion diagnosis button in the upper right corner of the detection page to transfer the image processed in the second step to the XX network model on the cloud server corresponding to the domain name for classification to obtain the diagnosis result.

[0157] (4) This platform displays the diagnosis results obtained on the server in the third step in the lower right corner of the page, and displays the EBTB lesion type with the highest diagnosis probability at the top. At the same time, considering the establishment of the EBTB dataset, this platform simultaneously assumes the function of the save button when the EBTB lesion detection button is clicked. With the user's knowledge and consent, the image transmitted to the platform is saved in the EBTB database, continuously improving the EBTB dataset and contributing to the subsequent development of EBTB medical diagnosis and intelligent diagnosis.

[0158] Figure 5 This is a schematic diagram of the diagnosis interface of the bronchial tuberculosis artificial intelligence-assisted diagnosis cloud service platform in one embodiment of the present application. The bronchial tuberculosis artificial intelligence cloud service platform is a website that provides services for patients with bronchial tuberculosis. After the patient registers and logs in to the cloud platform and enters the EBTB detection system, click "Upload Image" to upload the image on the bronchial endoscopy report to the cloud platform. Then click the "Start Diagnosis" button, and the cloud platform can identify the bronchial tuberculosis in the image. Through this interface, a typing prompt can be given so that the user can clearly see the diagnosis result.

[0159] Figure 6 This is a schematic diagram of the historical record interface of the bronchial tuberculosis artificial intelligence-assisted diagnosis cloud service platform in one embodiment of the present application. Through this interface, users can see the historical consultation time and diagnosis results.

[0160] Figure 7 This is a schematic diagram of the popular science interface of the cloud service platform for artificial intelligence-assisted diagnosis of bronchial tuberculosis in one embodiment of the present application. In addition to intelligent diagnosis, the website can also store diagnosis history records and provide popular science information on the diagnosis and treatment of tuberculosis, so that patients can understand bronchial tuberculosis and better cooperate with doctors in treatment. If the patient does not know how to use the cloud service platform, he or she can click the "Instructions for Use" button to learn how to use this detection system.

[0161] Based on the same inventive concept, the embodiment of the present application also provides a bronchial tuberculosis auxiliary determination device for implementing the above-mentioned bronchial tuberculosis auxiliary determination method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations of one or more bronchial tuberculosis auxiliary determination device embodiments provided below can refer to the limitations of the bronchial tuberculosis auxiliary determination method above, and will not be repeated here.

[0162] In an exemplary embodiment, a bronchial tuberculosis auxiliary determination device is provided, comprising:

[0163] The dataset construction module is used to construct an image sample dataset.

[0164] A diagnostic model construction module is used to construct a bronchial tuberculosis diagnostic model incorporating multi-head self-attention; the bronchial tuberculosis diagnostic model incorporating multi-head self-attention includes: a first residual module, a second residual module, a third residual module and a fourth residual module; wherein the first residual module includes multiple ordinary residual blocks; the second residual group includes a dotted residual block and multiple ordinary residual blocks; the third residual module includes: a dotted residual block, multiple ordinary residual blocks and a residual block with a multi-head self-attention mechanism; the fourth residual module includes: a dotted residual block, an ordinary residual block and a residual block with a multi-head self-attention mechanism.

[0165] A training module is used to train the bronchial tuberculosis diagnosis model incorporating multi-head self-attention based on the image sample data set.

[0166] The bronchial tuberculosis classification determination module is used to input the user's bronchial endoscope image to be processed into a trained bronchial tuberculosis diagnosis model that incorporates multi-head self-attention to obtain the classification of bronchial tuberculosis.

[0167] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 8 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store auxiliary determination data of bronchial tuberculosis. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for auxiliary determination of bronchial tuberculosis is implemented.

[0168] Those skilled in the art will understand that Figure 8The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0169] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0170] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0171] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0172] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0173] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0174] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0175] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0176] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for assisting in the determination of bronchial tuberculosis, characterized in that: The auxiliary determination method for bronchial tuberculosis comprises: Build a sample image dataset; Construct a bronchial tuberculosis diagnosis model incorporating multi-head self-attention; the bronchial tuberculosis diagnosis model incorporating multi-head self-attention includes: a first residual module, a second residual module, a third residual module and a fourth residual module; wherein the first residual module includes a plurality of ordinary residual blocks; the second residual group includes a dotted residual block and a plurality of ordinary residual blocks; the third residual module includes: a dotted residual block, a plurality of ordinary residual blocks and a residual block with a multi-head self-attention mechanism; the fourth residual module includes: a dotted residual block, a common residual block and a residual block with a multi-head self-attention mechanism; Training the bronchial tuberculosis diagnosis model incorporating multi-head self-attention based on the image sample data set; The user's bronchial endoscope image to be processed is input into the trained bronchial tuberculosis diagnosis model with multi-head self-attention to obtain the classification of bronchial tuberculosis.

2. The auxiliary determination method for bronchial tuberculosis according to claim 1, characterized in that: The bronchial tuberculosis diagnosis model incorporating multi-head self-attention also includes: 7×7 convolutional layer, pooling layer, global average pooling layer and fully connected layer; The 7×7 convolutional layer, the pooling layer, the first residual module, the second residual module, the third residual module, the fourth residual module, the global average pooling layer and the fully connected layer are connected in sequence.

3. The auxiliary determination method for bronchial tuberculosis according to claim 1, characterized in that: The first residual module includes a first common residual block, a second common residual block and a third common residual block; The first common residual block, the second common residual block and the third common residual block each include two 3×3 convolutional layers; The first common residual block, the second common residual block and the third common residual block are connected in sequence; The second residual module includes: a first dotted line residual block, a fourth normal residual block, a fifth normal residual block and a sixth normal residual block; The first dotted line residual block, the fourth normal residual block, the fifth normal residual block and the sixth normal residual block each include two 3×3 convolutional layers; The first dotted line residual block, the fourth normal residual block, the fifth normal residual block and the sixth normal residual block are connected in sequence; The third residual module includes: a second dotted line residual block, a seventh ordinary residual block, an eighth ordinary residual block, a ninth ordinary residual block, a tenth ordinary residual block and a first residual block with a multi-head self-attention mechanism; The second dotted line residual block, the seventh ordinary residual block, the eighth ordinary residual block, the ninth ordinary residual block and the tenth ordinary residual block each include two 3×3 convolutional layers; the first residual block with multi-head self-attention mechanism includes a 3×3 convolutional layer and a convolutional layer integrated with the multi-head self-attention mechanism; The fourth residual module includes: a third dotted line residual block, an eleventh ordinary residual block and a second residual block with a multi-head self-attention mechanism; The third dotted line residual block and the eleventh ordinary residual block both include two 3×3 convolutional layers, and the second residual block with multi-head self-attention mechanism includes a 3×3 convolutional layer and a convolutional layer integrated with the multi-head self-attention mechanism.

4. The auxiliary determination method for bronchial tuberculosis according to claim 1, characterized in that: Training the bronchial tuberculosis diagnosis model incorporating multi-head self-attention based on the image sample data set specifically includes the following steps: Preprocessing the image sample data set; Initializing the weights of the bronchial tuberculosis diagnosis model incorporating multi-head self-attention; Setting parameters of the bronchial tuberculosis diagnosis model incorporating multi-head self-attention; the parameters include: model input size, training cycle, batch size, optimizer, loss function, basic learning rate, and weight decay rate; The actual output of the bronchial tuberculosis diagnosis model incorporating multi-head self-attention is calculated through forward propagation; Construct loss function; Calculate a loss value based on the loss function and the actual output; Calculating the gradient of the loss value relative to the parameters of the bronchial tuberculosis diagnosis model incorporating multi-head self-attention by back-propagation, and propagating the gradient of the loss from the output layer to the input layer by the chain rule; The optimizer is used to update the parameters of the bronchial tuberculosis diagnosis model incorporating multi-head self-attention according to the gradient until the bronchial tuberculosis diagnosis model incorporating multi-head self-attention converges.

5. The auxiliary determination method for bronchial tuberculosis according to claim 4, characterized in that: The expression of the loss function is as follows: L Total =L WCE +α×L red ; Among them, L Total is the total loss, L WCE is the weighted classification loss, L red It is a color feature loss designed based on doctors’ expertise in diagnosing bronchial tuberculosis, and α is a trainable weight parameter.

6. The auxiliary determination method for bronchial tuberculosis according to claim 4, characterized in that: The gradient of the loss value relative to the parameters of the bronchial tuberculosis diagnosis model incorporating multi-head self-attention is calculated by back propagation using the following formula: Among them, θ is the parameter of the model, is the gradient of the classification loss with respect to the parameters, is the gradient of the color loss with respect to the parameter.

7. A device for assisting in determining bronchial tuberculosis, characterized in that: The auxiliary determination device for bronchial tuberculosis comprises: A dataset construction module, used to construct an image sample dataset; A diagnostic model construction module, used to construct a bronchial tuberculosis diagnostic model incorporating multi-head self-attention; the bronchial tuberculosis diagnostic model incorporating multi-head self-attention includes: a first residual module, a second residual module, a third residual module and a fourth residual module; wherein the first residual module includes a plurality of ordinary residual blocks; the second residual group includes a dotted residual block and a plurality of ordinary residual blocks; the third residual module includes: a dotted residual block, a plurality of ordinary residual blocks and a residual block with a multi-head self-attention mechanism; the fourth residual module includes: a dotted residual block, a common residual block and a residual block with a multi-head self-attention mechanism; A training module, used for training the bronchial tuberculosis diagnosis model incorporating multi-head self-attention based on the image sample data set; The bronchial tuberculosis classification determination module is used to input the user's bronchial endoscope image to be processed into a trained bronchial tuberculosis diagnosis model that incorporates multi-head self-attention to obtain the classification of bronchial tuberculosis.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the bronchial tuberculosis auxiliary determination method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the bronchial tuberculosis auxiliary determination method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the bronchial tuberculosis auxiliary determination method according to any one of claims 1 to 6 are implemented.

Images