Bronchial tuberculosis auxiliary determination method, device, equipment, medium and product

Abstract

The application discloses a bronchial tuberculosis auxiliary determination method and device, equipment, medium and product, relates to the field of bronchial tuberculosis auxiliary diagnosis, and includes the following steps: constructing an image sample data set; constructing a bronchial tuberculosis diagnosis model with multi-head self-attention, including a first residual module, a second residual module, a third residual module and a fourth residual module; the first residual module includes a plurality of ordinary residual blocks; the second residual module includes a dotted line residual block and a plurality of ordinary residual blocks; the third residual module includes a dotted line residual block, a plurality of ordinary residual blocks and a residual block with a multi-head self-attention mechanism; the fourth residual module includes a dotted line residual block, an ordinary residual block and a residual block with a multi-head self-attention mechanism; the bronchial tuberculosis diagnosis model with multi-head self-attention is trained, and the typing of bronchial tuberculosis is obtained based on the trained model. The application can greatly improve the diagnosis accuracy of bronchial tuberculosis.

Description

Technical Field

[0001] This application relates to the field of auxiliary diagnosis of bronchial tuberculosis, and in particular to a method, apparatus, equipment, medium and product for auxiliary determination of bronchial tuberculosis. Background Technology

[0002] Tuberculosis (TB) is one of the world's leading infectious disease killers, causing millions of cases annually and killing more than 3,500 people every day. The WHO's "Global Tuberculosis Report 2023," released on November 7, 2023, indicated that in 2022, my country had 748,000 new TB cases (95% CI: 634,000–872,000), ranking third globally, with approximately 250,000 new cases of bronchial tuberculosis annually. It is mainly distributed in the less developed central and western regions.

[0003] In my country, over 60% of cases result in severe complications such as atelectasis, bronchial stenosis, impaired lung function, and lung destruction, sometimes requiring lobectomy for a cure. This is primarily because bronchial tuberculosis lacks specific clinical symptoms and imaging findings, relying mainly on bronchoscopy for accurate diagnosis. Accurate diagnosis requires experienced physicians, leading to frequent misdiagnosis and missed diagnosis in underdeveloped areas, thus delaying optimal treatment. Therefore, if an AI-based cloud-based diagnostic platform could intelligently diagnose bronchoscopic images uploaded by patients and provide accurate classification suggestions, patients in underdeveloped areas or those lacking experienced physicians could easily receive accurate diagnoses, effectively reducing misdiagnosis and missed diagnosis of bronchial tuberculosis, enabling early detection and treatment, and minimizing the risk of tuberculosis transmission.

[0004] Therefore, improving the diagnostic accuracy of tuberculosis in patients in underdeveloped areas has become an urgent problem to be solved in this field. Summary of the Invention

[0005] The purpose of this application is to provide a method, device, equipment, medium, and product for the auxiliary diagnosis of bronchial tuberculosis, which can greatly improve the diagnostic accuracy of tuberculosis in underdeveloped areas and reduce missed diagnoses and misdiagnoses.

[0006] To achieve the above objectives, this application provides the following solution:

[0007] In a first aspect, this application provides a method for the auxiliary determination of bronchial tuberculosis, comprising:

[0008] Construct an image sample dataset;

[0009] A diagnostic model for bronchial tuberculosis incorporating multi-head self-attention is constructed. This model comprises a first residual module, a second residual module, a third residual module, and a fourth residual module. The first residual module includes multiple ordinary residual blocks. The second residual module includes a dashed residual block and multiple ordinary residual blocks. The third residual module includes a dashed residual block, multiple ordinary residual blocks, and a residual block with a multi-head self-attention mechanism. The fourth residual module includes a dashed residual block, an ordinary residual block, and a residual block with a multi-head self-attention mechanism.

[0010] The bronchial tuberculosis diagnostic model incorporating multi-head self-attention was trained based on the image sample dataset.

[0011] The images from the user's bronchial endoscopy are input into a trained bronchial tuberculosis diagnostic model incorporating multi-head self-attention to obtain the classification of bronchial tuberculosis.

[0012] Optionally, the bronchial tuberculosis diagnostic model incorporating multi-head self-attention further includes:

[0013] 7×7 convolutional layers, pooling layers, global average pooling layers, and fully connected layers;

[0014] The 7×7 convolutional layer, pooling layer, first residual module, second residual module, third residual module, fourth residual module, global average pooling layer, and fully connected layer are connected in sequence.

[0015] Optionally, the first residual module includes a first ordinary residual block, a second ordinary residual block, and a third ordinary residual block;

[0016] The first ordinary residual block, the second ordinary residual block, and the third ordinary residual block each include two 3×3 convolutional layers;

[0017] The first ordinary residual block, the second ordinary residual block, and the third ordinary residual block are connected in sequence;

[0018] The second residual module includes: a first dashed residual block, a fourth ordinary residual block, a fifth ordinary residual block, and a sixth ordinary residual block;

[0019] The first dashed residual block, the fourth ordinary residual block, the fifth ordinary residual block, and the sixth ordinary residual block all include two 3×3 convolutional layers;

[0020] The first dashed residual block, the fourth ordinary residual block, the fifth ordinary residual block, and the sixth ordinary residual block are connected in sequence;

[0021] The third residual module includes: a second dashed residual block, a seventh ordinary residual block, an eighth ordinary residual block, a ninth ordinary residual block, a tenth ordinary residual block, and a first residual block with a multi-head self-attention mechanism;

[0022] The second dashed residual block, the seventh ordinary residual block, the eighth ordinary residual block, the ninth ordinary residual block, and the tenth ordinary residual block all include two 3×3 convolutional layers; the first residual block with multi-head self-attention mechanism includes a 3×3 convolutional layer and a convolutional layer incorporating multi-head self-attention mechanism.

[0023] The fourth residual module includes: the third dashed residual block, the eleventh ordinary residual block, and the second residual block with a multi-head self-attention mechanism;

[0024] The third dashed residual block and the eleventh ordinary residual block both include two 3×3 convolutional layers, and the second residual block with multi-head self-attention mechanism includes a 3×3 convolutional layer and a convolutional layer incorporating multi-head self-attention mechanism.

[0025] Optionally, training the bronchial tuberculosis diagnostic model incorporating multi-head self-attention based on the image sample dataset specifically includes the following steps:

[0026] The image sample dataset is preprocessed;

[0027] Initialize the weights of the bronchial tuberculosis diagnostic model incorporating multi-head self-attention;

[0028] The parameters of the bronchial tuberculosis diagnostic model incorporating multi-head self-attention are set; the parameters include: model input size, training period, batch size, optimizer, loss function, base learning rate, and weight decay rate;

[0029] The actual output of a bronchial tuberculosis diagnostic model incorporating multi-head self-attention through forward propagation computation;

[0030] Construct the loss function;

[0031] Calculate the loss value based on the loss function and the actual output;

[0032] The gradient of the loss value relative to the parameters of the bronchial tuberculosis diagnostic model incorporating multi-head self-attention is calculated by backpropagation, and the gradient of the loss is propagated from the output layer to the input layer by the chain rule.

[0033] The optimizer updates the parameters of the bronchial tuberculosis diagnostic model incorporating multi-head self-attention based on the gradient until the bronchial tuberculosis diagnostic model incorporating multi-head self-attention converges.

[0034] Optionally, the expression for the loss function is as follows:

[0035] L Total =L WCE +α×L red ;

[0036] Among them, L Total It is the total loss, L WCE It is a weighted classification loss, L red It is a color feature loss designed based on doctors' professional knowledge of diagnosing bronchial tuberculosis, and α is a trainable weight parameter.

[0037] Optionally, the gradient of the loss value relative to the parameters of the bronchial tuberculosis diagnostic model incorporating multi-head self-attention, calculated via backpropagation, specifically uses the following formula:

[0038]

[0039] Where θ are the parameters of the model. It is the gradient of the classification loss with respect to the parameters. It is the gradient of the color loss with respect to the parameters.

[0040] Secondly, this application provides an auxiliary device for the determination of bronchial tuberculosis, the auxiliary device for the determination of bronchial tuberculosis comprising:

[0041] The dataset building module is used to build image sample datasets;

[0042] A diagnostic model construction module is used to construct a bronchial tuberculosis diagnostic model incorporating multi-head self-attention. The bronchial tuberculosis diagnostic model incorporating multi-head self-attention includes: a first residual module, a second residual module, a third residual module, and a fourth residual module. The first residual module includes multiple ordinary residual blocks; the second residual module includes a dashed residual block and multiple ordinary residual blocks; the third residual module includes a dashed residual block, multiple ordinary residual blocks, and a residual block with a multi-head self-attention mechanism; the fourth residual module includes a dashed residual block, a ordinary residual block, and a residual block with a multi-head self-attention mechanism.

[0043] The training module is used to train the bronchial tuberculosis diagnostic model incorporating multi-head self-attention based on the image sample dataset.

[0044] The bronchial tuberculosis classification module is used to input the images from the user's bronchial endoscopy into a trained bronchial tuberculosis diagnostic model incorporating multi-head self-attention, and obtain the classification of bronchial tuberculosis.

[0045] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the bronchial tuberculosis auxiliary determination method described in any one of the above.

[0046] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the bronchial tuberculosis auxiliary determination method described above.

[0047] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the bronchial tuberculosis auxiliary determination method described above.

[0048] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0049] This application provides a method, device, equipment, medium, and product for the auxiliary identification of bronchial tuberculosis. In the ResNet34 network, the second convolution in the last residual block of the third and fourth residual modules is replaced with a multi-head self-attention mechanism. This allows the model to simultaneously focus on global and local features of the bronchial image, significantly improving the model's accuracy and generalization ability. Furthermore, adding an attention mechanism to the last residual block of the third and fourth residual modules, compared to adding it only to the fourth residual module, enables global modeling of mid-level and high-level features by adding mid-level global features. On the one hand, it captures richer global feature information, further improving the model's accuracy. On the other hand, the model's generalization ability may be stronger. Additionally, integrating physician diagnostic expertise into model training, designing a corresponding loss function, and jointly optimizing it with a weighted classification loss function allows the model to learn to focus its attention on key areas during training, combined with features automatically extracted by the deep learning model, thereby improving classification accuracy. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a flowchart illustrating an auxiliary method for determining bronchial tuberculosis in one embodiment of this application.

[0052] Figure 2This is a schematic diagram of a bronchial tuberculosis diagnostic model incorporating multi-head self-attention in one embodiment of this application;

[0053] Figure 3 This is a functional diagram of the cloud service platform website for bronchial tuberculosis auxiliary diagnosis in one embodiment of this application;

[0054] Figure 4 This is a schematic diagram of the workflow of the cloud service platform system for auxiliary diagnosis of bronchial tuberculosis in one embodiment of this application;

[0055] Figure 5 This is a schematic diagram of the diagnostic interface of the cloud service platform for artificial intelligence-assisted diagnosis of bronchial tuberculosis in one embodiment of this application;

[0056] Figure 6 This is a schematic diagram of the historical record interface of the cloud service platform for artificial intelligence-assisted diagnosis of bronchial tuberculosis in one embodiment of this application;

[0057] Figure 7 This is a schematic diagram of the popular science interface of the cloud service platform for artificial intelligence-assisted diagnosis of bronchial tuberculosis in one embodiment of this application;

[0058] Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0059] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0060] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0061] Figure 1 This is a flowchart illustrating an auxiliary method for determining bronchial tuberculosis in one embodiment of this application. See also... Figure 1 The method in this application includes:

[0062] Step 101: Construct an image sample dataset.

[0063] Specifically, this application establishes a database of over 20,000 images from bronchoscopic procedures, all of which have been annotated by chief physicians to ensure accuracy.

[0064] Step 102: Construct a bronchial tuberculosis diagnostic model incorporating multi-head self-attention; the bronchial tuberculosis diagnostic model incorporating multi-head self-attention includes: a first residual module, a second residual module, a third residual module, and a fourth residual module; wherein, the first residual module includes multiple ordinary residual blocks; the second residual module includes a dashed residual block and multiple ordinary residual blocks; the third residual module includes: a dashed residual block, multiple ordinary residual blocks, and a residual block with a multi-head self-attention mechanism; the fourth residual module includes: a dashed residual block, a ordinary residual block, and a residual block with a multi-head self-attention mechanism.

[0065] This application proposes a novel diagnostic model for bronchial tuberculosis based on the ResNet34 framework and incorporating a multi-head self-attention mechanism. This model, based on the ResNet34 framework and the multi-head self-attention (MHSA) mechanism, constructs a bronchial tuberculosis diagnostic model (ResNet34-MHSA) based on the ResNet34 framework and incorporating the MHSA mechanism. Figure 2 As shown in the figure. MHSA is a multi-head self-attention mechanism.

[0066] The ResNet34 model is a shallow network. Shallow networks have a small number of parameters, so the cost of running a training program is not high, and the trained parameter file is also relatively small, making it suitable for hosting on cloud service platforms.

[0067] Combination Figure 2 The model specifically includes:

[0068] The system consists of a 7×7 convolutional layer, a pooling layer, a first residual module, a second residual module, a third residual module, a fourth residual module, a global average pooling layer, and a fully connected layer. The 7×7 convolutional layer, pooling layer, first residual module, second residual module, third residual module, fourth residual module, global average pooling layer, and fully connected layer are connected sequentially.

[0069] Specifically, the first residual module includes a first ordinary residual block, a second ordinary residual block, and a third ordinary residual block;

[0070] The first ordinary residual block, the second ordinary residual block, and the third ordinary residual block each include two 3×3 convolutional layers;

[0071] The first ordinary residual block, the second ordinary residual block, and the third ordinary residual block are connected in sequence;

[0072] The second residual module includes: a first dashed residual block, a fourth ordinary residual block, a fifth ordinary residual block, and a sixth ordinary residual block;

[0073] The first dashed residual block, the fourth ordinary residual block, the fifth ordinary residual block, and the sixth ordinary residual block all include two 3×3 convolutional layers;

[0074] The first dashed residual block, the fourth ordinary residual block, the fifth ordinary residual block, and the sixth ordinary residual block are connected in sequence;

[0075] The third residual module includes: a second dashed residual block, a seventh ordinary residual block, an eighth ordinary residual block, a ninth ordinary residual block, a tenth ordinary residual block, and a first residual block with a multi-head self-attention mechanism;

[0076] The second dashed residual block, the seventh ordinary residual block, the eighth ordinary residual block, the ninth ordinary residual block, and the tenth ordinary residual block all include two 3×3 convolutional layers; the first residual block with multi-head self-attention mechanism includes a 3×3 convolutional layer and a convolutional layer incorporating multi-head self-attention mechanism.

[0077] The fourth residual module includes: the third dashed residual block, the eleventh ordinary residual block, and the second residual block with a multi-head self-attention mechanism;

[0078] The third dashed residual block and the eleventh ordinary residual block both include two 3×3 convolutional layers, and the second residual block with multi-head self-attention mechanism includes a 3×3 convolutional layer and a convolutional layer incorporating multi-head self-attention mechanism.

[0079] Step 103: Train the bronchial tuberculosis diagnostic model incorporating multi-head self-attention based on the image sample dataset.

[0080] The specific training process is as follows:

[0081] A database of over 20,000 bronchoscopic image samples was established, all annotated by chief physicians to ensure accuracy. The model training process is as follows:

[0082] (1) Data preparation

[0083] Clinical data was collected in hospitals: a high-quality EBTB medical image dataset was constructed.

[0084] Data preprocessing: Preprocess the data to ensure training effectiveness.

[0085] (2) Model initialization

[0086] Definition as follows Figure 2 The diagram shows the structure of a bronchial tuberculosis diagnostic model based on the ResNet34 framework and incorporating multi-head self-attention.

[0087] Initialize weights: Initialize the weights of the model.

[0088] (3) Training the model

[0089] Set the model training parameters according to Table 1 below. The model will automatically calculate forward propagation, loss, backpropagation and update the parameters.

[0090] Table 1 Model Training Parameter Configuration

[0091]

[0092] 1) Forward propagation: The input image goes through a series of convolution, batch normalization, activation and pooling layers, and finally the output is obtained through a fully connected layer.

[0093] 2) Loss Calculation: A comprehensive loss function was constructed by combining color feature loss designed based on doctors' expertise in diagnosing bronchial tuberculosis with weighted cross-entropy loss. This design not only ensures that the model maintains efficient classification capabilities but also focuses on key color feature regions in the image that are closely related to the characteristics of type I disease.

[0094] The specific expression for the loss function is as follows:

[0095] L Total =L WCE +α×L red ;

[0096] Among them, L Total It is the total loss, L WCE It is a weighted classification loss, using cross-entropy loss to measure the difference between the model's output classification result and the actual label. red It is a color feature loss designed based on doctors' expertise in diagnosing bronchial tuberculosis. α is a trainable weight parameter used to balance the contributions of classification loss and color loss to the total loss.

[0097] In each training iteration, the model computes the output through forward propagation and calculates the loss value using the total loss function. Next, it calculates the gradient of the loss function with respect to the model parameters through backpropagation. During this process, the gradient of the color loss affects how the model parameters are updated, calculated as follows:

[0098]

[0099] In the formula: θ is the parameter of the model, It is the gradient of the classification loss with respect to the parameters. It is the gradient of the color loss with respect to the parameters.

[0100] The model will adjust its parameters based on the gradients mentioned above, so that the total loss function value gradually decreases.

[0101] 3) Backpropagation: Calculate the gradient of the loss with respect to the model parameters. Using the chain rule, propagate the gradient of the loss from the output layer all the way to the input layer.

[0102] 4) Parameter Update: The optimizer (SGD) updates the parameters based on the calculated gradients. The optimizer adjusts the parameters of each layer to reduce the loss. This process is repeated for multiple training epochs until the model converges.

[0103] After the model training is completed, an optimized model can be obtained for the auxiliary diagnosis of bronchial tuberculosis.

[0104] Step 104: Input the images from the user's bronchial endoscopy into the trained bronchial tuberculosis diagnostic model incorporating multi-head self-attention to obtain the classification of bronchial tuberculosis.

[0105] Specifically, the internal data processing procedure of the bronchial tuberculosis diagnostic model incorporating multi-head self-attention is as follows:

[0106] ① The 224*224 pixel image information is first processed by a 7x7 convolutional layer with conv(7,7,cin=3,cout=64,padding=3,stride=2) convolution operation, which can extract the initial features of the image in a larger field of view, reduce the image size by half and increase the depth of the feature map, and obtain a 64*112*112 feature matrix.

[0107] ② Input the feature matrix obtained in step ① into the pooling layer and perform a conv(3,3,cin=3,cout=64,padding=1,stride=2) convolution operation to reduce the size by half, resulting in a 64*56*56 feature matrix. The role of the pooling layer is to highlight salient features and reduce the computational cost of subsequent convolutional layers, thereby improving the overall computational efficiency of the network.

[0108] ③ Input the feature matrix obtained in step ② into the first residual module. This module contains three residual blocks, each with two 3x3 convolutional layers. The input and output of each residual block are added together to form a residual connection. The specific operation of each residual block is shown in the following formula. After the operation of this module, a 64*56*56 feature matrix is ​​obtained. The main function of this module is to further extract and enhance features, while reducing the gradient vanishing problem through residual connections.

[0109] The calculation formula for a typical residual block is as follows:

[0110] y=F(x,{W i})+W 1×1 *x

[0111] Where y represents the output, F(x,{W i})=ReLU(BN(W2*ReLU(BN(W1*x)))), where x represents the input, BN represents batch normalization, a regularization technique; ReLU represents nonlinear computation, ReLU(x)=max(0,x). Max is the maximum value operation; W i In the diagram, i = 1, 2, W1 represents the first convolution operation and its weights, W2 represents the second convolution operation and its weights, and W... 1×1 This represents a 1x1 convolution operation and its weights.

[0112] ④ The feature matrix obtained in step ③ is input into the second residual module. This module contains four residual blocks, each consisting of two 3x3 convolutional layers. The first residual block is a dashed residual block, calculated as shown in the formula below. The other three residual blocks are ordinary residual blocks, calculated using the formula in step ③. The second residual module outputs a 128*28*28 feature matrix. Compared to the first residual module, this stage of feature extraction is more in-depth, learning more complex features through more residual blocks.

[0113] The calculation formula for the dashed line residual block is as follows:

[0114] y=F(x,{W i})+W 1×1 *x

[0115] ⑤ The feature matrix obtained in step ④ is input into the third residual module. This module contains three residual blocks, each consisting of two 3x3 convolutional layers. The first residual block is a dashed residual block, calculated using the formula in step ⑤. The next four residual blocks are ordinary residual blocks, calculated using the formula in step ③. The last residual block is a residual block with Multi-Head Self-Attention (MHSA), meaning its second convolutional layer incorporates MHSA, calculated as shown in Equation 2. MHSA is introduced to capture the relationships and contextual information between distant features, improving the globality of feature representation and enhancing feature expressiveness. The third residual module outputs a 256*14*14 feature matrix. The third residual module is the core of feature extraction, extracting high-level features through a large number of residual blocks.

[0116] The formula for calculating residual blocks with multi-head self-attention mechanism (MHSA) is as follows:

[0117] y = ReLU(BN(W) 3×3 *MHSA(X)))+W 1×1 *x

[0118] Where MHSA(X) = Concat(O1,O2,…,O) hW o W 3×3 W 1×1 This represents a 3x3 convolution operation and weights. `Concat` is a merging function that combines the outputs of multiple heads that have performed parallel self-attention calculations on different sub-controls. `MHSA(X) = Concat(O1, O2, ..., O...)` h W o O i =Attention(O1,O1,O1).

[0119] ⑥ The image obtained in step ⑤ is input into the fourth residual module. This module contains three residual blocks. The first residual block is a dashed residual block, calculated using the formula in step ④. The next residual block is a regular residual block, calculated using the formula in step ①. The last residual block is a residual block with Multi-Head Self-Attention (MHSA), meaning its second convolutional layer incorporates MHSA, calculated using the formula in step ⑤. The final output is a 512*7*7 feature matrix. The fourth residual module integrates and classifies the previously extracted features, enabling the model to simultaneously focus on both global and local image features, thus improving the model's accuracy.

[0120] ⑦ The feature matrix obtained in step ⑥ is input into the global average pooling layer (avg pool), which outputs a 512*1*1 feature vector. The global average pooling layer extracts global features through dimensionality reduction, replaces fully connected layers, smooths feature maps, and promotes the interpretability of feature maps.

[0121] ⑧ Based on the feature vector output in step ⑦, a fully connected layer is used to perform the classification task, ultimately classifying the bronchial tuberculosis images. The specific calculation formula for the fully connected layer is:

[0122] 1) Linear transformation z = Wh + b where:

[0123] Where h is the input feature vector with a dimension of 512; W is the weight matrix of the fully connected layer with a dimension of 4×512; and b is the bias vector of the fully connected layer with a dimension of 4.

[0124] 2) Convert to a probability distribution using the Softmax function: p = Softmax(z)

[0125] The formula for the Softmax function is:

[0126]

[0127] Where, p i z is the probability of the i-th category. i It is the i-th element of the linear transformation output z.

[0128] To further demonstrate the diagnostic accuracy of the model in this application, we conducted experiments using the EBTB image database constructed in this application. The dataset was divided into training, validation, and test sets in an 8:1:1 ratio.

[0129] Experimental results show that:

[0130] The overall classification accuracy of existing models is 81.04%;

[0131] The overall classification accuracy of the model in this application is 88.06%, which shows a significant improvement in diagnostic accuracy.

[0132] This application also provides a cloud service platform, which mainly consists of a cloud server and a domain name. By binding the domain name to the IP address of the cloud server and setting the port number, the program on the cloud server can be bound to the domain name. The web page logic program app.py file is run on the cloud server, connecting the redirection logic between each web page and enabling interaction between the web pages and the program under the domain name.

[0133] In the program, set it to open the webpage homepage, then click the login button to enter the login interface. Once the entered username and password match those in the database, you can access the AI-assisted diagnostic cloud service platform.

[0134] When a user uploads a bronchoscopic image to the webpage, the program calls a pre-trained bronchial tuberculosis diagnostic model to read the image. The model calculates whether the image indicates bronchial tuberculosis; if so, it automatically provides a classification suggestion. If not, it displays "No bronchial tuberculosis." The image, along with the diagnosis and classification results, is then saved to the "History" section.

[0135] Figure 3 This is a functional diagram of the cloud service platform website for bronchial tuberculosis auxiliary diagnosis in one embodiment of this application. See also: Figure 3 The website's functional diagram primarily revolves around the numerous functions of the user center. The flowchart begins with "Login," indicating that users must first log in to the platform. If they already have an account, they enter their username and password to log in, and the system will verify this information against the database. If they haven't registered, they are redirected to the registration page to complete the information. After successful registration, the information is encrypted using the AES algorithm and stored in the database. Subsequently, users will enter the "User Center" interface. This interface provides entry points to multiple functional modules, including account information, user feedback, inquiries, tuberculosis information, history, service usage, and user instructions.

[0136] Within the user center, the process is broken down into several functions. First, users can manage their accounts, including changing passwords and binding mobile phones. If users have any questions during use, they can ask questions and get help through the "Questions" module. In addition, the platform also provides a "Science Popularization" module to convey medical knowledge and prevention methods related to bronchial tuberculosis to users.

[0137] At the heart of the user center is a prominent "Diagnostic Services" module. This module is the core function of the entire platform, providing auxiliary diagnostic services for bronchial tuberculosis. Users can upload their medical records, examination results, and other information here. The platform uses advanced algorithms and models to process and analyze the data, thereby providing diagnostic suggestions or predictions.

[0138] In addition to diagnostic services, the user center also offers "User Feedback" and "History" modules. Users can use the "User Feedback" module to provide feedback on their experience or offer suggestions, helping the platform to continuously optimize and improve. The "History" module records past user actions and behaviors, allowing users to easily review and revisit them at any time.

[0139] Furthermore, regarding user data security in the bronchial tuberculosis auxiliary diagnosis cloud service platform system, the backend application ensures that users are authenticated when logging in and using the service. Here, symmetric encryption technology was chosen to ensure user data security, employing the AES-CBC mode to further guarantee the secure storage and transmission of user data.

[0140] Database AES encryption implementation

[0141] (1) Key and Initialization Vector (IV) Management

[0142] Key generation: The system uses a secure and reliable random number generator to generate a 256-bit AES key. This key is kept strictly confidential and used only during encryption and decryption.

[0143] Initialization Vector: To enhance encryption security, the system also generates an initialization vector (IV) the same size as the encryption block. The IV is unique in each encryption process and is not the same as the key. It is used for block chaining in CBC mode to ensure that even the same plaintext data will not produce the same ciphertext in two encryption processes.

[0144] (2) Data encryption process

[0145] Plaintext data preparation: When user data needs to be stored or transmitted, the system first ensures that the data exists in plaintext form. This includes sensitive information such as user personal information and diagnostic results.

[0146] AES Encryption: The system uses AES-CBC mode and the generated key and IV to encrypt plaintext data. The encryption process involves multiple rounds of iterative operations, including byte substitution, row shifting, column obfuscation, and round key addition, ultimately converting plaintext data into ciphertext data.

[0147] Ciphertext storage: The encrypted ciphertext, along with the IV, is securely stored in the cloud service platform's database.

[0148] (3) Data decryption process

[0149] Encrypted data retrieval: When accessing user data, the system first retrieves the encrypted data and IV from the database.

[0150] AES decryption: Using the same key, AES-CBC mode, and IV as the encryption key, the system decrypts the ciphertext data. The decryption process is the reverse of the encryption process, including steps such as reversing the round key addition, reverse column obfuscation, reverse row shifting, and reverse byte substitution, ultimately restoring the ciphertext data to the original plaintext data.

[0151] Plaintext data usage: Decrypted plaintext data can be used for subsequent processing, analysis, or display to users.

[0152] In summary, by employing the AES algorithm for data encryption, the cloud service platform system for bronchial tuberculosis auxiliary diagnosis can ensure the security of user data during storage and transmission.

[0153] See Figure 4 This is a schematic diagram of the workflow of the cloud service platform system for auxiliary diagnosis of bronchial tuberculosis. The workflow includes:

[0154] (1) Users can open any browser in a place with network access, enter the domain name of this EBTB network diagnostic platform (www.ebtbpredict.asia), and press Enter to enter the platform's login page.

[0155] (2) Enter your account and password and click Login. If the account and password are entered incorrectly, the platform will pop up an error box, prompting the user that the account or password is incorrect and to re-enter it. If the account and password are entered correctly, you can enter the platform's diagnostic page. Then, click the corresponding function button in the upper left corner of the diagnostic page and select the local EBTB image. At this time, the platform will also perform a preliminary screening of the EBTB image selected by the user. When the selected image is not in JPG format, the webpage will pop up a warning box and prompt you to select the correct format EBTB image. Then, the platform will display the uploaded EBTB image in the image box on the webpage and preprocess the uploaded image in the server background to ensure that the input image can meet the input requirements of the platform's diagnostic model.

[0156] (3) Click the EBTB lesion diagnosis button in the upper right corner of the detection page, and transfer the image processed in the second step to the XX network model on the cloud server corresponding to the domain name for classification to obtain the diagnosis result.

[0157] (4) This platform displays the diagnostic results obtained from the server in the third step in the lower right corner of the page, and displays the EBTB lesion type with the highest diagnostic probability at the top. At the same time, considering the establishment of the EBTB dataset, this platform also performs the function of the save button when the EBTB lesion detection button is clicked. With the user's knowledge and consent, the images transmitted to the platform are saved in the EBTB database, continuously improving the EBTB dataset and contributing to the development of subsequent EBTB medical diagnosis and intelligent diagnosis.

[0158] Figure 5 This is a schematic diagram of the diagnostic interface of an AI-assisted cloud service platform for bronchial tuberculosis diagnosis, as described in one embodiment of this application. The AI-assisted cloud service platform for bronchial tuberculosis is a website providing services to patients with bronchial tuberculosis. After registering and logging into the cloud platform and entering the EBTB detection system, the patient clicks "Upload Image" to upload an image from their bronchoscopy report to the cloud platform. Then, by clicking the "Start Diagnosis" button, the cloud platform can identify the bronchial tuberculosis in the image. Through this interface, the platform provides classification prompts, allowing users to clearly see the diagnostic results.

[0159] Figure 6 This is a schematic diagram of the historical record interface of the cloud service platform for artificial intelligence-assisted diagnosis of bronchial tuberculosis in one embodiment of this application. Through this interface, users can see the historical consultation time and diagnosis results.

[0160] Figure 7 This is a schematic diagram of the popular science interface of the cloud service platform for artificial intelligence-assisted diagnosis of bronchial tuberculosis in one embodiment of this application. In addition to intelligent diagnosis, the website can also store diagnostic history records and provide popular science information on tuberculosis diagnosis and treatment, so that patients can understand bronchial tuberculosis and better cooperate with doctors in treatment. If patients do not know how to use the cloud service platform, they can click the "Instructions for Use" button to learn how to use this detection system.

[0161] Based on the same inventive concept, this application also provides a bronchial tuberculosis auxiliary determination device for implementing the above-mentioned auxiliary determination method for bronchial tuberculosis. The solution provided by this device is similar to the solution described in the above-described method. Therefore, the specific limitations of one or more embodiments of the bronchial tuberculosis auxiliary determination device provided below can be found in the limitations of the bronchial tuberculosis auxiliary determination method described above, and will not be repeated here.

[0162] In one exemplary embodiment, a device for assisting in the diagnosis of bronchial tuberculosis is provided, comprising:

[0163] The dataset building module is used to build image sample datasets.

[0164] A diagnostic model construction module is used to construct a bronchial tuberculosis diagnostic model incorporating multi-head self-attention. The bronchial tuberculosis diagnostic model incorporating multi-head self-attention includes: a first residual module, a second residual module, a third residual module, and a fourth residual module. The first residual module includes multiple ordinary residual blocks; the second residual module includes a dashed residual block and multiple ordinary residual blocks; the third residual module includes a dashed residual block, multiple ordinary residual blocks, and a residual block with a multi-head self-attention mechanism; the fourth residual module includes a dashed residual block, an ordinary residual block, and a residual block with a multi-head self-attention mechanism.

[0165] The training module is used to train the bronchial tuberculosis diagnostic model incorporating multi-head self-attention based on the image sample dataset.

[0166] The bronchial tuberculosis classification module is used to input the images from the user's bronchial endoscopy into a trained bronchial tuberculosis diagnostic model incorporating multi-head self-attention, and obtain the classification of bronchial tuberculosis.

[0167] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 8 As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data for auxiliary determination of bronchial tuberculosis. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for auxiliary determination of bronchial tuberculosis.

[0168] Those skilled in the art will understand that Figure 8The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0169] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0170] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0171] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0172] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0173] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0174] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0175] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0176] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for auxiliary determination of bronchial tuberculosis, characterized in that, The auxiliary method for diagnosing bronchial tuberculosis includes: Construct an image sample dataset; A diagnostic model for bronchial tuberculosis incorporating multi-head self-attention is constructed. This model comprises a first residual module, a second residual module, a third residual module, and a fourth residual module. The first residual module includes multiple ordinary residual blocks; the second residual module includes a dashed residual block and multiple ordinary residual blocks; the third residual module includes a dashed residual block, multiple ordinary residual blocks, and a residual block with a multi-head self-attention mechanism; and the fourth residual module includes a dashed residual block, an ordinary residual block, and a residual block with a multi-head self-attention mechanism. The bronchial tuberculosis diagnostic model incorporating multi-head self-attention was trained based on the image sample dataset. The images from the user's bronchial endoscopy are input into a trained bronchial tuberculosis diagnostic model incorporating multi-head self-attention to obtain the classification of bronchial tuberculosis. The first residual module includes a first ordinary residual block, a second ordinary residual block, and a third ordinary residual block; The first ordinary residual block, the second ordinary residual block, and the third ordinary residual block each include two 3×3 convolutional layers; The first ordinary residual block, the second ordinary residual block, and the third ordinary residual block are connected in sequence; The second residual module includes: a first dashed residual block, a fourth ordinary residual block, a fifth ordinary residual block, and a sixth ordinary residual block; The first dashed residual block, the fourth ordinary residual block, the fifth ordinary residual block, and the sixth ordinary residual block all include two 3×3 convolutional layers; The first dashed residual block, the fourth ordinary residual block, the fifth ordinary residual block, and the sixth ordinary residual block are connected in sequence; The third residual module includes: a second dashed residual block, a seventh ordinary residual block, an eighth ordinary residual block, a ninth ordinary residual block, a tenth ordinary residual block, and a first residual block with a multi-head self-attention mechanism; The second dashed residual block, the seventh ordinary residual block, the eighth ordinary residual block, the ninth ordinary residual block, and the tenth ordinary residual block all include two 3×3 convolutional layers; the first residual block with multi-head self-attention mechanism includes a 3×3 convolutional layer and a convolutional layer incorporating multi-head self-attention mechanism. The fourth residual module includes: the third dashed residual block, the eleventh ordinary residual block, and the second residual block with a multi-head self-attention mechanism; The third dashed residual block and the eleventh ordinary residual block both include two 3×3 convolutional layers, and the second residual block with multi-head self-attention mechanism includes a 3×3 convolutional layer and a convolutional layer incorporating multi-head self-attention mechanism.

2. The method for auxiliary determination of bronchial tuberculosis according to claim 1, characterized in that, The bronchial tuberculosis diagnostic model incorporating multi-head self-attention also includes: 7×7 convolutional layers, pooling layers, global average pooling layers, and fully connected layers; The 7×7 convolutional layer, pooling layer, first residual module, second residual module, third residual module, fourth residual module, global average pooling layer, and fully connected layer are connected in sequence.

3. The method for auxiliary determination of bronchial tuberculosis according to claim 1, characterized in that, Training the bronchial tuberculosis diagnostic model incorporating multi-head self-attention based on the aforementioned image sample dataset specifically includes the following steps: The image sample dataset is preprocessed; Initialize the weights of the bronchial tuberculosis diagnostic model incorporating multi-head self-attention; The parameters of the bronchial tuberculosis diagnostic model incorporating multi-head self-attention are set; the parameters include: model input size, training period, batch size, optimizer, loss function, base learning rate, and weight decay rate; The actual output of a bronchial tuberculosis diagnostic model incorporating multi-head self-attention through forward propagation computation; Construct the loss function; Calculate the loss value based on the loss function and the actual output; The gradient of the loss value relative to the parameters of the bronchial tuberculosis diagnostic model incorporating multi-head self-attention is calculated by backpropagation, and the gradient of the loss is propagated from the output layer to the input layer by the chain rule. The optimizer updates the parameters of the bronchial tuberculosis diagnostic model incorporating multi-head self-attention based on the gradient until the bronchial tuberculosis diagnostic model incorporating multi-head self-attention converges.

4. The method for auxiliary determination of bronchial tuberculosis according to claim 3, characterized in that, The expression for the loss function is as follows: ； in, It is the total loss. It is a weighted classification loss. The color feature loss was designed based on doctors' professional knowledge of diagnosing bronchial tuberculosis. These are trainable weight parameters.

5. The method for auxiliary determination of bronchial tuberculosis according to claim 3, characterized in that, The gradient of the loss value relative to the parameters of the bronchial tuberculosis diagnostic model incorporating multi-head self-attention, calculated via backpropagation, specifically uses the following formula: ； in, These are the parameters of the model. It is the gradient of the classification loss with respect to the parameters. It is the gradient of the color loss with respect to the parameters.

6. A device for auxiliary diagnosis of bronchial tuberculosis, characterized in that, The auxiliary device for diagnosing bronchial tuberculosis includes: The dataset building module is used to build image sample datasets; A diagnostic model construction module is used to construct a bronchial tuberculosis diagnostic model incorporating multi-head self-attention. The bronchial tuberculosis diagnostic model incorporating multi-head self-attention includes: a first residual module, a second residual module, a third residual module, and a fourth residual module. The first residual module includes multiple ordinary residual blocks; the second residual module includes a dashed residual block and multiple ordinary residual blocks; the third residual module includes a dashed residual block, multiple ordinary residual blocks, and a residual block with a multi-head self-attention mechanism; the fourth residual module includes a dashed residual block, a ordinary residual block, and a residual block with a multi-head self-attention mechanism. The training module is used to train the bronchial tuberculosis diagnostic model incorporating multi-head self-attention based on the image sample dataset. The bronchial tuberculosis classification module is used to input the images of the user's bronchial endoscopy into the trained bronchial tuberculosis diagnostic model incorporating multi-head self-attention, and obtain the classification of bronchial tuberculosis. The first residual module includes a first ordinary residual block, a second ordinary residual block, and a third ordinary residual block; The first ordinary residual block, the second ordinary residual block, and the third ordinary residual block each include two 3×3 convolutional layers; The first ordinary residual block, the second ordinary residual block, and the third ordinary residual block are connected in sequence; The second residual module includes: a first dashed residual block, a fourth ordinary residual block, a fifth ordinary residual block, and a sixth ordinary residual block; The first dashed residual block, the fourth ordinary residual block, the fifth ordinary residual block, and the sixth ordinary residual block all include two 3×3 convolutional layers; The first dashed residual block, the fourth ordinary residual block, the fifth ordinary residual block, and the sixth ordinary residual block are connected in sequence; The third residual module includes: a second dashed residual block, a seventh ordinary residual block, an eighth ordinary residual block, a ninth ordinary residual block, a tenth ordinary residual block, and a first residual block with a multi-head self-attention mechanism; The second dashed residual block, the seventh ordinary residual block, the eighth ordinary residual block, the ninth ordinary residual block, and the tenth ordinary residual block all include two 3×3 convolutional layers; the first residual block with multi-head self-attention mechanism includes a 3×3 convolutional layer and a convolutional layer incorporating multi-head self-attention mechanism. The fourth residual module includes: the third dashed residual block, the eleventh ordinary residual block, and the second residual block with a multi-head self-attention mechanism; The third dashed residual block and the eleventh ordinary residual block both include two 3×3 convolutional layers, and the second residual block with multi-head self-attention mechanism includes a 3×3 convolutional layer and a convolutional layer incorporating multi-head self-attention mechanism.

7. A computer device, comprising: The memory and processor contain a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the steps of the method for auxiliary determination of bronchial tuberculosis as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method for auxiliary determination of bronchial tuberculosis as described in any one of claims 1-5.

9. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method for auxiliary determination of bronchial tuberculosis as described in any one of claims 1-5.

Images