A method, system and related device for measuring the inhibition zone of drug-sensitive paper

Abstract

The present application discloses a method, system and related device for measuring the antibacterial zone of a drug-sensitive paper. It relates to the field of biomedical engineering, including: obtaining an image of a culture dish and performing image enhancement processing on the image of the culture dish to obtain a target culture dish image; obtaining drug information recorded on the drug-sensitive paper from the target culture dish image, and obtaining culture experiment information; inputting the drug information and culture experiment information into a preset decision tree model to obtain a target antibacterial zone category output by the preset decision tree model, and dividing the nodes in the preset decision tree model according to the factors affecting the antibacterial zone in the preset standard; inputting the target culture dish image and the target antibacterial zone category into a preset multimodal segmentation model to obtain the target antibacterial zone image range; converting the target antibacterial zone image range in the target culture dish image to obtain the physical size of the antibacterial zone in the culture dish. This method selects an antibacterial zone range that is more in line with the current drug-sensitive paper, and improves the accuracy of the antibacterial zone measurement.

Description

Technical Field

[0001] The present application relates to the field of biomedical engineering technology, and in particular to a method, system and related device for measuring the inhibition zone of a drug-sensitive paper sheet. Background Art

[0002] Susceptibility testing is a laboratory test used to screen for certain antibiotics that can effectively inhibit or kill specific bacteria.

[0003] In the drug sensitivity test, a filter paper containing a quantitative test drug is attached to the agar surface of the test bacteria culture dish. The filter paper records the code, concentration and other information of the test drug. The test drug in the filter paper dissolves and diffuses in the agar, forming a decreasing gradient concentration. The growth of the test bacteria within the antibacterial concentration range around the filter paper is inhibited, thereby forming a transparent inhibition zone in the agar of the culture dish. The size of the inhibition zone can reflect the sensitivity of the test bacteria to the test drug. Therefore, the measurement of the inhibition zone is one of the core steps in the drug sensitivity test.

[0004] Currently, the size of the inhibition zone in the culture dish image can be directly analyzed through image processing technology, but the existing image processing technology still has certain limitations in the accuracy of inhibition zone measurement. Summary of the invention

[0005] In view of the above problems, the present application provides a method, system and related device for measuring the inhibition zone of drug-sensitive paper sheets to achieve the purpose of improving the accuracy of inhibition zone measurement. The specific scheme is as follows:

[0006] The first aspect of the present application provides a method for measuring the inhibition zone of a drug-sensitive paper sheet, the method comprising:

[0007] Acquiring an image of a culture dish and performing image enhancement processing on the image of the culture dish to obtain an image of a target culture dish, wherein the culture dish has a drug-sensitive paper sheet;

[0008] Acquire the drug information recorded on the drug-sensitive paper from the target culture dish image, and acquire the culture experiment information;

[0009] Input the drug information and the culture experiment information into a preset decision tree model, obtain the target inhibition zone category output by the preset decision tree model, each inhibition zone category corresponds to an inhibition zone image range, and the nodes in the preset decision tree model are divided according to the influencing factors affecting the inhibition zone in the preset standard;

[0010] Inputting the target culture dish image and the target inhibition circle category into a preset multimodal segmentation model to obtain a target inhibition circle image range, wherein the target inhibition circle image range is: an inhibition circle image range among a plurality of inhibition circle image ranges segmented from the target culture dish image by the preset multimodal segmentation model, and the target inhibition circle image range corresponds to the target inhibition circle category;

[0011] The target inhibition zone image range in the target culture dish image is converted to obtain the physical size of the inhibition zone in the culture dish.

[0012] In a possible implementation, acquiring an image of a culture dish and performing image enhancement processing on the image of the culture dish to obtain a target culture dish image includes:

[0013] The image of the culture dish is enhanced by contrast-limited adaptive histogram equalization (CLAHE) to obtain the target culture dish image.

[0014] In a possible implementation, the drug information includes the name of the drug sensitivity paper and the drug concentration, the culture experiment information includes the culture medium type, the test bacteria type and the culture time, and the acquiring of the drug information recorded on the drug sensitivity paper from the target culture dish image and the acquiring of the culture experiment information include:

[0015] Identify the image area of ​​the drug-sensitive paper in the target culture dish image by using Tesseract OCR optical character recognition technology;

[0016] Acquire the drug-sensitive paper name and drug concentration recorded in the drug-sensitive paper from the image area of ​​the drug-sensitive paper;

[0017] The barcode of the culture dish is scanned by barcode recognition technology to obtain the test bacteria type, culture medium type and culture time of the culture dish.

[0018] In one possible implementation, the node types of the preset decision tree model include drug-susceptibility paper name, drug concentration, culture medium type, test bacteria type and culture time. The preset decision tree model includes multiple drug-susceptibility paper name nodes, each drug-susceptibility paper name node includes multiple drug concentration nodes, each drug concentration node includes multiple culture medium type nodes, each culture medium type node includes multiple test bacteria type nodes, each test bacteria type node includes multiple culture time nodes, and each culture time node corresponds to an inhibition zone category.

[0019] In a possible implementation, the physical size of the inhibition zone is the diameter of the inhibition zone in the culture dish, and converting the target inhibition zone image range in the target culture dish image to obtain the physical size of the inhibition zone in the culture dish includes:

[0020] The pixel size in the target inhibition zone image range is converted into a physical size to obtain the diameter of the inhibition zone in the culture dish corresponding to the target inhibition zone image range.

[0021] The second aspect of the present application provides a system for measuring the antibacterial zone of a drug-sensitive paper sheet, the system comprising:

[0022] An image acquisition unit, used for acquiring an image of a culture dish and performing image enhancement processing on the image of the culture dish to obtain an image of a target culture dish, wherein the culture dish has a drug-sensitive paper sheet;

[0023] A text acquisition unit, used to acquire the drug information recorded on the drug-sensitive paper from the target culture dish image, and acquire the culture experiment information;

[0024] A category prediction unit, used for inputting the drug information and the culture experiment information into a preset decision tree model, obtaining a target inhibition zone category output by the preset decision tree model, each inhibition zone category corresponds to an inhibition zone image range, and the nodes in the preset decision tree model are divided according to the influencing factors affecting the inhibition zone in the preset standard;

[0025] An image segmentation unit is used to input the target culture dish image and the target inhibition circle category into a preset multimodal segmentation model to obtain a target inhibition circle image range, wherein the target inhibition circle image range is: an inhibition circle image range among multiple inhibition circle image ranges segmented from the target culture dish image by the preset multimodal segmentation model, and the target inhibition circle image range corresponds to the target inhibition circle category;

[0026] The size conversion unit is used to convert the target inhibition zone image range in the target culture dish image to obtain the physical size of the inhibition zone in the culture dish.

[0027] In a possible implementation, the image acquisition unit is specifically configured as follows:

[0028] The image of the culture dish is enhanced by contrast-limited adaptive histogram equalization (CLAHE) to obtain the target culture dish image.

[0029] In a possible implementation, the drug information includes the name of the drug sensitivity paper and the drug concentration, the culture experiment information includes the culture medium type, the test bacteria type and the culture time, and the text acquisition unit is specifically configured as follows:

[0030] The image area of ​​the drug-sensitive paper in the target culture dish image is identified by Tesseract OCR optical character recognition technology; the drug-sensitive paper name and drug concentration recorded in the drug-sensitive paper are obtained from the image area of ​​the drug-sensitive paper; the barcode of the culture dish is scanned by barcode recognition technology to obtain the test bacteria type, culture medium type and culture time of the culture dish.

[0031] A third aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0032] The memory is used to store computer programs;

[0033] The processor is used to execute the computer program so that the electronic device can implement the method for measuring the inhibition zone of the drug-sensitive paper sheet according to the first aspect or any implementation manner of the first aspect.

[0034] The fourth aspect of the present application provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the method for measuring the inhibition zone of the drug-sensitive paper sheet according to the first aspect or any implementation of the first aspect.

[0035] By means of the above technical scheme, the present application provides a method, system and related device for measuring the inhibition zone of a drug-sensitive paper. The method first performs image enhancement processing on the culture dish image to obtain the target culture dish image to enhance the image features of the drug-sensitive paper and the inhibition zone in the culture dish image. The culture experiment information of the culture dish is obtained and the drug information recorded on the drug-sensitive paper is obtained from the target culture dish image, and the culture experiment information and the drug information are input into a preset decision tree model to obtain the target inhibition zone category. Since the nodes in the preset decision tree model are divided according to the factors affecting the inhibition zone in the preset standard, the culture experiment information and the drug information can be better feature-segmented to improve the accuracy of the inhibition zone classification. Since each inhibition zone category corresponds to an inhibition zone image range, the target inhibition zone image range corresponding to the target inhibition zone category can be marked in the target culture dish image through a preset multimodal segmentation model, so as to facilitate the conversion of the size of the inhibition zone in the actual culture dish according to the inhibition zone image range in the target culture dish image. This method first performs image enhancement processing on the culture dish image to improve the recognition accuracy of drug information and the accuracy of the preset multimodal segmentation model in segmenting the target culture dish image. This method predicts the category of the inhibition zone according to the preset decision tree model, and the nodes in the preset decision tree model are divided according to the factors affecting the inhibition zone in the preset standard, which improves the prediction accuracy and credibility of the inhibition zone category. The preset multimodal segmentation model selects the inhibition zone range that is more in line with the current drug-sensitive paper according to the inhibition zone category predicted by the preset decision tree model. Compared with directly analyzing the size of the inhibition zone in the culture dish image through image processing technology, this method can effectively improve the accuracy of the inhibition zone measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0037] Figure 1 A schematic diagram of a process for measuring the inhibition zone of a drug-sensitive paper sheet provided in an embodiment of the present application;

[0038] Figure 2 A schematic diagram of a culture dish provided in an embodiment of the present application;

[0039] Figure 3 A schematic diagram of the structure of a preset decision tree model provided in an embodiment of the present application;

[0040] Figure 4 A schematic diagram of a culture dish image with a marked inhibition zone provided in an embodiment of the present application;

[0041] Figure 5 A schematic diagram of the structure of a system for measuring the inhibition zone of a drug-sensitive paper sheet provided in an embodiment of the present application;

[0042] Figure 6 A hardware structure block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0044] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0045] The terms "including" and "having" and any variations thereof in the specification of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions, so that a process, method, system, product or apparatus that includes a series of units is not necessarily limited to those units, but may include other units not expressly listed or inherent to these processes, methods, products or apparatuses.

[0046] The drug sensitivity test is a laboratory testing method used to screen out some antibiotics that can effectively inhibit or kill specific bacteria. It can include paper diffusion method, dilution method (agar and broth dilution method), antibiotic concentration gradient method, etc. Among them, the paper diffusion method is to stick the filter paper containing quantitative test drugs on the agar surface of the test bacteria culture dish, and the filter paper records the code, concentration and other information of the test drugs. The test drugs in the filter paper are dissolved and diffused in the agar, forming a decreasing gradient concentration, and the growth of the test bacteria in the antibacterial concentration range around the filter paper is inhibited, thereby forming a transparent antibacterial circle in the agar of the culture dish.

[0047] Since the size of the inhibition zone can reflect the sensitivity of the test bacteria to the test drug. Therefore, the measurement of the inhibition zone is one of the core steps in the drug sensitivity test. The traditional inhibition zone measurement method manually measures the inhibition zone on the drug-sensitive paper. Although the operation is simple, it is often interfered by human factors, resulting in poor measurement accuracy and repeatability of the inhibition zone, and it takes a long time. Although there are some dedicated inhibition zone measurement instruments, the dedicated inhibition zone measurement instruments can provide higher measurement accuracy, but due to the high cost, it limits the popularization and application in the laboratory. The existing inhibition zone measurement technology also has an image processing method. The image processing technology identifies the range of the inhibition zone based on the image of the culture dish taken, but a culture dish is often affixed with multiple drug-sensitive paper sheets, and multiple drug-sensitive paper sheets may affect each other, resulting in inaccurate inhibition zones around the drug-sensitive paper sheets, resulting in inaccurate range of inhibition zones identified by image processing technology. Therefore, image processing technology still has certain limitations in the accuracy of inhibition zone measurement.

[0048] In order to solve the above problems, an embodiment of the present application provides a method for measuring the inhibition zone of a drug-sensitive paper sheet. The method performs image enhancement on the culture dish image and strengthens the image features, so that the preset multimodal segmentation model can segment multiple inhibition zone ranges for each drug-sensitive paper sheet in the culture dish image. The nodes in the preset decision tree model are divided according to the influencing factors affecting the inhibition zone in the preset standard, which improves the prediction accuracy and credibility of the inhibition zone category. The preset multimodal segmentation model can select an inhibition zone range from multiple inhibition zone ranges of each drug-sensitive paper sheet according to the classification results of the preset decision tree model, determine the inhibition zone range that the drug-sensitive paper sheet should have, and is not affected by other drug-sensitive paper sheets, thereby effectively improving the measurement accuracy.

[0049] The following is a detailed introduction to the method for measuring the inhibition zone of the drug-sensitive paper sheet in the embodiment of the present application in conjunction with the accompanying drawings.

[0050] Reference Figure 1 , Figure 1 A schematic diagram of a method for measuring the inhibition zone of a drug-sensitive paper sheet provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, a method for measuring the inhibition zone of a drug-sensitive paper sheet provided in an embodiment of the present application may include steps S10 to S14, and these steps are described in detail below.

[0051] S10, acquiring an image of the culture dish and performing image enhancement processing on the image of the culture dish to obtain an image of a target culture dish, wherein the culture dish has a drug-sensitive paper sheet.

[0052] In this embodiment, a high-resolution camera or scanner can be used to obtain an image of the culture dish, which includes the drug-sensitive paper and the antibacterial zone of the drug-sensitive paper. The image of the culture dish must be clear and cover the entire area of ​​the drug-sensitive paper. Figure 2 As shown in the schematic diagram of the culture dish, a culture dish may have multiple drug-sensitive paper sheets, and each drug-sensitive paper sheet has an inhibition zone around it. In order to increase the image features of the drug-sensitive paper sheet area in the culture dish image and the image features of the inhibition zone image, this embodiment performs image enhancement processing on the culture dish image. Specifically, this embodiment performs image enhancement processing (increasing image contrast) on the image of the culture dish by limited contrast adaptive histogram equalization CLAHE to obtain the target culture dish image. Among them, CLAHE (Contrast Limited Adaptive Histogram Equalization) is an algorithm used in the field of image processing to improve the local contrast of an image. CLAHE divides the culture dish image into small sub-regions (also called blocks) and applies histogram equalization to each sub-region. CLAHE combines the advantages of local contrast enhancement and introduces a contrast limitation mechanism to prevent image noise or artifacts caused by excessive enhancement of the culture dish image.

[0053] Furthermore, if the image of the culture dish is a color image, this embodiment can convert the image of the culture dish into a grayscale image, thereby reducing the interference of color information, and process the grayscale image through a denoising technique (such as Gaussian filtering) to remove background noise and improve the image quality of the culture dish. This embodiment can perform image enhancement processing on the image of the culture dish by using CLAHE after the aforementioned image processing.

[0054] S11. Obtain drug information recorded on the drug-sensitive paper from the target culture dish image, and obtain culture experiment information.

[0055] Among them, drug information may refer to the information recorded on the drug-sensitive paper, such as the name of the drug-sensitive paper (the name of the drug contained in the drug-sensitive paper) and drug concentration. In the process of conducting drug sensitivity tests, drug information is often recorded on the drug-sensitive paper on the culture dish, which is convenient for determining the drug that causes the inhibition zone in the culture dish. Cultivation experiment information refers to relevant information about the test bacteria cultured in the culture dish, such as the type of test bacteria, the culture time of the culture dish, the type of culture medium for the cultured bacteria, and the conditions of the culture medium (temperature, humidity, time, etc.). The specific process of obtaining drug information and cultivation experiment information in this embodiment can be shown as steps one to three:

[0056] Step 1: Identify the image area of ​​the drug-sensitive paper in the target culture dish image using Tesseract OCR optical character recognition technology;

[0057] Step 2: Obtain the drug sensitivity paper name and drug concentration recorded in the drug sensitivity paper from the image area of ​​the drug sensitivity paper;

[0058] Step 3: Scan the barcode of the culture dish through barcode recognition technology to obtain the test bacteria type, culture medium type and culture time of the culture dish.

[0059] Among them, Tesseract OCR (Optical Character Recognition) is an open source OCR engine that can identify the text recorded on the drug-sensitive paper from the culture dish image and convert the text into a computer-editable text format, which may include image preprocessing, text area detection, text line segmentation, character segmentation, character recognition and post-processing. In this embodiment, optical character recognition is performed on the image area where the drug-sensitive paper in the enhanced culture dish image is located to obtain the drug-sensitive paper name and drug concentration recorded in the drug-sensitive paper. Of course, this embodiment can also record the drug information in the attached drug-sensitive paper during the drug sensitivity test, so as to facilitate direct acquisition of drug information.

[0060] In order to facilitate laboratory management and tracking of culture dishes, a barcode containing culture dish information may be affixed to the culture dish. Therefore, in this embodiment, the barcode of the culture dish may be scanned by barcode recognition technology to obtain the culture experiment information in the barcode.

[0061] S12. Input the drug information and culture experiment information into a preset decision tree model to obtain a target inhibition zone category output by the preset decision tree model, each inhibition zone category corresponds to an inhibition zone image range, and the nodes in the preset decision tree model are divided according to the influencing factors affecting the inhibition zone in the preset standard.

[0062] The inhibition circle category may be a size category of the inhibition circle, which may be divided into a large inhibition circle, a small inhibition circle, and no inhibition circle required. The size category of the inhibition circle of each drug-sensitive paper may be different. In the target culture dish image, the size category of the inhibition circle corresponds to an image range. Specifically, a large inhibition circle may correspond to an image range, and a small inhibition circle may correspond to an image range. No inhibition circle required means that the culture dish will not produce an inhibition circle under the influence of the drug-sensitive paper, and therefore, no inhibition circle needs to be divided.

[0063] The target inhibition zone category is the final inhibition zone classification result obtained by the preset decision tree model after dividing the tree node characteristics according to the drug information and culture experiment information. Among them, the target inhibition zone category refers to: the predicted inhibition zone of the drug-sensitive paper with fixed drug information in the culture medium with fixed culture experiment information, which indicates the range of inhibition zones that the drug-sensitive paper with fixed drug concentration should produce in the culture medium (fixed test bacteria species, fixed culture time, fixed culture medium type).

[0064] This embodiment can be implemented through CART (Classification and Regression Trees), which is a decision tree algorithm for classification and regression. CART can generate a tree-structured model, where each node can represent a feature, each branch (the path between nodes, representing the direction of data flow in the tree structure, and the different paths in the decision-making process) represents an output, and each leaf node (terminal node) represents a classification result. Therefore, the decision tree model can classify or predict data by segmenting features. CART uses Gini impurity to evaluate the quality of node splitting, thereby determining multiple nodes in the tree structure. The calculation formula for Gini impurity can be expressed as follows:

[0065]

[0066] in, It can represent Gini impurity; Can represent categories At the node The probability of It can represent the total number of categories. By calculating the Gini impurity of each node, the node with the minimum weighted Gini impurity is selected as the split point, and the best feature of the node is selected by calculating the information gain before and after the node split. The specific form of calculating the information gain before and after the node split can be shown as follows:

[0067]

[0068] Among them, entropy ( ) refers to an indicator to measure the purity of a node.

[0069] The nodes in the preset decision tree model in this embodiment are divided according to the standards of the expert system (such as EUCAST). EUCAST (European Committee on Antimicrobial Susceptibility Testing) is an organization dedicated to antimicrobial susceptibility testing. Its main task is to formulate and promote standards and guidelines for antimicrobial susceptibility testing to help clinical laboratories accurately assess the sensitivity of microorganisms to antimicrobial drugs. The expert system records multiple main influencing factors affecting the inhibition zone, and also records the standard size of the inhibition zone under the specific values ​​of multiple influencing factors. In order to improve the classification accuracy of the inhibition zone of the preset decision tree model, this embodiment selects the name of the drug-sensitive paper sheet, drug concentration, culture medium type, test bacteria type and culture time that affect the size of the inhibition zone recorded by the expert standard as the core node in the preset decision tree model. The preset decision tree model can be pruned through the core node to prevent the preset decision tree model in this embodiment from overfitting, thereby improving the generalization ability of the preset decision tree model in this embodiment.

[0070] Specifically, in this embodiment, the node types of the preset decision tree model include drug-susceptibility paper name, drug concentration, culture medium type, test bacteria type and culture time, including multiple drug-susceptibility paper name nodes, multiple drug concentration nodes, multiple culture medium type nodes, multiple test bacteria type nodes and multiple culture time nodes. The specific structure of the preset decision tree model can be: each drug-susceptibility paper name node includes multiple drug concentration nodes, each drug concentration node includes multiple culture medium type nodes, each culture medium type node includes multiple test bacteria type nodes, each test bacteria type node includes multiple culture time nodes, and each culture time node corresponds to an inhibition zone category. Figure 3 The structural schematic diagram of the preset decision tree model shown, the preset decision tree model includes drug-susceptibility paper name 1 and drug-susceptibility paper name 2, drug-susceptibility paper name 2 includes drug concentration 1 and drug concentration 2, drug concentration 1 includes culture medium type 1 and culture medium type 2, culture medium type 2 includes test bacteria type 1 and test bacteria type 2, test bacteria type 1 includes culture time 1 and culture time 2, culture time 1 corresponds to an inhibition zone category 1, culture time 2 corresponds to an inhibition zone category 2, and the drug-susceptibility paper name 1 can also have the same structure as the drug-susceptibility paper name 2. Figure 3 This is just an example to simply illustrate the structure of the preset decision tree model. In actual situations, the number of nodes can be greater.

[0071] S13. Input the target culture dish image and the target inhibition circle category into a preset multimodal segmentation model to obtain a target inhibition circle image range, wherein the target inhibition circle image range is: the preset multimodal segmentation model segments an inhibition circle image range from multiple inhibition circle image ranges in the target culture dish image, and the target inhibition circle image range corresponds to the target inhibition circle category.

[0072] The target inhibition zone image range refers to the inhibition zone image range of each drug-sensitive paper sheet marked by the preset multimodal segmentation model on the target culture dish image, and the target inhibition zone image range corresponds to the target inhibition zone category output by the preset decision tree model. The preset multimodal segmentation model can output a segmented image of the target culture dish image, which can include the boundary contour and segmentation status of the inhibition zone. Figure 4 In the culture dish image shown, the preset multimodal segmentation model marks the inhibition zones of multiple drug-sensitive paper strips with red lines in the culture dish image. The marked red circle of inhibition in the lower left corner is different from the range of the inhibition zone (transparent circle) in the image. This is because according to the standard of the expert system, the drug concentration of the drug-sensitive paper strip under this culture medium should have the size of the red circle, not the range of the transparent circle. The range of the transparent circle in the image may be that the inhibition zone is affected by other factors and will expand, such as other drug-sensitive paper strips. Therefore, the inhibition zone marked by this method can be unaffected by other factors. Compared with the existing image processing technology, the accuracy of the inhibition zone measurement of this method is higher.

[0073] In this embodiment, the target culture dish image can be segmented by a preset multimodal segmentation model. For each drug-sensitive paper sheet, the preset multimodal segmentation model can first segment out multiple inhibition circle ranges, which may include large inhibition circles and small inhibition circles, and then use the preset decision tree model to predict the type of inhibition circle, and select an inhibition circle range from the multiple segmented inhibition circle ranges and mark it as the inhibition circle range that the drug-sensitive paper sheet should have. For example, the target culture dish image has an image of drug-sensitive paper sheet A and an image of drug-sensitive paper sheet B. The preset multimodal segmentation model identifies the target culture dish image, and segments a large inhibition circle and a small inhibition circle for drug-sensitive paper sheet A, and a large inhibition circle and a small inhibition circle for drug-sensitive paper sheet B. The preset decision tree model predicts that the inhibition zone category of drug-sensitive paper piece A is a large inhibition zone based on the drug information and culture medium experimental information of drug-sensitive paper piece A, and predicts that the inhibition zone category of drug-sensitive paper piece B is a small inhibition zone based on the drug information and culture medium experimental information of drug-sensitive paper piece B. The preset multimodal segmentation model marks a large inhibition zone for drug-sensitive paper piece A and a small inhibition zone for drug-sensitive paper piece B in the target culture dish image based on the results of the preset decision tree model.

[0074] Of course, this embodiment can also first use the preset decision tree model to predict the type of inhibition zone, and then directly input the target culture dish image and the target inhibition zone category into the preset multimodal segmentation model to obtain the range of inhibition zone that each drug-sensitive paper sheet should have. Furthermore, this embodiment can adjust the parameters of the preset decision tree model and the preset multimodal segmentation model and optimize the model according to specific needs.

[0075] The multimodal segmentation model adopts multimodal segmentation technology. The technical principle of multimodal segmentation technology is mainly based on the fusion and analysis of different types of data, feature extraction and fusion, and accurate segmentation of the target object based on the fused features.

[0076] The preset multimodal segmentation model in this embodiment is trained based on the Multimodal U-Net structure of BERT (Bidirectional Encoder Representations from Transformers, a pre-trained language model based on the Transformer architecture) to achieve accurate segmentation of the inhibition zone. In the preset multimodal segmentation model of this embodiment, the multimodal features are processed by feature splicing, and the features from different modes are spliced ​​along the channel dimension, which can be expressed as:

[0077]

[0078] Among them, BERT is based on the Transformer architecture and is a deep learning model for processing sequence data. Its core idea is to understand the context through a bidirectional encoder, making BERT's understanding of text information more comprehensive. The encoder part of BERT uses the self-attention mechanism to process the relationship between each word and other words in the input sequence. Among them, the attention weight calculation formula in the self-attention mechanism can be expressed as follows:

[0079]

[0080] in, Can represent query matrix (Query); Can represent key matrix (Key); Can represent value matrix (Value); It can represent the dimension of the key. Since the Transformer architecture does not have the inherent characteristics of sequence order, BERT uses position encoding to inject position information into the sequence. The calculation formula of position encoding can be shown as follows:

[0081]

[0082]

[0083] in, Can indicate location; Can represent dimension index; The dimension that can represent the position encoding.

[0084] The Multimodal U-Net structure is an extension of the traditional U-Net architecture. It performs multimodal feature fusion after the basic U-Net encoder. It is a deep learning model that processes multimodal data (fusion of different types of images or data).

[0085] After designing the tree structure of the decision tree model, this embodiment needs to train the decision tree model to obtain the above-mentioned preset decision tree model. Similarly, this embodiment also needs to train the multimodal segmentation model to obtain the preset multimodal segmentation model of this embodiment. The specific training process of the two models can be as follows:

[0086] Collect image data of multiple culture dishes and obtain text data from the image data of multiple culture dishes. Collect multiple high-resolution images of drug-sensitive paper strips and their inhibition zones under different conditions, each of which contains different types of inhibition zones. Record various text data such as the name of the drug-sensitive paper strip, drug concentration, culture medium type, test bacteria type, and culture time in each high-resolution image. Expand the image data set through image enhancement technology (such as rotation, scaling, flipping, and brightness adjustment). Generate more text data samples to cover different drug information and culture experiment information, and expand the text data.

[0087] The collected images were annotated to obtain an annotated data set. The expert system's inhibition zone size standard was used to annotate the inhibition zones in the image using annotation tools, including the center position, boundary contour, and inhibition zone category (large inhibition zone, small inhibition zone, and no inhibition zone required) of the inhibition zone, generating accurate image data (inhibition zone contour data). The recorded text data was compared with the drug information identified by Tesseract OCR and the culture experiment information identified by barcode recognition technology to obtain accurate text data.

[0088] The training data set of the decision tree model is sorted and generated. The training data set of the decision tree model includes multiple groups of text data (including drug information and culture experiment information) and corresponding annotated inhibition circle categories. The decision tree model is trained using the training data set to obtain the preset decision tree model of this embodiment. The training data set of the decision tree model may include text data of drug-sensitive paper and culture dishes (drug-sensitive paper name, drug concentration, culture medium type, test bacteria type and culture time) and corresponding inhibition circle categories. The training data set is input into the decision tree model for training, and the classification accuracy of the decision tree model is verified by the verification data set. The parameters of the decision tree model are continuously optimized during the training and verification process, so that the inhibition circle category output by the decision tree model for each group of text data gradually approaches the annotated inhibition circle category corresponding to each group of text data.

[0089] After obtaining the preset decision tree model in this embodiment, a training data set of a multimodal segmentation model is generated, and the training data set of the multimodal segmentation model includes multiple groups of labeled image data and the classification results predicted by the preset decision tree model based on the text data in the image. The multimodal segmentation model is trained using the training data set of the multimodal segmentation model to obtain the preset multimodal segmentation model of this embodiment. The multimodal segmentation model combines image data and classification results for multimodal learning, uses a test set to evaluate the segmentation effect of the multimodal segmentation model, verifies the performance of the multimodal segmentation model, and continuously adjusts the parameters of the multimodal segmentation model during training and verification to improve the segmentation accuracy of the multimodal segmentation model for the inhibition zone.

[0090] S14, converting the target inhibition zone image range in the target culture dish image to obtain the physical size of the inhibition zone in the culture dish.

[0091] Among them, the physical size of the inhibition circle is the actual size of the inhibition circle in the culture dish, which may include the radius and diameter of the inhibition circle. In this embodiment, the pixel size in the target inhibition circle image range is converted into the actual physical size to obtain the diameter of the inhibition circle in the culture dish corresponding to the target inhibition circle image range. Among them, the pixel size can be the number of pixels occupied by the target inhibition circle image range in the target culture dish image. For example, an image of a culture dish has a resolution of 300 DPI (dots per inch) and one inch is 2.54 centimeters. The target inhibition circle image range occupies 1200 pixels (pixel size) in the diameter direction in the image of the culture dish. Then the physical size = (number of pixels / resolution) × 2.54 = (1200 / 300) × 2.54 = 10.16. Then, in the image of the culture dish, the diameter of the inhibition circle corresponding to the target inhibition circle image range is 10.16 centimeters.

[0092] Furthermore, this embodiment can also output a final inhibition zone size report. In the inhibition zone size report, the drug information obtained from the culture dish image, the culture experiment information of the culture dish obtained by barcode recognition, the inhibition zone category output by the preset decision tree model, the marking results output by the preset multimodal segmentation model and the physical size of the inhibition zone can be identified.

[0093] The embodiment of the present application provides a method for measuring the inhibition zone of a drug-sensitive paper sheet. The method first performs image enhancement processing on the culture dish image to obtain the target culture dish image to enhance the image features of the drug-sensitive paper sheet and the inhibition zone in the culture dish image. The culture experiment information of the culture dish is obtained and the drug information recorded on the drug-sensitive paper sheet is obtained from the target culture dish image. The culture experiment information and the drug information are input into a preset decision tree model to obtain the target inhibition zone category. Since the nodes in the preset decision tree model are divided according to the factors affecting the inhibition zone in the preset standard, the culture experiment information and the drug information can be better feature segmented to improve the accuracy of the inhibition zone classification. Since each inhibition zone category corresponds to an inhibition zone image range, the target inhibition zone image range corresponding to the target inhibition zone category can be marked in the target culture dish image through a preset multimodal segmentation model, so as to facilitate the conversion of the size of the inhibition zone in the actual culture dish according to the inhibition zone image range in the target culture dish image. This method first performs image enhancement processing on the culture dish image to improve the recognition accuracy of drug information and the accuracy of the preset multimodal segmentation model in segmenting the target culture dish image. In addition, this method predicts the inhibition zone category according to the preset decision tree model, and the nodes in the preset decision tree model are divided according to the factors affecting the inhibition zone in the preset standard, which improves the prediction accuracy and credibility of the inhibition zone category. The preset multimodal segmentation model selects the inhibition zone range that is more in line with the current drug-sensitive paper according to the inhibition zone category predicted by the preset decision tree model. Compared with directly analyzing the size of the inhibition zone in the culture dish image through image processing technology, this method provides a basis for the inhibition zone marking through the combination of AI neural network and expert system. In the case of multiple drug-sensitive paper sheets influencing each other, the inhibition zone range that the drug-sensitive paper sheet should have can still be determined, effectively improving the accuracy of the inhibition zone measurement. This method can also avoid the subjective errors and high cost problems in traditional measurement methods, and can also improve the efficiency of drug sensitivity tests, providing more accurate support for drug research and development and clinical applications, and helping to promote the development of antibiotic application and microbiology research.

[0094] A method for measuring the inhibition zone of a drug-sensitive paper sheet provided in an embodiment of the present application is introduced above. A system for executing the above-mentioned method for measuring the inhibition zone of a drug-sensitive paper sheet will be introduced below.

[0095] See also Figure 5 , Figure 5 This is a schematic diagram of the structure of a system for measuring the inhibition zone of a drug-sensitive paper sheet provided in an embodiment of the present application. Figure 5 As shown, the inhibition zone measurement system of the drug sensitive paper sheet includes:

[0096] An image acquisition unit 100 is used to acquire an image of a culture dish and perform image enhancement processing on the image of the culture dish to obtain an image of a target culture dish, wherein the culture dish has a drug-sensitive paper sheet;

[0097] The text acquisition unit 110 is used to acquire the drug information recorded on the drug-sensitive paper from the target culture dish image and acquire the culture experiment information;

[0098] The category prediction unit 120 is used to input the drug information and the culture experiment information into a preset decision tree model to obtain the target inhibition zone category output by the preset decision tree model, each inhibition zone category corresponds to an inhibition zone image range, and the nodes in the preset decision tree model are divided according to the influencing factors affecting the inhibition zone in the preset standard;

[0099] The image segmentation unit 130 is used to input the target culture dish image and the target inhibition circle category into a preset multimodal segmentation model to obtain a target inhibition circle image range, where the target inhibition circle image range is: an inhibition circle image range among multiple inhibition circle image ranges segmented by the preset multimodal segmentation model in the target culture dish image, and the target inhibition circle image range corresponds to the target inhibition circle category;

[0100] The size conversion unit 140 is used to convert the target inhibition zone image range in the target culture dish image to obtain the physical size of the inhibition zone in the culture dish.

[0101] In a possible implementation, the image acquisition unit 100 may be specifically configured as follows:

[0102] The image of the culture dish is enhanced by contrast-limited adaptive histogram equalization (CLAHE) to obtain the target culture dish image.

[0103] In a possible implementation, the drug information includes the name of the drug sensitivity paper and the drug concentration, and the culture experiment information includes the culture medium type, the test bacteria type and the culture time. The above text acquisition unit 110 can be specifically configured as follows:

[0104] The image area of ​​the drug-susceptibility paper in the target culture dish image is identified by Tesseract OCR optical character recognition technology; the drug-susceptibility paper name and drug concentration recorded in the drug-susceptibility paper are obtained from the image area of ​​the drug-susceptibility paper; the barcode of the culture dish is scanned by barcode recognition technology to obtain the test bacteria type, culture medium type and culture time of the culture dish.

[0105] In one possible implementation, the node types of the preset decision tree model include drug-susceptibility paper name, drug concentration, culture medium type, test bacteria type and culture time. The preset decision tree model includes multiple drug-susceptibility paper name nodes, each drug-susceptibility paper name node includes multiple drug concentration nodes, each drug concentration node includes multiple culture medium type nodes, each culture medium type node includes multiple test bacteria type nodes, each test bacteria type node includes multiple culture time nodes, and each culture time node corresponds to an inhibition zone category.

[0106] In a possible implementation, the physical size of the inhibition zone is the diameter of the inhibition zone in the culture dish, and the size conversion unit 140 may be specifically configured as follows:

[0107] The pixel size in the target inhibition zone image range is converted into a physical size to obtain the diameter of the inhibition zone in the culture dish corresponding to the target inhibition zone image range.

[0108] The present application also provides an electronic device in an embodiment. Figure 6 As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 6 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0109] like Figure 6 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 to a random access memory (RAM) 603. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0110] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a memory card, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0111] An embodiment of the present application also provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any method for measuring the inhibition zone of a drug-sensitive paper sheet provided in the embodiment of the present application.

[0112] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any method for measuring the inhibition zone of a drug-sensitive paper sheet provided in an embodiment of the present application.

[0113] It should also be noted that the system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the system embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0114] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0115] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0116] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0117] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0118] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0119] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for measuring the inhibition zone of a drug-sensitive paper sheet, characterized in that: The method for measuring the inhibition zone of the drug-sensitive paper sheet comprises: Acquiring an image of a culture dish and performing image enhancement processing on the image of the culture dish to obtain an image of a target culture dish, wherein the culture dish has a drug-sensitive paper sheet; Acquire the drug information recorded on the drug-sensitive paper from the target culture dish image, and acquire the culture experiment information; Input the drug information and the culture experiment information into a preset decision tree model, obtain the target inhibition zone category output by the preset decision tree model, each inhibition zone category corresponds to an inhibition zone image range, and the nodes in the preset decision tree model are divided according to the influencing factors affecting the inhibition zone in the preset standard; Inputting the target culture dish image and the target inhibition circle category into a preset multimodal segmentation model to obtain a target inhibition circle image range, wherein the target inhibition circle image range is: an inhibition circle image range among a plurality of inhibition circle image ranges segmented from the target culture dish image by the preset multimodal segmentation model, and the target inhibition circle image range corresponds to the target inhibition circle category; The target inhibition zone image range in the target culture dish image is converted to obtain the physical size of the inhibition zone in the culture dish.

2. The method for measuring the inhibition zone of the drug-sensitive paper according to claim 1, characterized in that: The step of acquiring an image of the culture dish and performing image enhancement processing on the image of the culture dish to obtain a target culture dish image comprises: The image of the culture dish is enhanced by contrast-limited adaptive histogram equalization (CLAHE) to obtain the target culture dish image.

3. The method for measuring the inhibition zone of the drug-sensitive paper according to claim 1, characterized in that: The drug information includes the name of the drug-sensitive paper and the drug concentration, the culture experiment information includes the culture medium type, the test bacteria type and the culture time, and the drug information recorded on the drug-sensitive paper is obtained from the target culture dish image to obtain the culture experiment information, including: Identify the image area of ​​the drug-sensitive paper in the target culture dish image by using Tesseract OCR optical character recognition technology; Acquire the drug-sensitive paper name and drug concentration recorded in the drug-sensitive paper from the image area of ​​the drug-sensitive paper; The barcode of the culture dish is scanned by barcode recognition technology to obtain the test bacteria type, culture medium type and culture time of the culture dish.

4. The method for measuring the inhibition zone of the drug-sensitive paper according to claim 3, characterized in that: The node types of the preset decision tree model include drug-susceptibility paper name, drug concentration, culture medium type, test bacteria type and culture time. The preset decision tree model includes multiple drug-susceptibility paper name nodes, each drug-susceptibility paper name node includes multiple drug concentration nodes, each drug concentration node includes multiple culture medium type nodes, each culture medium type node includes multiple test bacteria type nodes, each test bacteria type node includes multiple culture time nodes, and each culture time node corresponds to an inhibition zone category.

5. The method for measuring the inhibition zone of the drug-sensitive paper according to claim 1, characterized in that: The physical size of the inhibition zone is the diameter of the inhibition zone in the culture dish, and converting the target inhibition zone image range in the target culture dish image to obtain the physical size of the inhibition zone in the culture dish includes: The pixel size in the target inhibition zone image range is converted into a physical size to obtain the diameter of the inhibition zone in the culture dish corresponding to the target inhibition zone image range.

6. A system for measuring the inhibition zone of a drug-sensitive paper sheet, characterized in that: The inhibition zone measurement system of the drug-sensitive paper sheet comprises: An image acquisition unit, used for acquiring an image of a culture dish and performing image enhancement processing on the image of the culture dish to obtain an image of a target culture dish, wherein the culture dish has a drug-sensitive paper sheet; A text acquisition unit, used to acquire the drug information recorded on the drug-sensitive paper from the target culture dish image, and acquire the culture experiment information; A category prediction unit, used for inputting the drug information and the culture experiment information into a preset decision tree model, obtaining a target inhibition zone category output by the preset decision tree model, each inhibition zone category corresponds to an inhibition zone image range, and the nodes in the preset decision tree model are divided according to the influencing factors affecting the inhibition zone in the preset standard; An image segmentation unit is used to input the target culture dish image and the target inhibition circle category into a preset multimodal segmentation model to obtain a target inhibition circle image range, wherein the target inhibition circle image range is: an inhibition circle image range among multiple inhibition circle image ranges segmented from the target culture dish image by the preset multimodal segmentation model, and the target inhibition circle image range corresponds to the target inhibition circle category; The size conversion unit is used to convert the target inhibition zone image range in the target culture dish image to obtain the physical size of the inhibition zone in the culture dish.

7. The inhibition zone measurement system of the drug-sensitive paper sheet according to claim 6, characterized in that: The image acquisition unit is specifically configured as follows: The image of the culture dish is enhanced by contrast-limited adaptive histogram equalization (CLAHE) to obtain the target culture dish image.

8. The inhibition zone measurement system of the drug-sensitive paper sheet according to claim 6, characterized in that: The drug information includes the name of the drug-sensitive paper and the drug concentration, the culture experiment information includes the culture medium type, the test bacteria type and the culture time, and the text acquisition unit is specifically configured as follows: The image area of ​​the drug-sensitive paper in the target culture dish image is identified by Tesseract OCR optical character recognition technology; the drug-sensitive paper name and drug concentration recorded in the drug-sensitive paper are obtained from the image area of ​​the drug-sensitive paper; the barcode of the culture dish is scanned by barcode recognition technology to obtain the test bacteria type, culture medium type and culture time of the culture dish.

9. An electronic device, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the method for measuring the inhibition zone of the drug-sensitive paper sheet as described in any one of claims 1 to 5.

10. A computer program product, characterized in that The method comprises computer-readable instructions, and when the computer-readable instructions are executed on an electronic device, the electronic device implements the method for measuring the inhibition zone of a drug-sensitive paper sheet as claimed in any one of claims 1 to 5.

Images