Automatic tuberculosis skin test result identification method and system based on artificial intelligence

Abstract

The embodiment of the invention provides a tuberculosis skin test result automatic identification method and system based on artificial intelligence, and the method comprises the steps: obtaining to-be-processed image data, the to-be-processed image being an image obtained based on a binocular camera after a tuberculosis skin test; generating three-dimensional voxel data based on the to-be-processed image data; the three-dimensional voxel data is input into a pre-trained three-dimensional convolutional neural network model, a pixel classification result in the three-dimensional voxel data is obtained, and the pre-trained three-dimensional convolutional neural network model is a model based on a Mask R-CNN architecture; and based on the pixel classification result, determining the thickness of the target data in the to-be-processed image data and / or the volume of the target data in the to-be-processed image data.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and in particular to an automatic recognition method and system for tuberculosis skin test results based on artificial intelligence. Background Art

[0002] Tuberculosis is a chronic infectious disease caused by infection with Mycobacterium tuberculosis. Tuberculosis may invade various organs of the human body, but mainly invades the lungs, which is called pulmonary tuberculosis. The screening for pulmonary tuberculosis usually uses the tuberculin skin test (TST) or the Mantoux test, commonly known as the tuberculosis skin test.

[0003] In the prior art, the tuberculosis skin test method usually relies on manual measurement of the thickness and volume of edema in the skin reaction area, which is usually measured within 48 to 72 hours after the tuberculosis skin test is completed. First, observe the reaction at the injection site, focusing on whether there is a bulge (edema). Use a special thickness gauge or ruler to accurately measure the thickness of the widest part of the edema, which is usually recorded in millimeters. For a preliminary estimate of the volume, the diameter of the edema and the range of the surrounding redness and swelling can be measured, combined with geometric formulas for calculation (for example, the volume is estimated by approximating the edema shape as an ellipsoid). Ultimately, by combining the thickness of the edema with the subject's risk factors, the doctor can assess whether the individual may be infected with tuberculosis.

[0004] The existing manual measurement method of skin test, because the thickness of the skin reaction is usually evaluated by vision and manual touch, different operators may be affected by subjective factors during measurement, resulting in inconsistent results. For example, the amount of pressure applied, the angle of observation, or the difference in the use of measuring tools may cause measurement errors. In addition, local skin conditions (such as itching, inflammation, or allergic reactions) may also affect the formation of edema, causing the actual measurement results to deviate from the true infection status. Therefore, the existing manual measurement method of skin test is time-consuming, cannot be quickly screened and judged, the accuracy of the results is poor, and the reliance on manual recording of information can easily cause problems such as data omissions, which is not conducive to the prevention and treatment of tuberculosis.

[0005] Based on this, the embodiments of this specification provide an automatic identification method and system for tuberculosis skin test results based on artificial intelligence. Summary of the invention

[0006] In order to solve the problems in the related technology, the embodiments of the present disclosure provide an automatic identification method and system for tuberculosis skin test results based on artificial intelligence, which are used to solve the following technical problems: the existing manual measurement skin test method is time-consuming, cannot quickly perform screening and judgment, the result accuracy is poor, and the recording of information relies on manual labor, which is prone to data omissions and other problems, which is not conducive to the prevention and treatment of tuberculosis.

[0007] On the one hand, an embodiment of the present disclosure provides an automatic recognition method for tuberculosis skin test results based on artificial intelligence, and the automatic recognition system includes:

[0008] Acquiring image data to be processed, wherein the image to be processed is an image after a tuberculosis skin test acquired based on a binocular camera;

[0009] generating three-dimensional voxel data based on the image data to be processed;

[0010] Inputting the three-dimensional voxel data into a pre-trained three-dimensional convolutional neural network model to obtain a pixel classification result in the three-dimensional voxel data, wherein the pre-trained three-dimensional convolutional neural network model is a model based on the Mask R-CNN architecture;

[0011] Based on the pixel classification result, the thickness of the target data in the image data to be processed and / or the volume of the target data in the image data to be processed are determined.

[0012] The present disclosure also provides an automatic recognition system for tuberculosis skin test results based on artificial intelligence, the automatic recognition system comprising:

[0013] An acquisition module, which acquires image data to be processed, wherein the image to be processed is an image after a tuberculosis skin test acquired based on a binocular camera;

[0014] A three-dimensional voxel generation module, which generates three-dimensional voxel data based on the image data to be processed;

[0015] A classification module, inputting the three-dimensional voxel data into a pre-trained three-dimensional convolutional neural network model to obtain a pixel classification result in the three-dimensional voxel data, wherein the pre-trained three-dimensional convolutional neural network model is a model based on the MaskR-CNN architecture;

[0016] A calculation module determines the thickness of the target data in the image data to be processed and / or the volume of the target data in the image data to be processed based on the pixel classification result.

[0017] According to the technical solution of the embodiment of the present disclosure, by acquiring image data to be processed, the image to be processed is an image after a tuberculosis skin test acquired based on a binocular camera; based on the image data to be processed, three-dimensional voxel data is generated; the three-dimensional voxel data is input into a pre-trained three-dimensional convolutional neural network model to obtain a pixel classification result in the three-dimensional voxel data, and the pre-trained three-dimensional convolutional neural network model is a model based on the Mask R-CNN architecture; based on the pixel classification result, the thickness of the target data in the image data to be processed and / or the volume of the target data in the image data to be processed is determined, thereby realizing automatic recognition of tuberculosis skin test results, improving the accuracy and efficiency of skin tests, and being able to be used on a large scale as an auxiliary means for clinical diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Other features, objectives and advantages of the present disclosure will become more apparent through the following detailed description of non-limiting embodiments in conjunction with the accompanying drawings. In the accompanying drawings:

[0019] Figure 1 A schematic diagram of the system architecture of an automatic identification method for tuberculosis skin test results based on artificial intelligence provided in an embodiment of this specification;

[0020] Figure 2 A schematic flow chart of an automatic identification method for tuberculosis skin test results based on artificial intelligence provided in an embodiment of this specification;

[0021] Figure 3 A schematic diagram of an automatic recognition system for tuberculosis skin test results based on artificial intelligence provided in an embodiment of this specification;

[0022] Figure 4 The structure diagram of a computer system suitable for implementing an artificial intelligence-based automatic identification method and system for tuberculosis skin test results provided in an embodiment of this specification is schematically shown. DETAILED DESCRIPTION

[0023] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can more easily implement them. In addition, for the sake of clarity, parts not related to the description of the exemplary embodiments are omitted in the accompanying drawings.

[0024] In the present disclosure, it should be understood that terms such as "include" or "have" are intended to indicate the presence of features, numbers, steps, behaviors, components, parts, or a combination thereof disclosed in the present specification, and are not intended to exclude the possibility that one or more other features, numbers, steps, behaviors, components, parts, or a combination thereof exist or are added.

[0025] It should also be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0026] It should be noted that the acquisition or display of data in the present disclosure is authorized, confirmed, or actively selected by the user.

[0027] The technical solutions provided by various embodiments of the present disclosure are described in detail below in conjunction with the accompanying drawings.

[0028] Figure 1 This is a schematic diagram of the system architecture of an automatic identification method for tuberculosis skin test results based on artificial intelligence provided in the embodiments of this specification. Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0029] The terminal devices 101, 102, 103 interact with the server 105 via the network 104 to receive or send messages, etc. Various client applications may be installed on the terminal devices 101, 102, 103. For example, a dedicated application for automatic recognition of tuberculosis skin test results based on artificial intelligence.

[0030] The terminal devices 101, 102, 103 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, they may be various dedicated or general electronic devices, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, etc. When the terminal devices 101, 102, 103 are software, they may be installed in the electronic devices listed above. They may be implemented as multiple software or software modules (e.g., multiple software or software modules for providing distributed services), or they may be implemented as a single software or software module.

[0031] The server 105 may be a server that provides various services, such as a backend server that provides services for client applications installed on the terminal devices 101, 102, and 103. For example, the server may train and run a pre-trained three-dimensional convolutional neural network model to display the automatic recognition results of the tuberculosis skin test results based on artificial intelligence on the terminal devices 101, 102, and 103.

[0032] The server 105 may be hardware or software. When the server 105 is hardware, it may be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server 105 is software, it may be implemented as multiple software or software modules (e.g., multiple software or software modules for providing distributed services), or as a single software or software module.

[0033] The method for automatically identifying tuberculosis skin test results based on artificial intelligence provided in the embodiment of the present disclosure can be executed by the server 105, for example, or by the terminal devices 101, 102, and 103. Alternatively, the method for automatically identifying tuberculosis skin test results based on artificial intelligence in the embodiment of the present disclosure can be partially executed by the terminal devices 101, 102, and 103, and the other part can be executed by the server 105.

[0034] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0035] Figure 2 The following is a flowchart of an automatic identification method for tuberculosis skin test results based on artificial intelligence provided in the embodiments of this specification. From the program perspective, the execution subject of the process can be a program installed on an application server or application terminal. It can be understood that the method can be executed by any device, equipment, platform, or device cluster with computing and processing capabilities. Figure 2 As shown, the automatic identification method includes:

[0036] Step S201: obtaining image data to be processed, wherein the image to be processed is an image after a tuberculosis skin test obtained based on a binocular camera.

[0037] In the embodiments of this specification, the image data to be processed are images of the target area taken from different angles by the binocular camera, that is, images of the tuberculosis skin test area taken from different angles by the binocular camera. The image data to be processed are multiple image data, taken from different angles, and the shooting angle is not specifically limited. Generally, the image data to be processed are at least 4 images.

[0038] Step S203: Generate three-dimensional voxel data based on the image data to be processed.

[0039] Since the dual-sided camera collects data, there may be lens distortion, so the image data to be processed needs to be corrected and aligned so that it can be accurately matched.

[0040] In the embodiment of this specification, generating three-dimensional voxel data based on the image data to be processed specifically includes:

[0041] Correcting the image data to be processed to obtain corrected image data;

[0042] generating a depth map based on the corrected image data;

[0043] Based on the depth map, the three-dimensional voxel data is generated by a triangular mesh.

[0044] In the embodiment of this specification, the correcting the image data to be processed to obtain the corrected image data specifically includes:

[0045] Based on the focal length, principal point position and lens distortion coefficient of a single camera in the binocular camera and the rotation matrix and translation vector of the binocular camera, the image data to be processed is corrected to obtain the corrected image data;

[0046] The generating a depth map based on the corrected image data specifically includes:

[0047] Aligning the corrected image data to obtain aligned image data;

[0048] Using a block matching BM algorithm to obtain a disparity map of the aligned image data;

[0049] Based on the disparity point determined by the disparity map of the aligned image data, the distance between the binocular cameras is used as the baseline length. According to the disparity map, the baseline length and the focal length of the binocular cameras, the depth of each pixel in the aligned image data is determined to form the depth map.

[0050] In the embodiments of this specification, the binocular camera is calibrated to correct the lens distortion and ensure that the acquired image has an accurate geometric structure.

[0051] In the embodiments of the present specification, the lens distortion coefficient is achieved through calibration. Specifically, after the calibration image is collected, the feature points on the calibration plate are marked in the collected calibration image. The corner points of the chessboard can be used as feature points. The known calibration plate geometry and the detected feature point coordinates are used to solve the camera's intrinsic parameter matrix (including focal length, principal point position, etc.), extrinsic parameter matrix (including: rotation matrix and translation vector) and lens distortion coefficient through mathematical optimization methods (such as least squares method).

[0052] In the embodiment of this specification, a block matching BM algorithm is used to obtain a disparity map of the aligned image data, specifically including:

[0053] Taking the center point of the left image data in the aligned image data as the center pixel, and taking the center pixel as the center point, selecting 5*5 pixels as the selected block;

[0054] Using the maximum inspection range in the horizontal direction of the right image data in the aligned image data as a fixed search window;

[0055] In the fixed search window, searching for a matching block that matches the selected block, and calculating the similarity between the matching block and the selected block;

[0056] Determine the disparity value of the current pixel based on the similarity between the matching block and the selected block;

[0057] Repeat the above steps until all pixels in the left image are traversed to obtain a disparity map of the aligned image data.

[0058] In the embodiment of the present specification, the similarity between the matching block and the selected block can be calculated by using the sum of absolute differences (SAD), the sum of mean differences (SSD), or the normalized cross correlation (NCC), preferably the NCC.

[0059] Based on the similarity between the matching block and the selected block, the disparity value of the current pixel is determined. Specifically, the disparity value that minimizes the similarity metric (for SAD and SSD) or maximizes (for NCC) is selected as the disparity value of the current pixel.

[0060] In the embodiment of this specification, the calculation formula of depth is:

[0061]

[0062] in,

[0063] Z represents depth;

[0064] B represents the baseline length;

[0065] f represents the focal length of the binocular camera, expressed in pixels;

[0066] d is the disparity, expressed in pixels.

[0067] d is obtained from the disparity map.

[0068] Step S205: inputting the three-dimensional voxel data into a pre-trained three-dimensional convolutional neural network model to obtain pixel classification results in the three-dimensional voxel data, wherein the pre-trained three-dimensional convolutional neural network model is a model based on the MaskR-CNN architecture.

[0069] In the embodiment of the present specification, the pre-trained three-dimensional convolutional neural network model includes: a backbone network, a region proposal network, a region of interest alignment module, a bounding box regression module and a classification head module;

[0070] in,

[0071] The backbone network adopts ResNet combined with a feature pyramid network, and the backbone network is used to obtain a feature map corresponding to the three-dimensional voxel data;

[0072] The region proposal network adopts an RPN convolutional layer, and the region proposal module is used to determine the candidate region in the feature map;

[0073] The region of interest alignment module is a bilinear pooling network, and the region of interest is used to extract effective features from the candidate region;

[0074] The bounding box regression module includes a first fully connected layer and a second fully connected layer, and the bounding box regression module is used to regress the target bounding box;

[0075] The classification head module shares the first fully connected layer and the second fully connected layer with the bounding box regression model, and the classification head module further includes: a softmax layer, and the classification head module is used for pixel classification.

[0076] In this embodiment of the present specification, the number of neurons in the first fully connected layer is 2048, and the number of neurons in the second fully connected layer is 2048.

[0077] In the embodiment of this specification, the RPN convolution layer includes: a sliding window convolution layer, two 1*1 convolution layers, and the two 1*1 convolution layers are respectively input to the cls layer and the reg layer, and the proposal layer;

[0078] The sliding window convolution layer is 3*3, the step size is 1, and the padding is 1;

[0079] The cls layer is a prediction category branch, and the cls layer is a 1*1 convolution kernel;

[0080] The reg layer predicts the regression parameter branch, and the reg layer is a 1*convolution kernel;

[0081] The proposal layer is used for correction to obtain the candidate region in the feature map.

[0082] In the embodiment of this specification, the loss function used by the pre-trained three-dimensional convolutional neural network model is:

[0083]

[0084] in,

[0085] is the classification loss;

[0086] α is the weight of the classification loss;

[0087] L box is the bounding box loss;

[0088] β is the weight of the bounding box loss;

[0089] L mask is the binary mask loss function;

[0090] γ is the weight of the binary mask loss function;

[0091] α+β+γ=1.

[0092] In the embodiment of the present specification, the optimizer used by the pre-trained three-dimensional convolutional neural network model is the Adam optimizer, the hyperparameter β1 of the optimizer is 0.9, and the hyperparameter β2 of the optimizer is 0.999.

[0093] In an embodiment of the present specification, during the training process of the pre-trained three-dimensional convolutional neural network model, the iteration termination condition is: the loss function adopted by the pre-trained three-dimensional convolutional neural network model, or the performance of the pre-trained three-dimensional convolutional neural network model on the validation set has not improved in 10 epochs.

[0094] In the embodiment of the present specification, the pixel classification result includes: a target area and a non-target area. In the embodiment of the present specification, the target area is an area belonging to tuberculosis skin test edema, and the non-target area includes a background area.

[0095] Step S207: Based on the pixel classification result, determine the thickness of the target data in the image data to be processed and / or the volume of the target data in the image data to be processed.

[0096] In an embodiment of the present specification, the thickness of the target data and / or the volume of the target data in the image data to be processed is determined by determining the target area through the pixel classification result, and then calculating the thickness of the target data and / or the volume of the target data in the image data to be processed based on the depth information of the target area and the pixels in the target area.

[0097] According to the thickness of the target data in the image data to be processed and / or the volume of the target data in the image data to be processed, the tuberculosis skin test result is automatically identified as an auxiliary means for clinical diagnosis.

[0098] The embodiments of this specification provide an automatic identification method for tuberculosis skin test results based on artificial intelligence, which can realize automatic identification of tuberculosis skin test results, improve the accuracy and efficiency of skin tests, can be used on a large scale, and can be used as an auxiliary means of clinical diagnosis.

[0099] The above content describes in detail an automatic identification method for tuberculosis skin test results based on artificial intelligence. Correspondingly, this specification also provides an automatic identification system for tuberculosis skin test results based on artificial intelligence, such as Figure 3 shown. Figure 3 A schematic diagram of an automatic recognition system for tuberculosis skin test results based on artificial intelligence provided in an embodiment of this specification, the automatic recognition system comprising:

[0100] An acquisition module 301 acquires image data to be processed, wherein the image to be processed is an image after a tuberculosis skin test acquired based on a binocular camera;

[0101] A three-dimensional voxel generation module 303 generates three-dimensional voxel data based on the image data to be processed;

[0102] A classification module 305, inputting the three-dimensional voxel data into a pre-trained three-dimensional convolutional neural network model to obtain a pixel classification result in the three-dimensional voxel data, wherein the pre-trained three-dimensional convolutional neural network model is a model based on a MaskR-CNN architecture;

[0103] The calculation module 307 determines the thickness of the target data in the image data to be processed and / or the volume of the target data in the image data to be processed based on the pixel classification result.

[0104] The artificial intelligence-based automatic recognition method and system for tuberculosis skin test results provided in the embodiments of this specification, when used clinically, fix the arm of the person to be tested by an arm fixing device, and then collect the image to be processed by a binocular camera. The image to be processed is transmitted to the automatic recognition system for automatic recognition of the tuberculosis skin test results, and the automatic recognition results are presented to the diagnostician in the form of an analysis report. Of course, the automatic recognition results can also be modified.

[0105] Figure 4 The structure diagram of a computer system suitable for implementing an artificial intelligence-based automatic identification method and system for tuberculosis skin test results provided in an embodiment of this specification is schematically shown.

[0106] like Figure 4 As shown, the computer system 400 includes a processing unit 401, which can perform various processes in the above-mentioned embodiments according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage part 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the system 400 are also stored. The processing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0107] The following components are connected to the I / O interface 405: an input part 406 including a keyboard, a mouse, etc.; an output part 407 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage part 408 including a hard disk, etc.; and a communication part 409 including a network interface card such as a LAN card, a modem, etc. The communication part 409 performs communication processing via a network such as the Internet. The drive 410 is also connected to the I / O interface 1105 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as needed, so that the computer program read therefrom is installed into the storage part 408 as needed. Among them, the processing unit 401 can be implemented as a processing unit such as a CPU, a GPU, a TPU, an FPGA, an NPU, etc.

[0108] In particular, according to an embodiment of the present disclosure, the method described above can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program tangibly contained on a machine-readable medium, and the computer program includes a program code for executing the above method. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 409, and / or installed from a removable medium 411.

[0109] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0110] The units or modules involved in the embodiments described in the present disclosure may be implemented by software or programmable hardware. The units or modules described may also be set in a processor, and the names of these units or modules do not constitute limitations on the units or modules themselves in some cases.

[0111] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0112] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0113] The apparatus, electronic device, non-volatile computer storage medium and method provided in the embodiments of this specification correspond to each other, and therefore, the apparatus, electronic device and non-volatile computer storage medium also have similar beneficial technical effects as the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding apparatus, electronic device and non-volatile computer storage medium will not be repeated here.

[0114] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0115] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0116] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0117] For the convenience of description, the above devices are described in terms of functions and are divided into various units. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0118] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may be in the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of this specification may be in the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0119] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0120] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0121] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0122] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0123] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0124] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0125] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0126] The specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media including storage devices.

[0127] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0128] The above is only an embodiment of this specification and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. An automatic identification method for tuberculosis skin test results based on artificial intelligence, characterized in that: The automatic identification method comprises: Acquiring image data to be processed, wherein the image to be processed is an image after a tuberculosis skin test acquired based on a binocular camera; generating three-dimensional voxel data based on the image data to be processed; Inputting the three-dimensional voxel data into a pre-trained three-dimensional convolutional neural network model to obtain a pixel classification result in the three-dimensional voxel data, wherein the pre-trained three-dimensional convolutional neural network model is a model based on the Mask R-CNN architecture; Based on the pixel classification result, the thickness of the target data in the image data to be processed and / or the volume of the target data in the image data to be processed are determined.

2. The automatic identification method according to claim 1, characterized in that: The generating three-dimensional voxel data based on the image data to be processed specifically includes: Correcting the image data to be processed to obtain corrected image data; generating a depth map based on the corrected image data; Based on the depth map, the three-dimensional voxel data is generated by a triangular mesh.

3. The automatic identification method according to claim 2, characterized in that: Correcting the image data to be processed to obtain corrected image data specifically includes: Based on the focal length, principal point position and lens distortion coefficient of a single camera in the binocular camera and the rotation matrix and translation vector of the binocular camera, the image data to be processed is corrected to obtain the corrected image data; The generating a depth map based on the corrected image data specifically includes: Aligning the corrected image data to obtain aligned image data; Using a block matching BM algorithm to obtain a disparity map of the aligned image data; Based on the disparity point determined by the disparity map of the aligned image data, the distance between the binocular cameras is used as the baseline length. According to the disparity map, the baseline length and the focal length of the binocular cameras, the depth of each pixel in the aligned image data is determined to form the depth map.

4. The automatic identification method according to claim 1, characterized in that: The pre-trained 3D convolutional neural network model includes: a backbone network, a region proposal network, a region of interest alignment module, a bounding box regression module and a classification head module; in, The backbone network adopts ResNet combined with a feature pyramid network, and the backbone network is used to obtain a feature map corresponding to the three-dimensional voxel data; The region proposal network adopts an RPN convolutional layer, and the region proposal module is used to determine the candidate region in the feature map; The region of interest alignment module is a bilinear pooling network, and the region of interest is used to extract effective features from the candidate region; The bounding box regression module includes a first fully connected layer and a second fully connected layer, and the bounding box regression module is used to regress the target bounding box; The classification head module shares the first fully connected layer and the second fully connected layer with the bounding box regression model, and the classification head module further includes: a softmax layer, and the classification head module is used for pixel classification.

5. The automatic identification method according to claim 4, characterized in that: The number of neurons in the first fully connected layer is 2048, and the number of neurons in the second fully connected layer is 2048.

6. The automatic identification method according to claim 4, characterized in that: The RPN convolution layer includes: a sliding window convolution layer, two 1*1 convolution layers, and the two 1*1 convolution layers are respectively input to the cls layer and the reg layer, and the proposal layer; The sliding window convolution layer is 3*3, the step size is 1, and the padding is 1; The cls layer is a prediction category branch, and the cls layer is a 1*1 convolution kernel; The reg layer predicts the regression parameter branch, and the reg layer is a 1*convolution kernel; The proposal layer is used for correction to obtain the candidate region in the feature map.

7. The automatic identification method according to claim 1, characterized in that: The loss function used by the pre-trained three-dimensional convolutional neural network model is: in, is the classification loss; α is the weight of the classification loss; L box is the bounding box loss; β is the weight of the bounding box loss; L mask is the binary mask loss function; γ is the weight of the binary mask loss function; α+β+γ=1.

8. The automatic identification method according to claim 1, characterized in that: The optimizer used in the pre-trained three-dimensional convolutional neural network model is the Adam optimizer, the hyperparameter β1 of the optimizer is 0.9, and the hyperparameter β2 of the optimizer is 0.

999.

9. The automatic identification method according to claim 1, characterized in that: During the training process of the pre-trained three-dimensional convolutional neural network model, the iteration termination condition is: the loss function adopted by the pre-trained three-dimensional convolutional neural network model, or the performance of the pre-trained three-dimensional convolutional neural network model on the validation set has not improved in 10 epochs.

10. An automatic recognition system for tuberculosis skin test results based on artificial intelligence, characterized in that: The automatic identification system comprises: An acquisition module, which acquires image data to be processed, wherein the image to be processed is an image after a tuberculosis skin test acquired based on a binocular camera; A three-dimensional voxel generation module, which generates three-dimensional voxel data based on the image data to be processed; A classification module, inputting the three-dimensional voxel data into a pre-trained three-dimensional convolutional neural network model to obtain a pixel classification result in the three-dimensional voxel data, wherein the pre-trained three-dimensional convolutional neural network model is a model based on the Mask R-CNN architecture; A calculation module determines the thickness of the target data in the image data to be processed and / or the volume of the target data in the image data to be processed based on the pixel classification result.

Images