Multi-label multi-instance image detection method, device, equipment and storage medium

The pathological characteristics of biopsy images are detected through multi-label and multi-example models, which solves the problems of low diagnostic efficiency and insufficient accuracy in the prior art, and achieves efficient and accurate pathological image analysis.

CN111462048BActive Publication Date: 2024-11-22PING AN TECH (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010156699.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-03-09
Publication Date
2024-11-22
Estimated Expiration
2040-03-09

AI Technical Summary

Technical Problem

In clinical diagnosis, doctors need to perform cumbersome pathological characteristics labeling and analysis of large numbers of biopsy images, resulting in low diagnostic efficiency and insufficient accuracy, especially since each pathological characteristic requires a separate model to be labeled, increasing the number of models and the labeling process complexity.

Method used

Multi-label and multi-example model is used for pathological image detection, and offline training is performed by obtaining confirmed biopsy images and their corresponding pathological feature descriptions to generate a preset multi-example and multi-label model for texture feature extraction and Structured expression, call training models for identification and annotation, and generate multiple target pathological tags.

Benefits of technology

It reduces the duration of pathological image analysis, improves the analysis efficiency and accuracy of results, simplifies the process of marking pathological characteristics, and reduces the requirements for doctors' professional knowledge and judgment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111462048B_ABST
    Figure CN111462048B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of artificial intelligence, and discloses a multi-label multi-instance image detection method, device, equipment and storage medium, which are used to reduce the analysis time of pathological images and improve the accuracy of analysis results. The method of the present invention includes: obtaining the pathological area of ​​the confirmed biopsy image and the pathological feature description corresponding to the pathological area; training the multi-label multi-instance model offline according to the pathological area and the pathological feature description corresponding to the pathological area to obtain a preset multi-instance multi-label model; obtaining the original biopsy image, which is used to indicate the pathological characteristics of human tissue; extracting texture features from the original biopsy image, and expressing the extracted texture features in a structured feature to generate a target graph structure; calling a preset training model to identify the target graph structure to obtain a target feature vector group; labeling the target feature vector group through a preset multi-instance multi-label model to obtain multiple target pathology labels.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of region extraction, and in particular to a multi-label and multi-instance image detection method, device, equipment and storage medium. Background Art

[0002] Biopsy images are obtained by slicing the tissue of the patient's lesion and magnifying it under a microscope. Biopsy images directly reflect the lesions inside the tissue and are an important basis for doctors to diagnose diseases, and are even the final basis for diagnosing certain serious diseases. For example, in the diagnosis of cancer, a preliminary diagnosis can be made by observing the lesions with imaging methods (X-rays, CT, MRI, etc.), including observing whether there are shadows, plaques, and dilation of blood vessels, etc., combined with blood tests. However, to make a final diagnosis, living tissue from the lesion must be extracted, observed under a microscope, and biochemical tests must be performed. This process is called a biopsy.

[0003] Taking dermatology as an example, there are more than 3,000 common skin diseases, but only a few hundred can be diagnosed by observing the skin surface. Most of the other types require the removal of tissue from the lesion for observation under a microscope. On the other hand, the types and number of pathological characteristics of biopsy images are relatively stable. Each medical department has its own pathological characteristics that need to be paid attention to, most of which are between 100 and 200. With the increasing number of patients and higher requirements for accurate diagnosis of diseases, the demand for pathological examinations, especially microscopic image analysis, has increased exponentially. To accurately judge these 100 to 200 different pathological characteristics, doctors are required to have extremely high professional knowledge, judgment, and concentration when reading the film.

[0004] In clinical diagnosis, doctors will analyze the entire pathological image to obtain a diagnosis report. In most cases, the diagnosis report will not clearly indicate which area has which pathological characteristics. In order to correspond the characteristics in the diagnosis report to different areas in the image, it is necessary to call the corresponding model for marking. N characteristics require N models. The number of models used is large and the marking process is cumbersome. Summary of the invention

[0005] The present invention provides a multi-label multi-instance image detection method, device, equipment and storage medium, which are used to perform pathological image detection in real time through a multi-label multi-instance model, generate pathological characteristics of multiple biopsy images, reduce the analysis time of pathological images, improve analysis efficiency, and improve the accuracy of analysis results.

[0006] A first aspect of an embodiment of the present invention provides a multi-label multi-instance image detection method, comprising: obtaining a pathological area of ​​a confirmed biopsy image and a pathological feature description corresponding to the pathological area; offline training a multi-label multi-instance model based on the pathological area and the pathological feature description corresponding to the pathological area to obtain a preset multi-instance multi-label model; obtaining an original biopsy image, wherein the original biopsy image is used to indicate the pathological characteristics of human tissue; extracting texture features from the original biopsy image, and expressing the extracted texture features as structured features to generate a target graph structure; calling a preset training model to identify the target graph structure to obtain a target feature vector group; and labeling the target feature vector group through the preset multi-instance multi-label model to obtain multiple target pathology labels.

[0007] Optionally, in a first implementation of the first aspect of the embodiment of the present invention, the multi-label multi-instance model is offline trained according to the pathological area and the pathological feature description corresponding to the pathological area to obtain a preset multi-instance multi-label model, including:

[0008] According to the pathological area, texture and structural features are extracted to generate a feature vector group; the feature vector group is used as the input of the offline training multi-label multi-instance model, and the pathological feature description corresponding to the pathological area is used as the output of the offline training multi-label multi-instance model, and the multi-label multi-instance model F is obtained by training. D (f 1 (θ 1 x 1 ),f 1 (θ 2 x 2 ),f 1 (θ 3 x 3 )…f 1 (θ ni x ni )|θ 1 ,θ 2 ,θ 3 ...θ ni );Adjust θ 1 ,θ 2 ,θ 3 ...θ ni , so that the likelihood function value is minimized, the likelihood function is like(θ)=F D (f 1 (θ 1 x 1 ),f 1 (θ 2 x 2 ),f 1 (θ 3 x 3 )…f1 (θ ni x ni )|θ 1 ,θ 2 ,θ 3 ...θ ni )-F D (T); Generate a preset multi-instance multi-label model and save the model parameter θ corresponding to the minimum likelihood function value 1 ,θ 2 ,θ 3 ...θ ni , the preset multi-instance multi-label model is F D (f s (θ 1 x 1 ),f s (θ 2 x 2 ),f s (θ 3 x 3 )…f s (θ ni x ni )|θ 1 ,θ 2 ,θ 3 ...θ ni ).

[0009] Optionally, in a second implementation manner of the first aspect of the embodiment of the present invention, the texture features of the original biopsy image are extracted, and the extracted texture features are expressed as structured features to generate a target graph structure, including: extracting the texture features of the original biopsy image to obtain multiple target texture feature images; using a preset clustering algorithm to classify the pixel points in the multiple target texture feature images to generate multiple local similar pixel clusters; and using the centers of the multiple local similar pixel clusters as vertices and triangulating them to generate a target graph structure.

[0010] Optionally, in a third implementation manner of the first aspect of the embodiment of the present invention, the texture features of the original biopsy image are extracted to obtain multiple target texture feature images, including: dividing the original biopsy image into n*n target areas, where n is a positive integer; comparing a target pixel in each target area with 8 pixels in a circular neighborhood around the target pixel in a clockwise or counterclockwise manner; if the pixel value of the target pixel is greater than the pixel value of the adjacent pixel, the adjacent pixel is assigned a value of 1; if the pixel value of the target pixel is less than or equal to the pixel value of the adjacent pixel, the adjacent pixel is assigned a value of 0; generating an 8-bit binary number for each pixel according to the multiple assignment results of each pixel, and converting it into a decimal value; calculating the histogram of each target area, and performing normalization processing to obtain a statistical histogram; connecting the statistical histograms of each target area to generate multiple target texture feature images of the original biopsy image.

[0011] Optionally, in a fourth implementation of the first aspect of the embodiment of the present invention, calling a preset training model to identify the target graph structure to obtain a target feature vector group includes: calling a preset training model to identify the target graph structure to generate multiple candidate features; dividing the multiple candidate features into a target feature vector group x 1 ,x 2 ,...,x ni , where the vector x ni Contains ni candidate features.

[0012] Optionally, in a fifth implementation of the first aspect of the embodiment of the present invention, the target feature vector group is labeled by the preset multi-instance multi-label model to obtain multiple target pathology labels, including:

[0013] According to the target feature vector group x 1 ,x 2 ,...,x ni The number of vectors determines the number of prediction models; the target feature vector group is divided into multiple prediction vector groups x according to the preset candidate case data 1 ,x 2 ,...,x ni , each prediction vector group corresponds to a preset candidate case data; calling the preset multi-instance multi-label model for the pathological label T of each prediction vector group s , and obtain multiple predicted values ​​F D (T), the preset multi-instance multi-label model is: F D (f s (θ 1 x 1 ),f s (θ 2 x 2),f s (θ 3 x 3 )…f s (θ ni x ni )|θ 1 ,θ 2 ,θ 3 ...θ ni ); Based on multiple predicted values ​​F D (T) Generate multiple pathological labels T 1 ,T 2 ,T 3 ...T s , each pathology label corresponds to a pathology description text.

[0014] Optionally, in a sixth implementation manner of the first aspect of the embodiment of the present invention, after labeling the target feature vector group through the preset multi-instance multi-label model to obtain multiple target pathology labels, the method also includes: verifying the multiple target pathology labels and initializing model parameters of the preset multi-instance multi-label model.

[0015] The second aspect of an embodiment of the present invention provides a multi-label multi-instance image detection device, comprising: a first acquisition unit, used to acquire a pathological area of ​​a confirmed biopsy image and a pathological feature description corresponding to the pathological area; a training unit, used to perform offline training on a multi-label multi-instance model based on the pathological area and the pathological feature description corresponding to the pathological area, to obtain a preset multi-instance multi-label model; a second acquisition unit, used to acquire an original biopsy image, wherein the original biopsy image is used to indicate the pathological characteristics of human tissue; an extraction unit, used to extract texture features from the original biopsy image, and perform structured feature expression on the extracted texture features to generate a target graph structure; an identification unit, used to call a preset training model to identify the target graph structure, to obtain a target feature vector group; and a labeling unit, used to label the target feature vector group through the preset multi-instance multi-label model, to obtain multiple target pathology labels.

[0016] Optionally, in a first implementation of the second aspect of the embodiment of the present invention, the training unit is specifically used to: extract texture and structural features according to the pathological area to generate a feature vector group; use the feature vector group as input for offline training of a multi-label multi-instance model, use the pathological feature description corresponding to the pathological area as output of the offline training of the multi-label multi-instance model, and train to obtain a multi-label multi-instance model F D (f 1 (θ 1 x 1 ),f 1 (θ 2 x2 ),f 1 (θ 3 x 3 )…f 1 (θ ni x ni )|θ 1 ,θ 2 ,θ 3 ...θ ni );Adjust θ 1 ,θ 2 ,θ 3 ...θ ni , so that the likelihood function value is minimized, the likelihood function is

[0017] like(θ)=F D (f 1 (θ 1 x 1 ),f 1 (θ 2 x 2 ),f 1 (θ 3 x 3 )…f 1 (θ ni x ni )|θ 1 ,θ 2 ,θ 3 ...θ ni )-F D (T); Generate a preset multi-instance multi-label model and save the model parameter θ corresponding to the minimum likelihood function value 1 ,θ 2 ,θ 3 ...θ ni , the preset multi-instance multi-label model is F D (f s (θ 1 x 1 ),f s (θ 2 x 2 ),f s (θ 3 x 3 )…f s (θ ni x ni )|θ 1 ,θ 2 ,θ 3 ...θ ni ).

[0018] Optionally, in a second implementation of the second aspect of the embodiment of the present invention, the extraction unit includes: an extraction module, used to extract texture features of the original biopsy image to obtain multiple target texture feature images; a classification module, used to classify pixel points in the multiple target texture feature images using a preset clustering algorithm to generate multiple local similar pixel clusters; a generation module, used to use the centers of the multiple local similar pixel clusters as vertices and triangulate them to generate a target graph structure.

[0019] Optionally, in a third implementation manner of the second aspect of the embodiment of the present invention, the extraction module is specifically used to: divide the original biopsy image into n*n target areas, where n is a positive integer; compare a target pixel in each target area with 8 pixels in a circular neighborhood around the target pixel in a clockwise or counterclockwise manner; if the pixel value of the target pixel is greater than the pixel value of the adjacent pixel, the adjacent pixel is assigned a value of 1; if the pixel value of the target pixel is less than or equal to the pixel value of the adjacent pixel, the adjacent pixel is assigned a value of 0; generate an 8-bit binary number for each pixel based on multiple assignment results of each pixel, and convert it into a decimal value; calculate the histogram of each target area, and perform normalization processing to obtain a statistical histogram; connect the statistical histograms of each target area to generate multiple target texture feature images of the original biopsy image.

[0020] Optionally, in a fourth implementation of the second aspect of the embodiment of the present invention, the recognition unit is specifically used to: call a preset training model to recognize the target graph structure and generate multiple candidate features; divide the multiple candidate features into a target feature vector group x 1 ,x 2 ,...,x ni , where the vector x ni Contains ni candidate features.

[0021] Optionally, in a fifth implementation of the second aspect of the embodiment of the present invention, the labeling unit is specifically configured to: 1 ,x 2 ,...,x ni The number of vectors determines the number of prediction models; the target feature vector group is divided into multiple prediction vector groups x according to the preset candidate case data 1 ,x 2 ,...,x ni , each prediction vector group corresponds to a preset candidate case data; calling the preset multi-instance multi-label model for the pathological label T of each prediction vector group s , and obtain multiple predicted values ​​F D (T), the preset multi-instance multi-label model is: F D (f s (θ1 x 1 ),f s (θ 2 x 2 ),f s (θ 3 x 3 )…f s (θ ni x ni )|θ 1 ,θ 2 ,θ 3 ...θ ni ); Based on multiple predicted values ​​F D (T) Generate multiple pathological labels T 1 ,T 2 ,T 3 ...T s , each pathology label corresponds to a pathology description text.

[0022] Optionally, in a sixth implementation of the second aspect of the embodiment of the present invention, the multi-label multi-instance image detection device also includes: a verification initialization unit, used to verify the multiple target pathology labels and initialize the model parameters of the preset multi-instance multi-label model.

[0023] A third aspect of an embodiment of the present invention provides a multi-label and multi-instance image detection device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the computer program, the multi-label and multi-instance image detection method described in any one of the above embodiments is implemented.

[0024] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the multi-label multi-instance image detection method described in any of the above embodiments are implemented.

[0025] In the technical solution provided by the embodiment of the present invention, the pathological area of ​​the confirmed biopsy image and the pathological feature description corresponding to the pathological area are obtained; the multi-label multi-instance model is offline trained according to the pathological area and the pathological feature description corresponding to the pathological area to obtain a preset multi-instance multi-label model; the original biopsy image is obtained, and the original biopsy image is used to indicate the pathological characteristics of human tissue; the texture feature of the original biopsy image is extracted, and the extracted texture feature is expressed as a structured feature to generate a target graph structure; the preset training model is called to identify the target graph structure to obtain a target feature vector group; the target feature vector group is annotated by the preset multi-instance multi-label model to obtain multiple target pathological labels. In the embodiment of the present invention, pathological image detection is performed in real time through a multi-label multi-instance model to generate pathological characteristics of multiple biopsy images, which reduces the analysis time of the pathological image, improves the analysis efficiency, and improves the accuracy of the analysis results. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A schematic diagram of an embodiment of a multi-label multi-instance image detection method according to an embodiment of the present invention;

[0027] Figure 2 A schematic diagram of another embodiment of the multi-label multi-instance image detection method according to an embodiment of the present invention;

[0028] Figure 3 A schematic diagram of an embodiment of a multi-label multi-instance image detection device in an embodiment of the present invention;

[0029] Figure 4 It is a schematic diagram of another embodiment of the multi-label multi-instance image detection device according to an embodiment of the present invention;

[0030] Figure 5 The figure is a schematic diagram of an embodiment of a multi-label and multi-instance image detection device in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The present invention provides a multi-label multi-instance image detection method, device, equipment and storage medium, which are used to perform pathological image detection in real time through a multi-label multi-instance model, generate pathological characteristics of multiple biopsy images, reduce the analysis time of pathological images, improve analysis efficiency, and improve the accuracy of analysis results.

[0032] In order to enable those skilled in the art to better understand the solutions of the present invention, the embodiments of the present invention will be described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0033] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] See also Figure 1 , a flowchart of a multi-label multi-instance image detection method provided by an embodiment of the present invention specifically includes:

[0035] 101. Obtain a pathological region of a confirmed biopsy image and a description of pathological features corresponding to the pathological region.

[0036] The server obtains the confirmed pathological region of the biopsy image and the pathological feature description corresponding to the pathological region.

[0037] It should be noted that the pathological feature description is a text description of the biopsy image in different states, which is used to reflect the medical characteristics of the pathological area in the biopsy image. For example, the text description of the biopsy image of the patient's stomach and the biopsy image of the patient's heart must be different, which are used to reflect the medical characteristics of the stomach and heart respectively.

[0038] It is understandable that the execution subject of the present invention may be a multi-label multi-instance image detection device, or a terminal or a server, which is not limited here. The embodiment of the present invention is described by taking the server as the execution subject as an example.

[0039] 102. Offline training is performed on a multi-label multi-instance model according to the pathological region and the pathological feature description corresponding to the pathological region to obtain a preset multi-instance multi-label model.

[0040] Specifically, the server extracts texture and structural features according to the pathological area to generate a feature vector group; the server uses the feature vector group as the input of the offline training multi-label multi-instance model, and uses the pathological feature description corresponding to the pathological area as the output of the offline training multi-label multi-instance model, and trains to obtain the multi-label multi-instance model F D (f 1 (θ 1 x 1 ),f 1 (θ 2 x 2 ),f1 (θ 3 x 3 )…f 1 (θ ni x ni )|θ 1 ,θ 2 ,θ 3 ...θ ni ); Server adjustment θ 1 ,θ 2 ,θ 3 ...θ ni , so that the likelihood function value is minimized, the likelihood function is

[0041] like(θ)=F D (f 1 (θ 1 x 1 ),f 1 (θ 2 x 2 ),f 1 (θ 3 x 3 )…f 1 (θ ni x ni )|θ 1 ,θ 2 ,θ 3 ...θ ni )-F D (T); The server generates a preset multi-instance multi-label model and saves the model parameter θ corresponding to the minimum likelihood function value 1 ,θ 2 ,θ 3 ...θ ni , the preset multi-instance multi-label model is F D (f s (θ 1 x 1 ),f s (θ 2 x 2 ),f s (θ 3 x 3 )…f s (θ ni x ni )|θ 1 ,θ 2 ,θ 3 ...θ ni ).

[0042] It should be noted that the pathological images and the corresponding diagnostic descriptions are used as training data to train the model. A multi-instance multi-label learning model based on Bayesian learning is used. The advantage of this model is that it can introduce prior distributions, that is, the empirical relationships between examples and labels can be introduced through prior probability distributions, so that the model can better use medical experience and knowledge. At the same time, this model can also use support vector machines as the final feature annotation output.

[0043] 103. Obtain an original biopsy image, where the original biopsy image is used to indicate pathological characteristics of human tissue.

[0044] The server obtains the original biopsy image, which is used to indicate the pathological characteristics of human tissue. Generally, when a patient's biopsy tissue is sent for examination, the laboratory will observe it under a microscope and take pictures. For parts with significant characteristics, a higher magnification may be used to take pictures. Therefore, a patient may have 5-10 biopsy images of different magnifications for the entire part or a few parts of the part to be examined, depending on the condition of the part to be examined.

[0045] 104. Extract texture features from the original biopsy image, express the extracted texture features in a structured manner, and generate a target graph structure.

[0046] The server extracts texture features from the original biopsy image, and expresses the extracted texture features in a structured feature to generate a target graph structure. Specifically, (1) the server extracts texture features from the original biopsy image to obtain multiple target texture feature images; step (1) specifically includes: the server divides the original biopsy image into n*n target areas, where n is a positive integer; the server compares a target pixel in each target area with 8 pixels in a circular neighborhood around the target pixel in a clockwise or counterclockwise manner; if the pixel value of the target pixel is greater than the pixel value of the adjacent pixel, the server assigns the adjacent pixel a value of 1; if the pixel value of the target pixel is less than or equal to the pixel value of the adjacent pixel, the server assigns the adjacent pixel a value of 0; the server generates an 8-bit binary number for each pixel based on the multiple assignment results of each pixel, and converts it into a decimal value; the server calculates the histogram of each target area, and performs normalization processing to obtain a statistical histogram; the server connects the statistical histograms of each target area to generate multiple target texture feature images of the original biopsy image.

[0047] (2) The server uses a preset clustering algorithm to classify the pixels in multiple target texture feature images and generate multiple local similar pixel clusters;

[0048] The server uses the K-means algorithm to cluster k points in the target texture feature image as the center to obtain k categories. For example, the server clusters pixels based on the similar properties of pixels in the same object area, starting from the initial area (such as a small neighborhood or even each pixel), and merges adjacent pixels or other areas with the same properties into the current area to gradually grow the area until there are no more pixels or target areas that can be merged.

[0049] (3) The server uses the centers of multiple local similar pixel clusters as vertices and triangulates them to generate the target graph structure.

[0050] The server triangulates each pixel cluster to obtain the three-dimensional information of the pixel cluster, prevents the loss of three-dimensional spatial information, and generates a three-dimensional target graph structure.

[0051] It should be noted that in the process of texture feature extraction, it is necessary to exclude the influence of filled pixels on the feature extraction results. Two-dimensional wavelet transform (2D discrete Wavelet transformation, 2D-DWT), gray-level co-occurrence matrix (gray-level co-occurrence matrix, GLCM), local binary pattern (local binary pattern, LBP) and other methods can also be used to extract features for a single area. Among them, when the input image resolution is the same as the training image resolution, the gray-level co-occurrence matrix method can be selected for texture feature extraction.

[0052] 105. Call the preset training model to identify the target graph structure and obtain the target feature vector group.

[0053] The server calls the preset training model to identify the target graph structure and obtains the target feature vector group. Specifically, the server calls the preset training model to identify the target graph structure and generates multiple candidate features; the server divides the multiple candidate features into the target feature vector group x 1 ,x 2 ,...,x ni , where the vector x ni Contains ni candidate features.

[0054] 106. The target feature vector group is labeled by using a preset multi-instance multi-label model to obtain multiple target pathology labels.

[0055] The server groups x according to the target feature vector 1 ,x 2 ,...,x niThe server divides the target feature vector group into multiple prediction vector groups x according to the preset candidate case data. 1 ,x 2 ,...,x ni , each prediction vector group corresponds to a preset candidate case data; the server calls the preset multi-instance multi-label model to calculate the pathological label T of each prediction vector group s , and obtain multiple predicted values ​​F D (T), the preset multi-instance multi-label model is:

[0056] F D (f s (θ 1 x 1 ),f s (θ 2 x 2 ),f s (θ 3 x 3 )…f s (θ ni x ni )|θ 1 ,θ 2 ,θ 3 ...θ ni ); the server calculates the predicted value F according to the multiple predicted values ​​F D (T) Generate multiple pathological labels T 1 ,T 2 ,T 3 ...T s , each pathology label corresponds to a pathology description text.

[0057] It is understandable that one pathological label corresponds to a paragraph of description text. For example, a biopsy image corresponds to four pathological labels, namely: epidermal spinous layer thickening, elongated dermal process, superficial dermal blood vessels dense, lymphoid tissue cell infiltration. For another example, another biopsy image corresponds to six pathological labels, namely: epidermal hyperkeratosis; papilloma-like hyperplasia; pseudo-horn cysts are visible; dermal process cells are thick or thin strips, interwoven into a network; superficial dermal blood vessels dense; lymphoid tissue cell infiltration.

[0058] The embodiment of the present invention performs pathological image detection in real time through a multi-label multi-instance model to generate pathological characteristics of multiple biopsy images, thereby reducing the analysis time of the pathological images, improving the analysis efficiency, and improving the accuracy of the analysis results.

[0059] See also Figure 2 , another flow chart of the multi-label multi-instance image detection method provided by an embodiment of the present invention, specifically includes:

[0060] 201. Obtain a pathological region of a confirmed biopsy image and a pathological feature description corresponding to the pathological region.

[0061] The server obtains the confirmed pathological region of the biopsy image and the pathological feature description corresponding to the pathological region.

[0062] It should be noted that the pathological feature description is a text description of the biopsy image in different states, which is used to reflect the medical characteristics of the pathological area in the biopsy image. For example, the text description of the biopsy image of the patient's stomach and the biopsy image of the patient's heart must be different, which are used to reflect the medical characteristics of the stomach and heart respectively.

[0063] It is understandable that the execution subject of the present invention may be a multi-label multi-instance image detection device, or a terminal or a server, which is not limited here. The embodiment of the present invention is described by taking the server as the execution subject as an example.

[0064] 202. Offline training is performed on a multi-label multi-instance model according to the pathological region and the pathological feature description corresponding to the pathological region to obtain a preset multi-instance multi-label model.

[0065] Specifically, the server extracts texture and structural features according to the pathological area to generate a feature vector group; the server uses the feature vector group as the input of the offline training multi-label multi-instance model, and uses the pathological feature description corresponding to the pathological area as the output of the offline training multi-label multi-instance model, and trains to obtain the multi-label multi-instance model F D (f 1 (θ 1 x 1 ),f 1 (θ 2 x 2 ),f 1 (θ 3 x 3 )…f 1 (θ ni x ni )|θ 1 ,θ 2 ,θ 3 ...θ ni ); Server adjustment θ 1 ,θ 2 ,θ 3 ...θ ni , so that the likelihood function value is minimized, the likelihood function is

[0066] like(θ)=F D (f 1 (θ 1 x 1 ),f 1 (θ 2 x2 ),f 1 (θ 3 x 3 )…f 1 (θ ni x ni )|θ 1 ,θ 2 ,θ 3 ...θ ni )-F D (T); The server generates a preset multi-instance multi-label model and saves the model parameter θ corresponding to the minimum likelihood function value 1 ,θ 2 ,θ 3 ...θ ni , the preset multi-instance multi-label model is F D (f s (θ 1 x 1 ),f s (θ 2 x 2 ),f s (θ 3 x 3 )…f s (θ ni x ni )|θ 1 ,θ 2 ,θ 3 ...θ ni ).

[0067] It should be noted that the pathological images and the corresponding diagnostic descriptions are used as training data to train the model. A multi-instance multi-label learning model based on Bayesian learning is used. The advantage of this model is that it can introduce prior distributions, that is, the empirical relationships between examples and labels can be introduced through prior probability distributions, so that the model can better use medical experience and knowledge. At the same time, this model can also use support vector machines as the final feature annotation output.

[0068] 203. Obtain an original biopsy image, where the original biopsy image is used to indicate pathological characteristics of human tissue.

[0069] The server obtains the original biopsy image, which is used to indicate the pathological characteristics of human tissue. Generally, when a patient's biopsy tissue is sent for examination, the laboratory will observe it under a microscope and take pictures. For parts with significant characteristics, a higher magnification may be used to take pictures. Therefore, a patient may have 5-10 biopsy images of different magnifications for the entire part or a few parts of the part to be examined, depending on the condition of the part to be examined.

[0070] 204. Extract texture features from the original biopsy image, and express the extracted texture features as structured features to generate a target graph structure.

[0071] The server extracts texture features from the original biopsy image, and expresses the extracted texture features in a structured feature to generate a target graph structure. Specifically, (1) the server extracts texture features from the original biopsy image to obtain multiple target texture feature images; step (1) specifically includes: the server divides the original biopsy image into n*n target areas, where n is a positive integer; the server compares a target pixel in each target area with 8 pixels in a circular neighborhood around the target pixel in a clockwise or counterclockwise manner; if the pixel value of the target pixel is greater than the pixel value of the adjacent pixel, the server assigns the adjacent pixel a value of 1; if the pixel value of the target pixel is less than or equal to the pixel value of the adjacent pixel, the server assigns the adjacent pixel a value of 0; the server generates an 8-bit binary number for each pixel based on the multiple assignment results of each pixel, and converts it into a decimal value; the server calculates the histogram of each target area, and performs normalization processing to obtain a statistical histogram; the server connects the statistical histograms of each target area to generate multiple target texture feature images of the original biopsy image.

[0072] (2) The server uses a preset clustering algorithm to classify the pixels in multiple target texture feature images and generate multiple local similar pixel clusters;

[0073] The server uses the K-means algorithm to cluster k points in the target texture feature image as the center to obtain k categories. For example, the server clusters pixels based on the similar properties of pixels in the same object area, starting from the initial area (such as a small neighborhood or even each pixel), and merges adjacent pixels or other areas with the same properties into the current area to gradually grow the area until there are no more pixels or target areas that can be merged.

[0074] (3) The server uses the centers of multiple local similar pixel clusters as vertices and triangulates them to generate the target graph structure.

[0075] The server triangulates each pixel cluster to obtain the three-dimensional information of the pixel cluster, prevents the loss of three-dimensional spatial information, and generates a three-dimensional target graph structure.

[0076] It should be noted that in the process of texture feature extraction, it is necessary to exclude the influence of filled pixels on the feature extraction results. Two-dimensional wavelet transform (2D discrete Wavelet transformation, 2D-DWT), gray-level co-occurrence matrix (gray-level co-occurrence matrix, GLCM), local binary pattern (local binary pattern, LBP) and other methods can also be used to extract features for a single area. Among them, when the input image resolution is the same as the training image resolution, the gray-level co-occurrence matrix method can be selected for texture feature extraction.

[0077] 205. Call a preset training model to identify the target graph structure and obtain a target feature vector group.

[0078] The server calls the preset training model to identify the target graph structure and obtains the target feature vector group. Specifically, the server calls the preset training model to identify the target graph structure and generates multiple candidate features; the server divides the multiple candidate features into the target feature vector group x 1 ,x 2 ,...,x ni , where the vector x ni Contains ni candidate features.

[0079] 206. Label the target feature vector group using a preset multi-instance multi-label model to obtain multiple target pathology labels.

[0080] The server groups x according to the target feature vector 1 ,x 2 ,...,x ni The server divides the target feature vector group into multiple prediction vector groups x according to the preset candidate case data. 1 ,x 2 ,...,x ni , each prediction vector group corresponds to a preset candidate case data; the server calls the preset multi-instance multi-label model to calculate the pathological label T of each prediction vector group s , and obtain multiple predicted values ​​F D (T), the preset multi-instance multi-label model is:

[0081] F D (f s (θ 1 x 1 ),f s (θ 2 x 2 ),f s (θ 3 x3 )…f s (θ ni x ni )|θ 1 ,θ 2 ,θ 3 ...θ ni ); the server calculates the predicted value F according to the multiple predicted values ​​F D (T) Generate multiple pathological labels T 1 ,T 2 ,T 3 ...T s , each pathology label corresponds to a pathology description text.

[0082] It is understandable that one pathological label corresponds to a paragraph of description text. For example, a biopsy image corresponds to four pathological labels, namely: epidermal spinous layer thickening, elongated dermal process, superficial dermal blood vessels dense, lymphoid tissue cell infiltration. For another example, another biopsy image corresponds to six pathological labels, namely: epidermal hyperkeratosis; papilloma-like hyperplasia; pseudo-horn cysts are visible; dermal process cells are thick or thin strips, interwoven into a network; superficial dermal blood vessels dense; lymphoid tissue cell infiltration.

[0083] 207. Verify multiple target pathology labels and initialize model parameters of a preset multi-instance multi-label model.

[0084] The server verifies multiple target pathology labels and initializes the model parameters of the preset multi-instance multi-label model. After the multi-instance multi-label learning model is designed and trained using historical data, the parameters of the model can be solidified, and a Web application can be implemented in Java to accept biopsy images uploaded by doctors and obtain the feature annotations of the biopsy images or the probabilities of their various features through the model in real time.

[0085] Use the pathological areas and corresponding pathological features of historically confirmed biopsy images to train the multi-label multi-instance learner offline, and solidify the trained parameters for the next prediction load. Perform offline training regularly, once a week or month, to ensure the accuracy of the model.

[0086] In the technical solution provided by the embodiment of the present invention, the pathological area of ​​the confirmed biopsy image and the pathological feature description corresponding to the pathological area are obtained; the multi-label multi-instance model is offline trained according to the pathological area and the pathological feature description corresponding to the pathological area to obtain a preset multi-instance multi-label model; the original biopsy image is obtained, and the original biopsy image is used to indicate the pathological characteristics of human tissue; the texture feature of the original biopsy image is extracted, and the extracted texture feature is expressed as a structured feature to generate a target graph structure; the preset training model is called to identify the target graph structure to obtain a target feature vector group; the target feature vector group is annotated by the preset multi-instance multi-label model to obtain multiple target pathological labels. In the embodiment of the present invention, pathological image detection is performed in real time through a multi-label multi-instance model to generate pathological characteristics of multiple biopsy images, which reduces the analysis time of the pathological image, improves the analysis efficiency, and improves the accuracy of the analysis results.

[0087] The above describes the multi-label multi-instance image detection method in the embodiment of the present invention. The following describes the multi-label multi-instance image detection device in the embodiment of the present invention. Figure 3 An embodiment of a multi-label multi-instance image detection device in an embodiment of the present invention includes:

[0088] A first acquisition unit 301 is used to acquire a confirmed pathological region of a biopsy image and a pathological feature description corresponding to the pathological region;

[0089] A training unit 302 is used to perform offline training on a multi-label multi-instance model according to the pathological region and the pathological feature description corresponding to the pathological region to obtain a preset multi-instance multi-label model;

[0090] A second acquisition unit 303 is used to acquire an original biopsy image, where the original biopsy image is used to indicate pathological characteristics of human tissue;

[0091] An extraction unit 304 is used to extract texture features from the original biopsy image, and perform structured feature expression on the extracted texture features to generate a target graph structure;

[0092] The recognition unit 305 is used to call a preset training model to recognize the target graph structure and obtain a target feature vector group;

[0093] The labeling unit 306 is used to label the target feature vector group by using the preset multi-instance multi-label model to obtain multiple target pathology labels.

[0094] The embodiment of the present invention performs pathological image detection in real time through a multi-label multi-instance model to generate pathological characteristics of multiple biopsy images, thereby reducing the analysis time of the pathological images, improving the analysis efficiency, and improving the accuracy of the analysis results.

[0095] See also Figure 4 Another embodiment of the multi-label multi-instance image detection device in the embodiment of the present invention includes:

[0096] A first acquisition unit 301 is used to acquire a confirmed pathological region of a biopsy image and a pathological feature description corresponding to the pathological region;

[0097] A training unit 302 is used to perform offline training on a multi-label multi-instance model according to the pathological region and the pathological feature description corresponding to the pathological region to obtain a preset multi-instance multi-label model;

[0098] A second acquisition unit 303 is used to acquire an original biopsy image, where the original biopsy image is used to indicate pathological characteristics of human tissue;

[0099] An extraction unit 304 is used to extract texture features from the original biopsy image, and perform structured feature expression on the extracted texture features to generate a target graph structure;

[0100] The recognition unit 305 is used to call a preset training model to recognize the target graph structure and obtain a target feature vector group;

[0101] The labeling unit 306 is used to label the target feature vector group by using the preset multi-instance multi-label model to obtain multiple target pathology labels.

[0102] Optionally, the training unit 302 is specifically configured to:

[0103] According to the pathological area, texture and structural features are extracted to generate a feature vector group; the feature vector group is used as the input of the offline training multi-label multi-instance model, and the pathological feature description corresponding to the pathological area is used as the output of the offline training multi-label multi-instance model, and the multi-label multi-instance model F is obtained by training. D (f 1 (θ 1 x 1 ),f 1 (θ 2 x 2 ),f 1 (θ 3 x 3 )…f 1 (θ ni x ni )|θ 1 ,θ 2 ,θ3 ...θ ni );Adjust θ 1 ,θ 2 ,θ 3 ...θ ni , so that the likelihood function value is minimized, the likelihood function is like(θ)=F D (f 1 (θ 1 x 1 ),f 1 (θ 2 x 2 ),f 1 (θ 3 x 3 )…f 1 (θ ni x ni )|θ 1 ,θ 2 ,θ 3 ...θ ni )-F D (T); Generate a preset multi-instance multi-label model and save the model parameter θ corresponding to the minimum likelihood function value 1 ,θ 2 ,θ 3 ...θ ni , the preset multi-instance multi-label model is F D (f s (θ 1 x 1 ),f s (θ 2 x 2 ),f s (θ 3 x 3 )…f s (θ ni x ni )|θ 1 ,θ 2 ,θ 3 ...θ ni ).

[0104] Optionally, the extraction unit 304 includes:

[0105] An extraction module 3041 is used to extract texture features of the original biopsy image to obtain a plurality of target texture feature images;

[0106] A classification module 3042 is used to classify the pixel points in the plurality of target texture feature images using a preset clustering algorithm to generate a plurality of local similar pixel clusters;

[0107] The generating module 3043 is used to use the centers of the multiple local similar pixel clusters as vertices and perform triangulation to generate a target graph structure.

[0108] Optionally, the extraction module 3041 is specifically used for:

[0109] The original biopsy image is divided into n*n target areas, where n is a positive integer; a target pixel in each target area is compared with 8 pixels in a circular neighborhood around the target pixel in a clockwise or counterclockwise manner; if the pixel value of the target pixel is greater than the pixel value of the adjacent pixel, the adjacent pixel is assigned a value of 1; if the pixel value of the target pixel is less than or equal to the pixel value of the adjacent pixel, the adjacent pixel is assigned a value of 0; an 8-bit binary number of each pixel is generated according to multiple assignment results of each pixel, and converted into a decimal value; the histogram of each target area is calculated, and normalized to obtain a statistical histogram; the statistical histograms of each target area are connected to generate multiple target texture feature images of the original biopsy image.

[0110] Optionally, the identification unit 305 is specifically configured to:

[0111] Call the preset training model to identify the target graph structure and generate multiple candidate features; divide the multiple candidate features into target feature vector groups x 1 ,x 2 ,...,x ni , where the vector x ni Contains ni candidate features.

[0112] Optionally, the labeling unit 306 is specifically used for:

[0113] According to the target feature vector group x 1 ,x 2 ,...,x ni The number of vectors determines the number of prediction models; the target feature vector group is divided into multiple prediction vector groups x according to the preset candidate case data 1 ,x 2 ,...,x ni , each prediction vector group corresponds to a preset candidate case data; calling the preset multi-instance multi-label model for the pathological label T of each prediction vector group s , and obtain multiple predicted values ​​F D (T), the preset multi-instance multi-label model is: F D (f s (θ 1 x 1 ),f s (θ 2 x 2 ),f s (θ3 x 3 )…f s (θ ni x ni )|θ 1 ,θ 2 ,θ 3 ...θ ni ); Based on multiple predicted values ​​F D (T) Generate multiple pathological labels T 1 ,T 2 ,T 3 ...T s , each pathology label corresponds to a pathology description text.

[0114] Optionally, the multi-label multi-instance image detection device further includes:

[0115] The verification initialization unit 307 is used to verify the multiple target pathology labels and initialize the model parameters of the preset multi-instance multi-label model.

[0116] In the technical solution provided by the embodiment of the present invention, the pathological area of ​​the confirmed biopsy image and the pathological feature description corresponding to the pathological area are obtained; the multi-label multi-instance model is offline trained according to the pathological area and the pathological feature description corresponding to the pathological area to obtain a preset multi-instance multi-label model; the original biopsy image is obtained, and the original biopsy image is used to indicate the pathological characteristics of human tissue; the texture feature of the original biopsy image is extracted, and the extracted texture feature is expressed as a structured feature to generate a target graph structure; the preset training model is called to identify the target graph structure to obtain a target feature vector group; the target feature vector group is annotated by the preset multi-instance multi-label model to obtain multiple target pathological labels. In the embodiment of the present invention, pathological image detection is performed in real time through a multi-label multi-instance model to generate pathological characteristics of multiple biopsy images, which reduces the analysis time of the pathological image, improves the analysis efficiency, and improves the accuracy of the analysis results.

[0117] above Figure 3 to Figure 4 The multi-label and multi-instance image detection device in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The multi-label and multi-instance image detection device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0118] Figure 5It is a structural schematic diagram of a multi-label multi-instance image detection device provided by an embodiment of the present invention. The multi-label multi-instance image detection device 500 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 501 (for example, one or more processors) and a memory 509, and one or more storage media 508 (for example, one or more massive storage devices) storing application programs 507 or data 506. Among them, the memory 509 and the storage medium 508 can be short-term storage or permanent storage. The program stored in the storage medium 508 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the multi-label multi-instance image detection device. Furthermore, the processor 501 can be configured to communicate with the storage medium 508 to execute a series of instruction operations in the storage medium 508 on the multi-label multi-instance image detection device 500.

[0119] The multi-label multi-instance image detection device 500 may also include one or more power supplies 502, one or more wired or wireless network interfaces 503, one or more input and output interfaces 504, and / or one or more operating systems 505, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will appreciate that Figure 5 The structure of the multi-label multi-instance image detection device shown in the figure does not constitute a limitation on the multi-label multi-instance image detection device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. The processor 501 can perform the functions of the first acquisition unit 301, the training unit 302, the second acquisition unit 303, the extraction unit 304, the recognition unit 305, the annotation unit 306 and the verification initialization unit 307 in the above-mentioned embodiments.

[0120] Combine the following Figure 5 The following is a detailed introduction to the various components of the multi-label and multi-instance image detection device:

[0121] The processor 501 is the control center of the multi-label multi-instance image detection device, and can process according to the set multi-label multi-instance image detection method. The processor 501 uses various interfaces and lines to connect the various parts of the entire multi-label multi-instance image detection device, and executes various functions and processes data of the multi-label multi-instance image detection device by running or executing software programs and / or modules stored in the memory 509, and calling the data stored in the memory 509, thereby reducing the analysis time of the pathological image, improving the analysis efficiency, and improving the accuracy of the analysis results. The storage medium 508 and the memory 509 are both carriers for storing data. In the embodiment of the present invention, the storage medium 508 can refer to an internal memory with a small storage capacity but a fast speed, and the memory 509 can be an external memory with a large storage capacity but a slow storage speed.

[0122] The memory 509 can be used to store software programs and modules. The processor 501 executes various functional applications and data processing of the multi-label multi-instance image detection device 500 by running the software programs and modules stored in the memory 509. The memory 509 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as extracting texture features from the original biopsy image, and expressing the extracted texture features as structured features to generate a target graph structure), etc.; the data storage area may store data created according to the use of the multi-label multi-instance image detection device (such as a target feature vector group, etc.), etc. In addition, the memory 509 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices. The multi-label multi-instance image detection method program and the received data stream provided in the embodiment of the present invention are stored in the memory, and when needed, the processor 501 calls from the memory 509.

[0123] When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, twisted pair) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., an optical disk), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0124] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0125] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0126] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0127] In addition, each functional unit in the embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0128] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0129] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-label multi-instance image detection method, characterized in that: include: Obtaining the pathological region of the confirmed biopsy image and the description of the pathological features corresponding to the pathological region; Offline training of a multi-label multi-instance model is performed according to the pathological region and the pathological feature description corresponding to the pathological region to obtain a preset multi-instance multi-label model; Acquiring an original biopsy image, wherein the original biopsy image is used to indicate pathological characteristics of human tissue; Extracting texture features from the original biopsy image, and expressing the extracted texture features as structured features to generate a target graph structure; Call the preset training model to identify the target graph structure and obtain the target feature vector group; The target feature vector group is labeled by using the preset multi-instance multi-label model to obtain multiple target pathology labels; The extracting of texture features from the original biopsy image and expressing the extracted texture features as structured features to generate a target graph structure includes: Extracting texture features of the original biopsy image to obtain a plurality of target texture feature images; Using a preset clustering algorithm to classify the pixel points in the multiple target texture feature images to generate multiple local similar pixel clusters; The centers of the multiple local similar pixel clusters are used as vertices and triangulated to generate a target graph structure; The calling of the preset training model to identify the target graph structure and obtain the target feature vector group includes: Call the preset training model to identify the target graph structure and generate multiple candidate features; Divide multiple candidate features into target feature vector groups x1, x2, ..., x ni , where the vector x ni Contains ni candidate features; The target feature vector group is labeled by the preset multi-instance multi-label model to obtain multiple target pathology labels, including: According to the target feature vector group x1,x2,...,x ni The number of vectors determines the number of prediction models; According to the preset candidate case data, the target feature vector group is divided into multiple prediction vector groups x1, x2, ..., x ni , each prediction vector group corresponds to a preset candidate case data; Call the preset multi-instance multi-label model to predict the pathological label T of each vector group s , and obtain multiple predicted values ​​F D (T), the preset multi-instance multi-label model is: F D (f s (θ1x1),f s (θ2x2),f s (θ3x3)…f s (i ni x ni )|θ1,θ2,θ3...θ ni ); According to multiple prediction values ​​F D (T) Generate multiple pathological labels T1, T2, T3...T s , each pathology label corresponds to a pathology description text.

2. The multi-label multi-instance image detection method according to claim 1, characterized in that: The method of performing offline training on a multi-label multi-instance model according to the pathological region and the pathological feature description corresponding to the pathological region to obtain a preset multi-instance multi-label model includes: Extracting texture and structural features according to the pathological area to generate a feature vector group; The feature vector group is used as the input of the offline training multi-label multi-instance model, and the pathological feature description corresponding to the pathological area is used as the output of the offline training multi-label multi-instance model to train the multi-label multi-instance model F. D (f1(θ1x1),f1(θ2x2),f1(θ3x3)…f1(θ ni x ni )|θ1,θ2,θ3...θ ni ); Adjust θ1, θ2, θ3...θ ni , so that the likelihood function value is minimized, the likelihood function is like(θ)=F D (f1(θ1x1),f1(θ2x2),f1(θ3x3)…f1(θ ni x ni )|θ1,θ2,θ3...θ ni )-F D (T); Generate a preset multi-instance multi-label model and save the model parameters θ1, θ2, θ3...θ corresponding to the minimum likelihood function value ni , the preset multi-instance multi-label model is F D (f s (θ1x1),f s (θ2x2),f s (θ3x3)…f s (θ ni x ni )|θ1,θ2,θ3...θ ni ).

3. The multi-label multi-instance image detection method according to claim 1, characterized in that: The extracting of texture features of the original biopsy image to obtain a plurality of target texture feature images includes: Dividing the original biopsy image into n*n target areas, where n is a positive integer; Compare a target pixel in each target area with eight pixels in a circular neighborhood around the target pixel in a clockwise or counterclockwise manner; If the pixel value of the target pixel is greater than the pixel value of the adjacent pixel, the adjacent pixel is assigned a value of 1; If the pixel value of the target pixel is less than or equal to the pixel value of the adjacent pixel, the adjacent pixel is assigned a value of 0; Generate an 8-bit binary number for each pixel according to the multiple assignment results of each pixel, and convert it into a decimal value; Calculate the histogram of each target area and perform normalization to obtain a statistical histogram; The statistical histograms of each target area are connected to generate multiple target texture feature images of the original biopsy image.

4. The multi-label multi-instance image detection method according to any one of claims 1 to 3, characterized in that: After labeling the target feature vector group by using the preset multi-instance multi-label model to obtain multiple target pathology labels, the method further includes: The multiple target pathology labels are verified, and model parameters of the preset multi-instance multi-label model are initialized.

5. A multi-label multi-instance image detection device, characterized in that: include: A first acquisition unit is used to acquire a pathological region of a confirmed biopsy image and a pathological feature description corresponding to the pathological region; A training unit, configured to perform offline training on a multi-label multi-instance model according to the pathological region and the pathological feature description corresponding to the pathological region to obtain a preset multi-instance multi-label model; A second acquisition unit, used to acquire an original biopsy image, where the original biopsy image is used to indicate pathological characteristics of human tissue; An extraction unit, used to extract texture features from the original biopsy image, and perform structured feature expression on the extracted texture features to generate a target graph structure; The recognition unit is used to call a preset training model to recognize the target graph structure and obtain a target feature vector group; A labeling unit, used for labeling the target feature vector group by using the preset multi-instance multi-label model to obtain multiple target pathology labels; The extraction unit includes: an extraction module, which is used to extract the texture features of the original biopsy image to obtain multiple target texture feature images; a classification module, which is used to classify the pixel points in the multiple target texture feature images using a preset clustering algorithm to generate multiple local similar pixel clusters; a generation module, which is used to use the centers of the multiple local similar pixel clusters as vertices and triangulate them to generate a target graph structure; The recognition unit is specifically used to: call a preset training model to recognize the target graph structure and generate multiple candidate features; divide the multiple candidate features into target feature vector groups x1, x2, ..., x ni , where the vector x ni Contains ni candidate features; The labeling unit is specifically used to: ni The number of vectors determines the number of prediction models; the target feature vector group is divided into multiple prediction vector groups x1, x2, ..., x according to the preset candidate case data. ni , each prediction vector group corresponds to a preset candidate case data; calling the preset multi-instance multi-label model for the pathological label T of each prediction vector group s , and obtain multiple predicted values ​​F D (T), the preset multi-instance multi-label model is: F D (f s (θ1x1),f s (θ2x2),f s (θ3x3)…f s (θ ni x ni )|θ1,θ2,θ3...θ ni );Based on multiple predicted values ​​F D (T) Generate multiple pathological labels T1, T2, T3...T s , each pathology label corresponds to a pathology description text.

6. The multi-label multi-instance image detection device according to claim 5, characterized in that: The training unit is specifically used to: extract texture and structural features according to the pathological area to generate a feature vector group; use the feature vector group as the input of the offline training multi-label multi-instance model, use the pathological feature description corresponding to the pathological area as the output of the offline training multi-label multi-instance model, and train to obtain a multi-label multi-instance model F D (f1(θ1x1),f1(θ2x2),f1(θ3x3)…f1(θ ni x ni )|θ1,θ2,θ3...θ ni );Adjust θ1,θ2,θ3...θ ni , so that the likelihood function value is minimized, the likelihood function is like(θ)=F D (f1(θ1x1),f1(θ2x2),f1(θ3x3)…f1(θ ni x ni )|θ1,θ2,θ3...θ ni )-F D (T); Generate a preset multi-instance multi-label model and save the model parameters θ1, θ2, θ3...θ corresponding to the minimum likelihood function value ni , the preset multi-instance multi-label model is F D (f s (θ1x1),f s (θ2x2),f s (θ3x3)…f s (θ ni x ni )|θ1,θ2,θ3...θ ni ).

7. The multi-label multi-instance image detection device according to claim 5, characterized in that: The extraction module is specifically used for: dividing the original biopsy image into n*n target areas, where n is a positive integer; comparing a target pixel in each target area with 8 pixels in a circular neighborhood around the target pixel in a clockwise or counterclockwise manner; if the pixel value of the target pixel is greater than the pixel value of the adjacent pixel, the adjacent pixel is assigned a value of 1; if the pixel value of the target pixel is less than or equal to the pixel value of the adjacent pixel, the adjacent pixel is assigned a value of 0; generating an 8-bit binary number for each pixel according to multiple assignment results of each pixel, and converting the 8-bit binary number into a decimal value; Calculate the histogram of each target area and perform normalization to obtain a statistical histogram; The statistical histograms of each target area are connected to generate multiple target texture feature images of the original biopsy image.

8. The multi-label multi-instance image detection device according to any one of claims 5 to 7, characterized in that: The multi-label multi-instance image detection device further includes: a verification initialization unit, which is used to verify the multiple target pathology labels and initialize the model parameters of the preset multi-instance multi-label model.

9. A multi-label multi-instance image detection device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the multi-label multi-instance image detection method according to any one of claims 1 to 4 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the multi-label multi-instance image detection method according to any one of claims 1 to 4 is implemented.