A method, device, equipment and storage medium for detecting biological tissue images

By calculating the characteristic distance between the vinegar dyed image and the normal saline image, selecting the vinegar dyed image with the largest difference for biopsy area prediction, solving the problem of the difference in the reaction time of the lesion area in the prior art, and improving the accuracy and effectiveness of the prediction .

CN115719345BActive Publication Date: 2024-11-29TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202211491324.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-11-25
Publication Date
2024-11-29
Estimated Expiration
2042-11-25

AI Technical Summary

Technical Problem

When selecting biological tissue images at fixed time after vinegar dyeing as biopsy area prediction images, the differences in the reaction time of the lesion area between different patients are not considered, resulting in a low accuracy of biopsy area prediction.

Method used

By acquiring multiple biological tissue images, calling the image classification model for identification and feature extraction, calculating the inter-class feature distance and intra-class feature distance between vinegar dyed images and normal saline images, and selecting the most different vinegar dyed images as the target vinegar dyed images The image is detected to determine the biopsy area.

Benefits of technology

The accuracy and effectiveness of biopsy area prediction were improved, and the differences in the response time of the lesion area between different patients were fully considered.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115719345B_ABST
    Figure CN115719345B_ABST
Patent Text Reader

Abstract

The present application discloses a biological tissue image detection method, device, equipment and storage medium based on artificial intelligence technology. The present application method includes: calling a first image classification model to identify each biological tissue image, and obtaining a staining classification result and image features of each biological tissue image; obtaining a physiological saline image and P vinegar-stained images according to the staining classification results; determining P inter-class feature distances and P intra-class feature distances according to the image features of the physiological saline image and the image features of each vinegar-stained image; determining the comprehensive feature distance of each vinegar-stained image according to the P inter-class feature distances and P intra-class feature distances; using the vinegar-stained image with the largest comprehensive feature distance as the target vinegar-stained image; calling a target detection model to detect the target vinegar-stained image. The present application can select the vinegar-stained image with the largest difference as the vinegar-stained image at the time of the maximum disease reaction, which is conducive to improving the accuracy and effectiveness of biopsy area prediction.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, equipment and storage medium for detecting biological tissue images. Background Art

[0002] Image recognition technology is an important field of artificial intelligence (AI). It refers to the technology of object recognition in images to identify targets and objects of various different patterns. In recent years, image recognition technology has been increasingly used in the medical field, and the results of image recognition can assist doctors in determining treatment plans.

[0003] At present, in the related technology, the prediction of the biopsy area can be realized based on the image recognition technology. Specifically, first, the extracted lesion tissue is stained with vinegar. Then, the biological tissue image at a fixed time after vinegar staining is collected. Then, the image recognition model is called to identify and detect the biological tissue image to obtain the biopsy area.

[0004] The inventors found that the current solution has at least the following problems: the image at a fixed time after vinegar staining is selected as the predicted image of the biopsy area, and the difference in the reaction time of the lesion area between different patients is not taken into account. Therefore, the image predicted for the biopsy area may not be the image at the time of the maximum disease reaction. As a result, the accuracy of the biopsy area prediction is low. Summary of the invention

[0005] The embodiments of the present application provide a biological tissue image detection method, apparatus, device and storage medium, which can select the acetic acid-stained image with the largest difference as the acetic acid-stained image at the moment of maximum disease reaction. Thus, the differences in the reaction time of the lesion area between different patients are fully taken into account, which is conducive to improving the accuracy and effectiveness of biopsy area prediction.

[0006] In view of this, the present application provides, on one hand, a method for detecting a biological tissue image, comprising:

[0007] Acquire a first image set to be identified, wherein the first image set includes at least two biological tissue images;

[0008] Calling the first image classification model to identify each biological tissue image in the first image set, and obtaining a staining classification result of each biological tissue image and an image feature of each biological tissue image, wherein the staining classification result is used to indicate the staining type to which the biological tissue image belongs;

[0009] According to the staining classification result of each biological tissue image, a physiological saline image and P vinegar-stained images are obtained from the first image set, wherein the physiological saline image and the vinegar-stained image are both biological tissue images, and P is an integer greater than or equal to 1;

[0010] According to the image features of the physiological saline image and the image features of each vinegar-stained image, P inter-class feature distances and P intra-class feature distances are determined, wherein the P inter-class feature distances include the inter-class feature distance between each vinegar-stained image and the physiological saline image, and the P intra-class feature distances include the intra-class feature distance corresponding to each vinegar-stained image;

[0011] According to the P inter-class feature distances and the P intra-class feature distances, the comprehensive feature distance corresponding to each vinegar-stained image is determined, wherein the comprehensive feature distance is used to characterize the difference significance of the vinegar-stained images;

[0012] Obtaining the maximum comprehensive feature distance from the comprehensive feature distances corresponding to each vinegar-stained image, and using the vinegar-stained image corresponding to the maximum comprehensive feature distance as the target vinegar-stained image;

[0013] The target detection model is called to detect the target aceto-stained image to obtain an image detection result of the target aceto-stained image, wherein the image detection result is used to determine a biopsy area in the target aceto-stained image.

[0014] Another aspect of the present application provides a biological tissue image detection device, comprising:

[0015] An acquisition module, used for acquiring a first image set to be identified, wherein the first image set includes at least two biological tissue images;

[0016] an identification module, used to call the first image classification model to identify each biological tissue image in the first image set, and obtain a staining classification result of each biological tissue image and an image feature of each biological tissue image, wherein the staining classification result is used to indicate the staining type to which the biological tissue image belongs;

[0017] The acquisition module is further used to acquire a physiological saline image and P vinegar-stained images from the first image set according to the staining classification result of each biological tissue image, wherein the physiological saline image and the vinegar-stained image are both biological tissue images, and P is an integer greater than or equal to 1;

[0018] A determination module, used to determine P inter-class feature distances and P intra-class feature distances according to the image features of the physiological saline image and the image features of each vinegar-stained image, wherein the P inter-class feature distances include the inter-class feature distance between each vinegar-stained image and the physiological saline image, and the P intra-class feature distances include the intra-class feature distance corresponding to each vinegar-stained image;

[0019] The determination module is further used to determine the comprehensive feature distance corresponding to each vinegar-stained image according to the P inter-class feature distances and the P intra-class feature distances, wherein the comprehensive feature distance is used to characterize the difference significance of the vinegar-stained images;

[0020] The determination module is further used to obtain the maximum comprehensive feature distance from the comprehensive feature distances corresponding to each vinegar-stained image, and use the vinegar-stained image corresponding to the maximum comprehensive feature distance as the target vinegar-stained image;

[0021] The detection module is used to call the target detection model to detect the target vinegar-stained image to obtain an image detection result of the target vinegar-stained image, wherein the image detection result is used to determine the biopsy area in the target vinegar-stained image.

[0022] In one possible design, in another implementation of another aspect of the embodiment of the present application,

[0023] an acquisition module, specifically used for a second image set to be identified, wherein the second image set includes at least two biological tissue images;

[0024] Calling the second image classification model to identify each biological tissue image in the second image set, and obtaining an occlusion classification result of each biological tissue image, wherein the occlusion classification result is used to indicate an occlusion condition of the biological tissue image;

[0025] According to the occlusion classification result of each biological tissue image, the biological tissue images whose occlusion classification results belong to the no-occlusion category in the second image set are used as the biological tissue images in the first image set.

[0026] In one possible design, in another implementation of another aspect of the embodiment of the present application, the biological tissue image detection device further includes a training module;

[0027] The acquisition module is further used to acquire a first image sample set, wherein the first image sample set includes at least one first image sample, and each first image sample corresponds to an occlusion classification label;

[0028] The acquisition module is further used to acquire, for each first image sample in the first image sample set, an occlusion probability distribution corresponding to each first image sample based on the second image classification model;

[0029] The acquisition module is further used to calculate the first loss value based on the occlusion probability distribution and the occlusion classification label corresponding to each first image sample;

[0030] A training module is used to update the model parameters of the second image classification model using the first loss value.

[0031] In one possible design, in another implementation of another aspect of the embodiment of the present application,

[0032] The recognition module is specifically used to obtain the image features of each biological tissue image in the first image set through the backbone network included in the first image classification model;

[0033] For each biological tissue image, a pooling feature of each biological tissue image is obtained through a pooling layer included in the first image classification model;

[0034] For each biological tissue image, a pooled feature is used to obtain a staining probability distribution of each biological tissue image through a fully connected network included in the first image classification model;

[0035] According to the staining probability distribution of each biological tissue image, the staining classification result of each biological tissue image is determined.

[0036] In one possible design, in another implementation of another aspect of the embodiment of the present application,

[0037] The recognition module is specifically used to compare the probability values ​​included in the staining probability distribution of each biological tissue image to obtain the maximum probability value in the staining probability distribution of each biological tissue image;

[0038] For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the first element feature position, the staining classification result of the biological tissue image is determined as a non-staining category;

[0039] For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the second element feature position, the staining classification result of the biological tissue image is determined as a physiological saline category, wherein the biological tissue image belonging to the physiological saline category is a physiological saline image;

[0040] For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the third element feature position, the staining classification result of the biological tissue image is determined as the vinegar staining category, wherein the biological tissue image belonging to the vinegar staining category is a vinegar staining image.

[0041] In one possible design, in another implementation of another aspect of the embodiment of the present application,

[0042] The recognition module is specifically used to compare the probability values ​​included in the staining probability distribution of each biological tissue image to obtain the maximum probability value in the staining probability distribution of each biological tissue image;

[0043] For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the first element feature position, the staining classification result of the biological tissue image is determined as a non-staining category;

[0044] For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the second element feature position, the staining classification result of the biological tissue image is determined as a physiological saline category, wherein the biological tissue image belonging to the physiological saline category is a physiological saline image;

[0045] For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the third element feature position, the staining classification result of the biological tissue image is determined as a vinegar staining category, wherein the biological tissue image belonging to the vinegar staining category is a vinegar staining image;

[0046] For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the fourth element characteristic position, the staining classification result of the biological tissue image is determined as the iodine staining category, wherein the biological tissue image belonging to the iodine staining category is an iodine staining image.

[0047] In one possible design, in another implementation of another aspect of the embodiment of the present application,

[0048] The acquisition module is further used to acquire a second image sample set, wherein the second image sample set includes at least one second image sample, and each second image sample corresponds to a staining classification label;

[0049] The acquisition module is further used to acquire, for each second image sample in the second image sample set, a staining probability distribution corresponding to each second image sample based on the first image classification model;

[0050] The acquisition module is further used to calculate a second loss value based on the staining probability distribution and the staining classification label corresponding to each second image sample;

[0051] The training module is also used to update the model parameters of the first image classification model using the second loss value.

[0052] In one possible design, in another implementation of another aspect of the embodiment of the present application,

[0053] A determination module is specifically used to calculate the inter-class distance of the image features of the physiological saline image and the image features of each vinegar-stained image, so as to obtain the inter-class feature distance between each vinegar-stained image and the physiological saline image;

[0054] Performing intra-class distance calculation on the image features of each vinegar-stained image and the corresponding image features of other vinegar-stained images to obtain the intra-class feature distance corresponding to each vinegar-stained image, wherein the other vinegar-stained images are the remaining vinegar-stained images except the vinegar-stained image in the P vinegar-stained images;

[0055] The determination module is specifically used to perform weighted summation of the inter-class feature distance and the intra-class feature distance corresponding to each vinegar-stained image to obtain the comprehensive feature distance corresponding to the vinegar-stained image.

[0056] In one possible design, in another implementation of another aspect of the embodiment of the present application,

[0057] The determination module is specifically used to calculate the element distance between each element feature in the image feature corresponding to the vinegar-stained image and the element feature at the corresponding position in the image feature corresponding to the physiological saline image for each vinegar-stained image, so as to obtain M element feature distances between the vinegar-stained image and the physiological saline image, where M is an integer greater than or equal to 1;

[0058] For each vinegar-stained image, image distance calculation is performed on the M element feature distances between the vinegar-stained image and the physiological saline image to obtain the inter-class feature distance between the vinegar-stained image and the physiological saline image.

[0059] In one possible design, in another implementation of another aspect of the embodiment of the present application,

[0060] The determination module is specifically used to calculate the element distance between each element feature in the image feature corresponding to the vinegar-stained image and the element features at the corresponding position in the image features corresponding to other vinegar-stained images for each vinegar-stained image, so as to obtain M element feature distances between each vinegar-stained image and other vinegar-stained images, wherein M is an integer greater than or equal to 1;

[0061] For each vinegar-stained image, performing image distance calculation on M element feature distances between the vinegar-stained image and other vinegar-stained images to obtain a first feature distance between the vinegar-stained image and other vinegar-stained images;

[0062] For each vinegar-stained image, the first characteristic distances between the vinegar-stained image and other vinegar-stained images are averaged to obtain the intra-class characteristic distance corresponding to the vinegar-stained image.

[0063] In one possible design, in another implementation of another aspect of the embodiment of the present application,

[0064] A determination module, specifically used to calculate the feature distance between the image features of the physiological saline image and the image features of each vinegar-stained image, to obtain the image feature distance between each vinegar-stained image and the physiological saline image;

[0065] Performing feature distance calculation on the histogram features of the physiological saline image and the histogram features of each vinegar-stained image to obtain the histogram feature distance between each vinegar-stained image and the physiological saline image;

[0066] For each vinegar-stained image, the image feature distance between the vinegar-stained image and the saline image, and the histogram feature distance between the vinegar-stained image and the saline image are weighted summed to obtain the inter-class feature distance between the vinegar-stained image and the saline image.

[0067] In one possible design, in another implementation of another aspect of the embodiment of the present application,

[0068] The determination module is specifically used to calculate the element distance between each element feature in the image feature corresponding to the vinegar-stained image and the element feature at the corresponding position in the image feature corresponding to the physiological saline image for each vinegar-stained image, so as to obtain M element feature distances between the vinegar-stained image and the physiological saline image, where M is an integer greater than or equal to 1;

[0069] For each vinegar-stained image, the image distance is calculated for the M element feature distances between the vinegar-stained image and the physiological saline image to obtain the image feature distance between the vinegar-stained image and the physiological saline image;

[0070] The determination module is specifically used to calculate the element distance between each element feature in the histogram feature corresponding to the vinegar-stained image and the element feature at the corresponding position in the histogram feature corresponding to the physiological saline image for each vinegar-stained image, so as to obtain N element feature distances between the vinegar-stained image and the physiological saline image, where N is an integer greater than or equal to 1;

[0071] For each vinegar-stained image, a histogram distance calculation is performed on N element feature distances between the vinegar-stained image and the physiological saline image to obtain a histogram feature distance between the vinegar-stained image and the physiological saline image.

[0072] In one possible design, in another implementation of another aspect of the embodiment of the present application,

[0073] A determination module, specifically used to calculate the average distance of the image features corresponding to each vinegar-stained image according to the image features of each vinegar-stained image;

[0074] According to the histogram features of each vinegar-stained image, the average distance of the histogram features corresponding to each vinegar-stained image is calculated;

[0075] For each vinegar-stained image, a weighted sum is performed on the average distance of image features corresponding to the vinegar-stained image and the average distance of histogram features corresponding to the vinegar-stained image to obtain the intra-class feature distance corresponding to the vinegar-stained image.

[0076] In one possible design, in another implementation of another aspect of the embodiment of the present application,

[0077] The determination module is specifically used to calculate the element distance between each element feature in the image feature corresponding to the vinegar-stained image and the element features at the corresponding position in the image features corresponding to other vinegar-stained images for each vinegar-stained image, so as to obtain M element feature distances between each vinegar-stained image and other vinegar-stained images, wherein M is an integer greater than or equal to 1;

[0078] For each vinegar-stained image, performing image distance calculation on M element feature distances between the vinegar-stained image and other vinegar-stained images to obtain a first feature distance between the vinegar-stained image and other vinegar-stained images;

[0079] For each vinegar-stained image, averaging the first characteristic distances between the vinegar-stained image and other vinegar-stained images to obtain an average image characteristic distance corresponding to the vinegar-stained image;

[0080] The determination module is specifically used to calculate the element distance between each element feature in the histogram feature corresponding to the vinegar-stained image and the element features at the corresponding position in the histogram features corresponding to other vinegar-stained images for each vinegar-stained image, so as to obtain N element feature distances between each vinegar-stained image and other vinegar-stained images, wherein N is an integer greater than or equal to 1;

[0081] For each vinegar-stained image, a histogram distance calculation is performed on N element characteristic distances between the vinegar-stained image and other vinegar-stained images to obtain a second characteristic distance between the vinegar-stained image and other vinegar-stained images;

[0082] For each vinegar-stained image, the second characteristic distances between the vinegar-stained image and other vinegar-stained images are averaged to obtain the histogram characteristic average distance corresponding to the vinegar-stained image.

[0083] In one possible design, in another implementation of another aspect of the embodiment of the present application, the biological tissue image detection device further includes a processing module;

[0084] A processing module, used for graying the physiological saline image to obtain a physiological saline grayscale image, wherein each pixel in the physiological saline grayscale image corresponds to a brightness value;

[0085] The processing module is further used to count the number of pixels corresponding to each preset brightness value based on the physiological saline grayscale image corresponding to the physiological saline image;

[0086] The processing module is further used to construct a histogram feature of the physiological saline image according to the number of pixels of the physiological saline grayscale image at each preset brightness value;

[0087] The processing module is further used to perform grayscale processing on each vinegar-dyed image to obtain a vinegar-dyed grayscale image, wherein each pixel in the vinegar-dyed grayscale image corresponds to a brightness value;

[0088] The processing module is further used to count the number of pixels corresponding to each preset brightness value for each vinegar-dyed image based on the vinegar-dyed grayscale image;

[0089] The processing module is also used to construct a histogram feature of each vinegar-dyed image according to the number of pixels of the vinegar-dyed grayscale image at each preset brightness value.

[0090] In one possible design, in another implementation of another aspect of the embodiment of the present application, the biological tissue image detection device further includes a display module;

[0091] A display module is used to call the target detection model to detect the target vinegar-stained image, and after obtaining the image detection result of the target vinegar-stained image, if the image detection result is used to indicate that the target vinegar-stained image does not have a biopsy area, a failure prompt message is displayed through the operation interface, or the image detection result is sent to the terminal so that the terminal displays a failure prompt message;

[0092] The display module is also used to display at least one biopsy area included in the target aceto-stained image through an operation interface if the image detection result is used to indicate that a biopsy area exists in the target aceto-stained image, or to send the image detection result to the terminal so that the terminal displays at least one biopsy area included in the target aceto-stained image.

[0093] Another aspect of the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned methods when executing the computer program.

[0094] Another aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the above-mentioned methods are implemented.

[0095] Another aspect of the present application provides a computer program product, including a computer program, which implements the above-mentioned methods when executed by a processor.

[0096] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0097] In an embodiment of the present application, a method for detecting biological tissue images is provided. First, a first image set to be identified is obtained, and then a first image classification model can be called to identify each biological tissue image to obtain a staining classification result of each biological tissue image and an image feature of each biological tissue image. Then, a physiological saline image and P vinegar-stained images are obtained from the first image set, and then P inter-class feature distances and P intra-class feature distances are determined. Based on this, the comprehensive feature distance corresponding to each vinegar-stained image can be determined according to the P inter-class feature distances and the P intra-class feature distances. Then, the maximum comprehensive feature distance is obtained from the comprehensive feature distances corresponding to each vinegar-stained image, and the vinegar-stained image corresponding to the maximum comprehensive feature distance is used as the target vinegar-stained image. Finally, the target detection model is called to detect the target vinegar-stained image to obtain the image detection result of the target vinegar-stained image. Through the above method, the intra-class feature distance of the vinegar-stained image and the inter-class feature distance between the vinegar-stained image and the saline image are introduced to jointly measure the difference significance of the vinegar-stained images. Therefore, the vinegar-stained image with the largest difference can be selected as the vinegar-stained image at the time of the maximum disease reaction, and the vinegar-stained image is used as the target vinegar-stained image for predicting the biopsy area. The above method fully considers the differences in the reaction time of the lesion area between different patients, which is conducive to improving the accuracy and effectiveness of the biopsy area prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] Figure 1 A schematic diagram of an implementation environment of the biological tissue image detection method in the embodiment of the present application;

[0099] Figure 2 This is another schematic diagram of an implementation environment of the biological tissue image detection method in an embodiment of the present application;

[0100] Figure 3 A schematic diagram of a process of a biological tissue image detection method in an embodiment of the present application;

[0101] Figure 4 A schematic diagram of the structure of the biological tissue image detection method in the embodiment of the present application;

[0102] Figure 5 Another schematic diagram of the structure of the biological tissue image detection method in the embodiment of the present application;

[0103] Figure 6 A schematic diagram of the architecture of the first image classification model in an embodiment of the present application;

[0104] Figure 7 Another schematic diagram of the architecture of the first image classification model in the embodiment of the present application;

[0105] Figure 8Another schematic diagram of the structure of the biological tissue image detection method in the embodiment of the present application;

[0106] Fig. 9 This is a schematic diagram of an operation interface for displaying a detection report based on image detection results in an embodiment of the present application;

[0107] Fig.10 This is a schematic diagram of an overall process of the biological tissue image detection method in the embodiment of the present application;

[0108] Fig.11 A schematic diagram of a biological tissue image detection device in an embodiment of the present application;

[0109] Fig.12 A schematic diagram of the structure of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0110] The embodiments of the present application provide a biological tissue image detection method, apparatus, device and storage medium, which can select the acetic acid-stained image with the largest difference as the acetic acid-stained image at the moment of maximum disease reaction. Thus, the differences in the reaction time of the lesion area between different patients are fully taken into account, which is conducive to improving the accuracy and effectiveness of biopsy area prediction.

[0111] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "corresponding to" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0112] Biopsy is the abbreviation of "live tissue examination", also known as surgical pathology examination. It refers to the technology of removing diseased tissue from the patient's body by cutting, clamping or puncturing for pathological examination in response to the needs of diagnosis and treatment. It is the most important part of diagnostic pathology. It can make a clear histopathological diagnosis for most cases submitted for examination and is used as the final clinical diagnosis. Biopsy is the standard for diagnosing cervical cancer, and colposcopy before biopsy is the key to determining whether a biopsy is needed and confirming the location of the biopsy. In the past, the judgment of colposcopy results mainly relied on the subjective experience of doctors. Once a misjudgment occurred, it would directly delay many patients from receiving timely diagnosis and treatment in the precancerous or early stages of cancer. The addition of artificial intelligence (AI) has brought a solution to electronic colposcopy. It can use AI technology to simulate the doctor's reading process, assist in the interpretation of precancerous lesions under colposcopy, assist doctors in quickly grading cervical cancer lesions and predicting biopsy points, and effectively improve the accuracy of colposcopy and biopsy diagnosis.

[0113] Among them, AI is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, AI is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. AI is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0114] AI technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, mechatronics, and other technologies. AI software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0115] Image recognition based on AI technology also involves computer vision (CV) technology and machine learning (ML) technology. Among them, CV is a science that studies how to make machines "see". To put it more specifically, it refers to machine vision such as using cameras and computers to replace human eyes to identify and measure targets, and further processing graphics to make computer processing into images that are more suitable for human eye observation or transmission to instruments for detection. As a scientific discipline, CV studies related theories and technologies and attempts to establish an AI system that can obtain information from images or multidimensional data. CV technology usually includes image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and map construction, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0116] ML is a multi-disciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. ML is the core of AI and the fundamental way to make computers intelligent. Its applications are spread across all areas of AI. ML and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0117] At present, the automatic diagnosis of cervical precancerous lesions based on colposcopy is still in its early stages. Based on this, the present application provides a detection method for biological tissue images, which can be applied to the prediction of biopsy areas of colposcopy. In addition, it can also be applied to the prediction of biopsy areas of other biological tissues, such as the prediction of biopsy areas of the liver, the heart, and the throat. The present application uses a fusion feature comparison method to select the vinegar-stained image with the most prominent lesions (i.e., the target vinegar-stained image) as the input for biopsy area prediction. The method provided in the present application can be applied to Figure 1 or Figure 2 The implementation environment shown, in which Figure 1 The implementation environment shown is suitable for background recognition, and the business logic of the background recognition is to transmit the biological tissue image to the server side for prediction of the biopsy area. Figure 2 The implementation environment shown is suitable for front-end recognition. The business logic of front-end recognition is to deploy the capabilities on the terminal side, and the biological tissue image completes the prediction of the biopsy area locally.

[0118] Figure 1The implementation environment shown includes a terminal 110 and a server 120, and the terminal 110 and the server 120 are connected via a communication network 130. The terminal 110 and the server 120 can communicate with each other via the communication network 130. Figure 2 The illustrated implementation environment includes a terminal 110 .

[0119] The communication network 130 involved in the present application uses standard communication technology and / or protocols, usually the Internet, but can also be any network, including but not limited to Bluetooth, local area network (LAN), metropolitan area network (MAN), wide area network (WAN), mobile, dedicated network or any combination of virtual private networks). In some embodiments, customized or dedicated data communication technology can be used to replace or supplement the above data communication technology.

[0120] The terminal 110 involved in this application includes but is not limited to medical detection equipment (for example, colposcope or endoscope), mobile phone, computer, intelligent voice interaction equipment, smart home appliances, vehicle-mounted terminal, aircraft, etc. Among them, the client is deployed on the terminal 110, and the client can be run on the terminal 110 in the form of a browser, or in the form of an independent APP. The server 120 involved in this application can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and basic cloud computing services such as big data and AI platforms.

[0121] The deep learning model 140 involved in the present application includes a first image classification model and a target detection model, and optionally, may also include a second image classification model. It should be noted that the present application takes the configuration of the deep learning model 140 deployed on the server 120 as an example for explanation. In some embodiments, the configuration of the deep learning model 140 may also be deployed on the terminal 110. In some embodiments, part of the configuration of the deep learning model 140 is deployed on the terminal 110, and part of the configuration is deployed on the server 120. This is not limited here.

[0122] Combined with the above Figure 1In the implementation environment shown, the terminal 110 sends the collected biological tissue image to the server 120 through the communication network 130. The server 120 calls the first image classification model in the deep learning model 140 to identify and analyze the collected biological tissue image, and selects the vinegar-stained image with the most prominent lesion as the target vinegar-stained image. Based on this, the server 120 calls the target detection model in the deep learning model 140 to perform target detection on the target vinegar-stained image to obtain an image detection result of the target vinegar-stained image.

[0123] Combined with the above Figure 2 In the implementation environment shown, the terminal 110 collects biological tissue images, and then calls the first image classification model in the deep learning model 140 to identify and analyze the collected biological tissue images, and selects the vinegar-stained image with the most prominent lesions as the target vinegar-stained image. Based on this, the terminal 110 calls the target detection model in the deep learning model 140 to perform target detection on the target vinegar-stained image to obtain an image detection result of the target vinegar-stained image.

[0124] Since this application involves some terms related to professional fields, they will be explained below for ease of understanding.

[0125] (1) Colposcope: It is a gynecological clinical diagnostic instrument and one of the gynecological endoscopes. It is suitable for the diagnosis of various cervical diseases and genital lesions, and is also an important method for early diagnosis of male and female sexual diseases.

[0126] (2) Endoscope: It is a detection instrument that integrates traditional optics, ergonomics, precision machinery, modern electronics, mathematics, and software. It has an image sensor, optical lens, light source, mechanical device, etc. It can enter the stomach through the mouth or other natural channels into the body.

[0127] (3) Cervical biopsy: Cervical biopsy is a living tissue examination of the cervix, which means taking a small piece or several pieces of tissue from the cervix for pathological examination to confirm the diagnosis.

[0128] (4) Sensitivity: The proportion of images detected as precancerous lesions among all images of precancerous lesions.

[0129] (5) Specificity: The proportion of images detected as normal among all normal images.

[0130] (6) Deep learning technology: a technology that uses deep neural network systems to perform machine learning.

[0131] (7) Image category: The category to which the image content belongs, for example, normal, precancerous lesions, etc.

[0132] (8) Target detection network (or, target detection model): A mathematical model obtained by learning labeled sample data (i.e., the correspondence between an image and the target position in the image) using machine learning technology. The parameters of the mathematical model are obtained during the learning and training process. During detection, the parameters of the mathematical model are loaded and the detection position of the target in the input sample is calculated.

[0133] (9) Image classification network (or, image classification model): A mathematical model obtained by learning labeled sample data (i.e., the correspondence between images and categories) using machine learning technology. The parameters of the mathematical model are obtained during the learning and training process. The parameters of the mathematical model are loaded during recognition and prediction, and the probability of the input sample belonging to each category is calculated.

[0134] In combination with the above introduction, the detection method of biological tissue images in this application will be introduced below. Figure 3 The biological tissue image detection method in the embodiment of the present application can be completed independently by the server, can be completed independently by the terminal, or can be completed by the terminal and the server in cooperation. The present application includes:

[0135] 210. Acquire a first image set to be identified, wherein the first image set includes at least two biological tissue images;

[0136] In one or more embodiments, a first image set is collected by a medical detection device (e.g., a colposcope or an endoscope), and the first image set includes at least two biological tissue images. The biological tissue image may be a colposcope image, or an image of other living tissue, such as a laryngoscope image, a colonoscope image, etc. This application is described by taking a colposcope image as an example, however, this should not be understood as a limitation of this application.

[0137] 220. Calling the first image classification model to identify each biological tissue image in the first image set, and obtaining a staining classification result of each biological tissue image and an image feature of each biological tissue image, wherein the staining classification result is used to indicate a staining type to which the biological tissue image belongs;

[0138] In one or more embodiments, each biological tissue image in the first image set is used as the input of the first image classification model, and the staining probability distribution of each biological tissue image is output by the first image classification model, and then the staining classification result of the biological tissue image can be determined based on the staining probability distribution. At the same time, based on the backbone network of the first image classification model, the image features of each biological tissue image can be obtained. It can be understood that the first image classification model can adopt convolutional neural networks (CNN), and the image features are CNN features, wherein the image features are represented in the form of feature vectors.

[0139] 230. According to the staining classification result of each biological tissue image, obtain a physiological saline image and P vinegar-stained images from the first image set, wherein the physiological saline image and the vinegar-stained image are both biological tissue images, and P is an integer greater than or equal to 1;

[0140] In one or more embodiments, in combination with the staining classification result of each biological tissue image, biological tissue images belonging to saline images and biological tissue images belonging to aceto-stained images can be screened out from the first image set. It is understandable that if the first image set includes multiple saline images, one of the saline images can be selected for subsequent processing. In addition, the first image set usually includes multiple aceto-stained images, and therefore, it is necessary to select a target aceto-stained image for predicting the biopsy area from the multiple aceto-stained images.

[0141] 240. Determine P inter-class feature distances and P intra-class feature distances according to the image features of the physiological saline image and the image features of each vinegar-stained image, wherein the P inter-class feature distances include the inter-class feature distance between each vinegar-stained image and the physiological saline image, and the P intra-class feature distances include the intra-class feature distance corresponding to each vinegar-stained image;

[0142] In one or more embodiments, after obtaining the physiological saline image and P vinegar-stained images, the image features of the physiological saline image and the image features of each vinegar-stained image are used to determine P inter-class feature distances. The P inter-class feature distances include the inter-class feature distances between each vinegar-stained image and the physiological saline image. At the same time, the image features of each vinegar-stained image are used to determine P intra-class feature distances. The P intra-class feature distances include the intra-class feature distances between each vinegar-stained image and the remaining (P-1) vinegar-stained images.

[0143] 250. Determine a comprehensive feature distance corresponding to each vinegar-dyed image according to the P inter-class feature distances and the P intra-class feature distances, wherein the comprehensive feature distance is used to characterize the difference significance of the vinegar-dyed images;

[0144] In one or more embodiments, it can be seen that for each of the P vinegar-stained images, there is an inter-class feature distance and an intra-class feature distance. Therefore, for each vinegar-stained image, the comprehensive feature distance is calculated according to the corresponding inter-class feature distance and intra-class feature distance. Thus, the comprehensive feature distance corresponding to each vinegar-stained image is obtained. Among them, the larger the comprehensive feature distance of the vinegar-stained image, the more significant the difference degree of the vinegar-stained image.

[0145] 260. Obtain a maximum comprehensive feature distance from the comprehensive feature distances corresponding to each vinegar-stained image, and use the vinegar-stained image corresponding to the maximum comprehensive feature distance as a target vinegar-stained image;

[0146] In one or more embodiments, the comprehensive feature distance is positively correlated with the difference significance of the vinegar-stained image. Therefore, the maximum comprehensive feature distance is first determined from the P comprehensive feature distances, and then the vinegar-stained image corresponding to the maximum comprehensive feature distance is used as the target vinegar-stained image.

[0147] 270. Call the target detection model to detect the target vinegar-stained image to obtain an image detection result of the target vinegar-stained image, wherein the image detection result is used to determine the biopsy area in the target vinegar-stained image.

[0148] In one or more embodiments, the target vinegar-stained image is used as the input of the target detection model, and the target detection model outputs the image detection result of the target vinegar-stained image, wherein the image detection result may include the center position (or, corner point position), length and width of the biopsy area.

[0149] The target detection model can use a single-stage detector (you only look once, YOLO), for example, YOLO V3. It should be noted that the target detection model can also use a region-based fully convolutional network (R-FCN), a fast region-based fully convolutional network (Faster R-CNN), a feature pyramid network (FPN) or a single shot multibox detector (SSD), etc., which are not limited here.

[0150] Specifically, for ease of understanding, see Figure 4 , Figure 4This is a schematic diagram of the structure of the biological tissue image detection method in the embodiment of the present application. As shown in the figure, taking the first image set including biological tissue image 1, biological tissue image 2 and biological tissue image 3 as an example, after the first image classification model, it can be determined that biological tissue image 1 is a physiological saline image, and biological tissue image 2 and biological tissue image 3 are both vinegar-stained images. At the same time, the image features of each biological tissue image can also be obtained through the backbone network of the first image classification model, that is, image feature F0, image feature F1 and image feature F2. Based on this, feature distance calculation is performed using image feature F0, image feature F1 and image feature F2 to obtain the comprehensive feature distance of vinegar-stained image 1 and the comprehensive feature distance of vinegar-stained image 2. Thus, the vinegar-stained image with a larger comprehensive feature distance is selected from vinegar-stained image 1 (i.e., biological tissue image 2) and vinegar-stained image 2 (i.e., biological tissue image 3) as the target vinegar-stained image. Finally, the target detection model is called to detect the target vinegar-stained image to obtain the prediction result of the biopsy area.

[0151] In an embodiment of the present application, a method for detecting biological tissue images is provided. Through the above method, the intra-class feature distance of the vinegar-stained image and the inter-class feature distance between the vinegar-stained image and the physiological saline image are introduced to jointly measure the difference significance of the vinegar-stained images. Thus, the vinegar-stained image with the largest difference can be selected as the vinegar-stained image at the time of the maximum disease reaction, and the vinegar-stained image is used as the target vinegar-stained image for predicting the biopsy area. The above method fully takes into account the differences in the reaction time of the lesion area between different patients, which is conducive to improving the accuracy and effectiveness of the biopsy area prediction.

[0152] Optionally, in the above Figure 3 On the basis of the corresponding embodiments, in another optional embodiment provided by the embodiment of the present application, obtaining the first image set to be identified may specifically include:

[0153] a second image set to be identified, wherein the second image set includes at least two biological tissue images;

[0154] Calling the second image classification model to identify each biological tissue image in the second image set, and obtaining an occlusion classification result of each biological tissue image, wherein the occlusion classification result is used to indicate an occlusion condition of the biological tissue image;

[0155] According to the occlusion classification result of each biological tissue image, the biological tissue images whose occlusion classification results belong to the no-occlusion category in the second image set are used as the biological tissue images in the first image set.

[0156] In one or more embodiments, a method for automatically identifying occluded images is introduced. As can be seen from the aforementioned embodiments, the biological tissue images included in the first image set are usually images that are not occluded by objects (e.g., cotton swabs or instruments, etc.). However, in actual situations, biological tissue images that are occluded by objects may be collected. Therefore, it is also necessary to screen out biological tissue images used to constitute the first image set from these images.

[0157] Specifically, for ease of understanding, see Figure 5 , Figure 5 This is another schematic diagram of the architecture of the biological tissue image detection method in the embodiment of the present application. As shown in the figure, taking the second image set including biological tissue image 1, biological tissue image 2, biological tissue image 3, biological tissue image 4, biological tissue image 5 and biological tissue image 6 as an example, these biological tissue images are used as the input of the second image classification model, and the occlusion probability distribution of each biological tissue image is obtained through the second image classification model, and then the occlusion classification result of the biological tissue image can be determined based on the occlusion probability distribution. The occlusion classification result can be an "occlusion category" or a "non-occlusion category". Among them, biological tissue image 1, biological tissue image 3 and biological tissue image 4 all belong to the "occlusion category", so these three biological tissue images can be eliminated. And biological tissue images 5, biological tissue image 2 and biological tissue image 6 belonging to the "non-occlusion category" are taken as the first image set.

[0158] Based on this, biological tissue image 5, biological tissue image 2 and biological tissue image 6 are used as inputs of the first image classification model, and the staining probability distribution of each biological tissue image is obtained through the first image classification model, and then the staining classification result of the biological tissue image can be determined based on the staining probability distribution. Among them, biological tissue image 5 is a physiological saline image, and biological tissue image 2 and biological tissue image 6 are both vinegar-stained images. At the same time, the image features of each biological tissue image, that is, image feature F0, image feature F1 and image feature F2, can also be obtained through the backbone network of the first image classification model. Feature distance calculation is performed using image feature F0, image feature F1 and image feature F2 to obtain the comprehensive feature distance of vinegar-stained image 1 (that is, biological tissue image 2) and the comprehensive feature distance of vinegar-stained image 2 (that is, biological tissue image 6). Therefore, the vinegar-stained image with a larger comprehensive feature distance is selected from vinegar-stained image 1 and vinegar-stained image 2 as the target vinegar-stained image. Finally, the target detection model is called to detect the target vinegar-stained image to obtain the prediction result of the biopsy area.

[0159] Secondly, in the embodiment of the present application, a method for automatically identifying occluded images is provided. Through the above method, occluded images that do not meet the biopsy area prediction input can be automatically excluded, avoiding using biological tissue images occluded by cotton swabs or other instruments as input, thereby improving the accuracy of subsequent detection.

[0160] Optionally, in the above Figure 3 On the basis of the corresponding embodiments, another optional embodiment provided by the embodiment of the present application may further include:

[0161] Acquire a first image sample set, wherein the first image sample set includes at least one first image sample, and each first image sample corresponds to an occlusion classification label;

[0162] For each first image sample in the first image sample set, obtaining an occlusion probability distribution corresponding to each first image sample based on the second image classification model;

[0163] Calculate a first loss value based on the occlusion probability distribution and the occlusion classification label corresponding to each first image sample;

[0164] The first loss value is used to update the model parameters of the second image classification model.

[0165] In one or more embodiments, a method for training a second image classification model is introduced. It can be seen from the aforementioned embodiments that a first image sample set needs to be constructed so as to train the second image classification model using the first image sample set. The first image sample set includes at least one first image sample, and each first image sample is annotated with a corresponding occlusion classification label. For example, the occlusion classification label of the first image sample 1 is "occlusion category", and the occlusion classification label of the first image sample 2 is "no occlusion category".

[0166] Specifically, the first image sample set can be regarded as a batch data set. Based on this, each first image sample in the first image sample set is used as the input of the second image classification model, and the occlusion probability distribution corresponding to each first image sample is output through the second image classification model. At the same time, according to the occlusion classification label corresponding to each first image sample, the true probability distribution corresponding to each first image sample can be obtained. For example, if the occlusion classification label is "occlusion category", the corresponding true probability distribution is (1,0). For another example, if the occlusion classification label is "no occlusion category", the corresponding true probability distribution is (0,1). The loss value of a first image sample can be calculated as follows:

[0167]

[0168] Where Loss1 represents the loss value of the first image sample. T represents the total number of occlusion classification labels. y represents the occlusion classification label. If it is the i-th occlusion classification label, then y i =1, otherwise it is equal to 0. i Represents the probability of belonging to the i-th occlusion classification label based on the occlusion probability distribution.

[0169] Thus, the loss value of each first image sample in the first image sample set is summed to obtain the first loss value. Then, the stochastic gradient descent (SGD) method can be used to perform gradient backward calculation on the first loss value and obtain the model parameter update value of the second image classification model.

[0170] It should be noted that by continuously collecting sample image samples and then increasing the training data set of the offline model, the recognition ability of the model can be enhanced.

[0171] Again, in the embodiment of the present application, a method for training a second image classification model is provided. Through the above method, a deep classification network (i.e., a second image classification model) is used to train a sample set with occlusion classification labels. In this way, the occlusion situation of the biological tissue images collected autonomously by the doctor can be identified, thereby achieving the purpose of automated detection.

[0172] Optionally, in the above Figure 3 On the basis of the corresponding embodiments, in another optional embodiment provided by the embodiment of the present application, calling the first image classification model to identify each biological tissue image in the first image set, and obtaining the staining classification result of each biological tissue image and the image features of each biological tissue image, may specifically include:

[0173] For each biological tissue image in the first image set, an image feature of each biological tissue image is acquired through a backbone network included in the first image classification model;

[0174] For each biological tissue image, a pooling feature of each biological tissue image is obtained through a pooling layer included in the first image classification model;

[0175] For each biological tissue image, a pooled feature is used to obtain a staining probability distribution of each biological tissue image through a fully connected network included in the first image classification model;

[0176] According to the staining probability distribution of each biological tissue image, the staining classification result of each biological tissue image is determined.

[0177] In one or more embodiments, a method for implementing feature processing based on a first image classification model is introduced. As can be seen from the foregoing embodiments, the first image classification model can not only output a staining probability distribution, but also output image features. The first image classification model will be introduced below as a residual network 50 (ResNet50) as an example. It can be understood that the first image classification model can also be ResNet18, ResNet34 or ResNet101, etc., which is not limited here.

[0178] Specifically, the first image classification model includes a backbone network, and image features are extracted based on the backbone network. For ease of understanding, please refer to Table 1, which is a schematic diagram of relevant parameters of each network layer in ResNet50.

[0179] Table 1

[0180]

[0181]

[0182] Wherein, the backbone network includes conv1, conv2_x, conv3_x, conv4_x and conv5_x. Each biological tissue image in the first image set is used as the input of the backbone network, and the image features of each biological tissue image are obtained through the backbone network. Based on this, the image features of each biological tissue image can be cached. Next, the image features of each biological tissue image are pooled, that is, the pooled features of each biological tissue image are obtained through the pooling layer. Then, the pooled features of each biological tissue image are used as the input of the fully connected network, and the staining probability distribution of each biological tissue image is obtained through the fully connected network. Thus, according to the staining probability distribution, the classification of the biological tissue image is realized, thereby obtaining the staining classification result of each biological tissue image. Wherein, the fully connected network includes a fully connected layer and a softmax layer.

[0183] Secondly, in the embodiment of the present application, a method for implementing feature processing based on the first image classification model is provided. Through the above method, the first image classification model can not only identify the staining classification results of the biological tissue image, but also extract the image features of the biological tissue image to learn and obtain the high-dimensional features of the biological tissue image. Thus, a corresponding basis is provided for subsequent feature processing, thereby improving the feasibility and operability of the solution.

[0184] Optionally, in the above Figure 3 On the basis of the corresponding embodiments, in another optional embodiment provided by the embodiment of the present application, determining the staining classification result of each biological tissue image according to the staining probability distribution of each biological tissue image may specifically include:

[0185] Comparing the probability values ​​included in the staining probability distribution of each biological tissue image to obtain the maximum probability value in the staining probability distribution of each biological tissue image;

[0186] For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the first element feature position, the staining classification result of the biological tissue image is determined as a non-staining category;

[0187] For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the second element feature position, the staining classification result of the biological tissue image is determined as a physiological saline category, wherein the biological tissue image belonging to the physiological saline category is a physiological saline image;

[0188] For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the third element feature position, the staining classification result of the biological tissue image is determined as the vinegar staining category, wherein the biological tissue image belonging to the vinegar staining category is a vinegar staining image.

[0189] In one or more embodiments, a method for determining a saline image and a vinegar-stained image is introduced. As can be seen from the above embodiments, the staining probability distribution of the biological tissue image can be output using a first image classification model. The first image classification model can identify a "non-stained category", a "saline category" and a "vinegar-stained category".

[0190] Specifically, for ease of understanding, see Figure 6 , Figure 6 A schematic diagram of the architecture of the first image classification model in the embodiment of the present application, as shown in the figure, each biological tissue image is used as the input of the backbone network in the first image classification model, and the corresponding image features are extracted by the backbone network. Then, the pooling features of each biological tissue image are obtained by the pooling layer. The staining probability distribution of each biological tissue image is obtained by the fully connected layer and the softmax layer. Wherein, the staining probability distribution can be expressed as a 3-dimensional feature vector, assuming that the first element feature position corresponds to the "no staining category" in the staining probability distribution, the second element feature position corresponds to the "normal saline category", and the third element feature position corresponds to the "vinegar staining category".

[0191] For example, it is assumed that the staining probability distribution of the biological tissue image is (0.8, 0.1, 0.1), wherein the maximum probability value is “0.8”. It can be seen that the maximum probability value corresponds to the first element feature position, and therefore, the staining classification result of the biological tissue image is determined as the non-staining category.

[0192] For example, it is assumed that the staining probability distribution of the biological tissue image is (0.1, 0.7, 0.2), where the maximum probability value is "0.7". It can be seen that the maximum probability value corresponds to the second element feature position, and therefore, the staining classification result of the biological tissue image is determined to be the physiological saline category, that is, the biological tissue image belongs to the physiological saline image.

[0193] For example, it is assumed that the staining probability distribution of the biological tissue image is (0.2, 0.1, 0.7), where the maximum probability value is "0.7". It can be seen that the maximum probability value corresponds to the third element feature position, and therefore, the staining classification result of the biological tissue image is determined to be the vinegar staining category, that is, the biological tissue image belongs to the vinegar staining image.

[0194] It should be noted that when the biological tissue image is predicted by the first image classification model, the image features obtained by the backbone network will be cached. After determining the staining category to which the biological tissue image belongs, the image features of the physiological saline image and the image features of the vinegar-stained image can be extracted for subsequent calculation of the feature distance.

[0195] Again, in an embodiment of the present application, a method for determining a saline image and a vinegar-stained image is provided. In the above manner, based on the staining probability distribution output by the first image classification model, the staining category to which the biological tissue image belongs can be determined. Then, the saline image and the vinegar-stained image can be automatically distinguished without the need for relevant personnel to operate, thereby saving labor costs.

[0196] Optionally, in the above Figure 3 On the basis of the corresponding embodiments, in another optional embodiment provided by the embodiment of the present application, determining the staining classification result of each biological tissue image according to the staining probability distribution of each biological tissue image may specifically include:

[0197] Comparing the probability values ​​included in the staining probability distribution of each biological tissue image to obtain the maximum probability value in the staining probability distribution of each biological tissue image;

[0198] For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the first element feature position, the staining classification result of the biological tissue image is determined as a non-staining category;

[0199] For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the second element feature position, the staining classification result of the biological tissue image is determined as a physiological saline category, wherein the biological tissue image belonging to the physiological saline category is a physiological saline image;

[0200] For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the third element feature position, the staining classification result of the biological tissue image is determined as a vinegar staining category, wherein the biological tissue image belonging to the vinegar staining category is a vinegar staining image;

[0201] For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the fourth element characteristic position, the staining classification result of the biological tissue image is determined as the iodine staining category, wherein the biological tissue image belonging to the iodine staining category is an iodine staining image.

[0202] In one or more embodiments, a method for determining a saline image, a vinegar-stained image, and an iodine-stained image is introduced. As can be seen from the above embodiments, the staining probability distribution of the biological tissue image can be output using the first image classification model. Among them, the first image classification model can identify a "non-staining category", a "saline category", a "vinegar-stained category", and an "iodine-stained category".

[0203] Specifically, for ease of understanding, see Figure 7 , Figure 7 Another schematic diagram of the architecture of the first image classification model in the embodiment of the present application, as shown in the figure, each biological tissue image is used as the input of the backbone network in the first image classification model, and the corresponding image features are extracted by the backbone network. Then, the pooling features of each biological tissue image are obtained by the pooling layer. The staining probability distribution of each biological tissue image is obtained by the fully connected layer and the softmax layer. Wherein, the staining probability distribution can be expressed as a 4-dimensional feature vector, assuming that the first element feature position corresponds to the "no staining category" in the staining probability distribution, the second element feature position corresponds to the "normal saline category", and the third element feature position corresponds to the "vinegar stain category". The fourth element feature position corresponds to the "iodine stain category".

[0204] For example, it is assumed that the staining probability distribution of the biological tissue image is (0.7, 0.1, 0.1, 0.1), where the maximum probability value is “0.7”. It can be seen that the maximum probability value corresponds to the first element feature position, and therefore, the staining classification result of the biological tissue image is determined as the non-staining category.

[0205] For example, it is assumed that the staining probability distribution of the biological tissue image is (0.1, 0.7, 0.1, 0.1), where the maximum probability value is "0.7". It can be seen that the maximum probability value corresponds to the second element feature position, and therefore, the staining classification result of the biological tissue image is determined to be the physiological saline category, that is, the biological tissue image belongs to the physiological saline image.

[0206] For example, it is assumed that the staining probability distribution of the biological tissue image is (0.2, 0.1, 0.6, 0.1), where the maximum probability value is "0.6". It can be seen that the maximum probability value corresponds to the third element feature position, and therefore, the staining classification result of the biological tissue image is determined to be the vinegar staining category, that is, the biological tissue image belongs to the vinegar staining image.

[0207] For example, it is assumed that the staining probability distribution of the biological tissue image is (0.2, 0.1, 0.1, 0.6), where the maximum probability value is "0.6". It can be seen that the maximum probability value corresponds to the fourth element feature position, and therefore, the staining classification result of the biological tissue image is determined to be the iodine staining category, that is, the biological tissue image belongs to the iodine staining image.

[0208] It should be noted that when the biological tissue image is predicted by the first image classification model, the image features obtained by the backbone network will be cached. After determining the staining category to which the biological tissue image belongs, the image features of the physiological saline image and the image features of the vinegar-stained image can be extracted for subsequent calculation of the feature distance.

[0209] It should be noted that when the biological tissue image is predicted by the first image classification model, the image features obtained by the backbone network will be cached. After determining the staining category to which the biological tissue image belongs, the image features of the physiological saline image and the image features of the vinegar-stained image can be extracted for subsequent calculation of the feature distance.

[0210] Again, in an embodiment of the present application, a method for determining a physiological saline image, a vinegar-stained image, and an iodine-stained image is provided. In the above manner, based on the staining probability distribution output by the first image classification model, the staining category to which the biological tissue image belongs can be determined. Then, the physiological saline image, the vinegar-stained image, and the iodine-stained image are automatically distinguished without the need for relevant personnel to operate, thereby saving labor costs.

[0211] Optionally, in the above Figure 3 On the basis of the corresponding embodiments, another optional embodiment provided by the embodiment of the present application may further include:

[0212] Acquire a second image sample set, wherein the second image sample set includes at least one second image sample, and each second image sample corresponds to a staining classification label;

[0213] For each second image sample in the second image sample set, obtaining a staining probability distribution corresponding to each second image sample based on the first image classification model;

[0214] Based on the staining probability distribution and the staining classification label corresponding to each second image sample, a second loss value is calculated;

[0215] The model parameters of the first image classification model are updated using the second loss value.

[0216] In one or more embodiments, a method for training a first image classification model is introduced. It can be seen from the aforementioned embodiments that a second image sample set needs to be constructed to train the first image classification model using the second image sample set. The second image sample set includes at least one second image sample, and each second image sample is annotated with a corresponding staining classification label. For example, the staining classification label of the second image sample 1 is "no staining category", and the staining classification label of the second image sample 2 is "normal saline category".

[0217] Specifically, the second image sample set can be used as a batch data set. Based on this, each second image sample in the second image sample set is used as the input of the first image classification model, and the staining probability distribution corresponding to each second image sample is output through the first image classification model. At the same time, according to the staining classification label corresponding to each second image sample, the true probability distribution corresponding to each second image sample can be obtained. For example, if the staining classification label is "unstained category", the corresponding true probability distribution is (1, 0, 0, 0). For another example, if the occlusion classification label is "normal saline category", the corresponding true probability distribution is (0, 1, 0, 0). For another example, if the occlusion classification label is "vinegar stain category", the corresponding true probability distribution is (0, 0, 1, 0). For another example, if the occlusion classification label is "iodine stain category", the corresponding true probability distribution is (0, 0, 0, 1). The loss value of a second image sample can be calculated as follows:

[0218]

[0219] Where Loss2 represents the loss value of the second image sample. K represents the total number of color classification labels. y′ i Indicates that based on the i-th coloring classification label, if it is the i-th coloring classification label, then y′ i =1, otherwise equal to 0. p′ i Represents the probability of belonging to the i-th coloring classification label based on the coloring probability distribution.

[0220] Thus, the loss value of each second image sample in the second image sample set is summed to obtain the second loss value. Then, SGD can be used to perform gradient backward calculation on the second loss value to obtain the model parameter update value of the first image classification model.

[0221] It should be noted that by continuously collecting sample image samples and then increasing the training data set of the offline model, the recognition ability of the model can be enhanced.

[0222] Secondly, in the embodiment of the present application, a method for training the first image classification model is provided. Through the above method, a deep classification network (i.e., the first image classification model) is used to train a sample set with marked staining classification labels. In this way, the staining conditions of biological tissue images collected autonomously by doctors can be identified to achieve the purpose of automated detection.

[0223] Optionally, in the above Figure 3 On the basis of the corresponding embodiments, in another optional embodiment provided by the embodiment of the present application, according to the image features of the physiological saline image and the image features of each vinegar-stained image, determining P inter-class feature distances and P intra-class feature distances may specifically include:

[0224] The inter-class distance is calculated for the image features of the physiological saline image and the image features of each vinegar-stained image, so as to obtain the inter-class feature distance between each vinegar-stained image and the physiological saline image;

[0225] Performing intra-class distance calculation on the image features of each vinegar-stained image and the corresponding image features of other vinegar-stained images to obtain the intra-class feature distance corresponding to each vinegar-stained image, wherein the other vinegar-stained images are the remaining vinegar-stained images except the vinegar-stained image in the P vinegar-stained images;

[0226] Determining the comprehensive feature distance corresponding to each vinegar-stained image according to the P inter-class feature distances and the P intra-class feature distances may specifically include:

[0227] For each vinegar-stained image, a weighted sum is performed on the inter-class feature distance and the intra-class feature distance corresponding to the vinegar-stained image to obtain a comprehensive feature distance corresponding to the vinegar-stained image.

[0228] In one or more embodiments, a method for determining the comprehensive feature distance corresponding to the vinegar-stained image is introduced. As can be seen from the above embodiments, after obtaining P vinegar-stained images, on the one hand, it is necessary to calculate the inter-class feature distance between each vinegar-stained image and the physiological saline image. On the other hand, it is necessary to calculate the intra-class feature distance between each vinegar-stained image and other vinegar-stained images.

[0229] Specifically, it is assumed that there are three vinegar-stained images (i.e., P is equal to 3), namely vinegar-stained image 1, vinegar-stained image 2, and vinegar-stained image 3. Based on this, the P inter-class feature distances include the inter-class feature distance 1 between vinegar-stained image 1 and the saline image, the inter-class feature distance 2 between vinegar-stained image 2 and the saline image, and the inter-class feature distance 3 between vinegar-stained image 3 and the saline image. The P intra-class feature distances include the intra-class feature distance 1 between vinegar-stained image 1 and the remaining two vinegar-stained images (i.e., vinegar-stained image 2 and vinegar-stained image 3), the intra-class feature distance 2 between vinegar-stained image 2 and the remaining two vinegar-stained images (i.e., vinegar-stained image 1 and vinegar-stained image 3), and the intra-class feature distance 3 between vinegar-stained image 3 and the remaining two vinegar-stained images (i.e., vinegar-stained image 1 and vinegar-stained image 2).

[0230] It can be seen that for each vinegar-stained image, its corresponding inter-class feature distance and intra-class feature distance can be obtained respectively. That is, vinegar-stained image 1 corresponds to inter-class feature distance 1 and intra-class feature distance 1. Vinegar-stained image 2 corresponds to inter-class feature distance 2 and intra-class feature distance 2. Vinegar-stained image 3 corresponds to inter-class feature distance 3 and intra-class feature distance 3. Therefore, the comprehensive feature distance corresponding to the vinegar-stained image can be calculated in the following way:

[0231] DisFinal i =γ·Dis i0 +δ·Dis i ;Formula (3)

[0232] Among them, DisFinal i Dis represents the comprehensive feature distance corresponding to the i-th vinegar-stained image. i0 Dis represents the inter-class feature distance between the i-th vinegar-stained image and the saline image. i Represents the intra-class feature distance corresponding to the i-th vinegar-stained image. Both γ and δ represent weighting coefficients, which usually need to be set manually, for example, γ = 0.6, δ = 0.4.

[0233] It can be understood that the comprehensive feature distance corresponding to each vinegar-stained image can be calculated using formula (3). Based on this, the vinegar-stained image with the largest comprehensive feature distance can be used as the target vinegar-stained image, and the target vinegar-stained image can be used as the input of the target detection network.

[0234] Secondly, in the embodiment of the present application, a method for determining the corresponding comprehensive feature distance of the vinegar-stained image is provided. In the above method, the inter-class feature distance and the intra-class feature distance of the vinegar-stained image are weighted and summed to obtain the comprehensive feature distance of each other vinegar-stained image. Thus, the image with the most obvious lesion in the vinegar-stained image can be automatically selected as the target vinegar-stained image for biopsy area prediction. Based on the comparison of the differences within and between classes, the visibility of the lesions in the vinegar-stained image is measured in a weighted manner, which can increase the robustness and effectiveness of the scheme.

[0235] Optionally, in the above Figure 3 On the basis of the corresponding embodiments, in another optional embodiment provided by the embodiment of the present application, the image features of the physiological saline image and the image features of each vinegar-stained image are calculated to obtain the inter-class feature distance between each vinegar-stained image and the physiological saline image, which may specifically include:

[0236] For each vinegar-stained image, element distance calculation is performed between each element feature in the image features corresponding to the vinegar-stained image and the element feature at the corresponding position in the image features corresponding to the saline image, to obtain M element feature distances between the vinegar-stained image and the saline image, where M is an integer greater than or equal to 1;

[0237] For each vinegar-stained image, image distance calculation is performed on the M element feature distances between the vinegar-stained image and the physiological saline image to obtain the inter-class feature distance between the vinegar-stained image and the physiological saline image.

[0238] In one or more embodiments, a method for calculating the inter-class feature distance is introduced. As can be seen from the above embodiments, the image feature distance between the vinegar-stained image and the saline image can be obtained based on the image features of the saline image and the image features of the vinegar-stained image. In one case, the image feature distance between the vinegar-stained image and the saline image can be directly used as the inter-class feature distance between the two.

[0239] Specifically, the distance between two image features can be defined as the spatial point distance. The image feature is represented as an M-dimensional feature vector, that is, an image feature includes M element features. Therefore, the inter-class feature distance between the vinegar-stained image and the saline image can be calculated as follows:

[0240]

[0241] Among them, Dis i0 F represents the inter-class feature distance between the i-th vinegar-stained image and the saline image. i represents the image feature of the i-th vinegar-stained image. F0 represents the image feature of the physiological saline image. x represents the x-th element feature in the image feature. M represents the total number of element features included in the image feature (ie, the length of the image feature).

[0242] It can be seen that for the i-th vinegar-stained image, the i-th vinegar-stained image corresponds to the x-th element feature in the image feature (i.e., F i (x)), and subtract the x-th element feature (i.e., F0(x)) from the image feature corresponding to the saline image to obtain the difference of the x-th element feature (i.e., F i(x)-F0(x)). Then square the difference of the x-th element feature to obtain the element feature distance of the x-th element feature (i.e., (F i (x)-F0(x)) 2 ). Thus, M element feature distances can be obtained. Based on this, the M element feature distances are first summed up (ie, ), and then take the square root of the summed result (i.e., ) to obtain the inter-class feature distance between the i-th vinegar-stained image and the saline image (i.e., Dis i0 ).

[0243] It should be noted that each vinegar-stained image uses the above method to calculate the inter-class feature distance between it and the saline image, which will not be described in detail here.

[0244] Again, in the embodiment of the present application, a method for calculating the inter-class feature distance is provided. In the above method, since the image features belong to high-dimensional features, and the high-dimensional features contain discriminative semantic information. Therefore, the inter-class feature distance calculated using the image features can better identify the semantic difference between the physiological saline image and the vinegar-stained image, thereby improving the accuracy of distinguishing the differences between different categories of images.

[0245] Optionally, in the above Figure 3 On the basis of the corresponding embodiments, in another optional embodiment provided by the embodiment of the present application, performing intra-class distance calculation on the image features of each vinegar-stained image and the corresponding image features of other vinegar-stained images to obtain the intra-class feature distance corresponding to each vinegar-stained image may specifically include:

[0246] For each vinegar-stained image, element distance calculation is performed between each element feature in the image features corresponding to the vinegar-stained image and the element features at the corresponding position in the image features corresponding to other vinegar-stained images, so as to obtain M element feature distances between each vinegar-stained image and other vinegar-stained images, where M is an integer greater than or equal to 1;

[0247] For each vinegar-stained image, performing image distance calculation on M element feature distances between the vinegar-stained image and other vinegar-stained images to obtain a first feature distance between the vinegar-stained image and other vinegar-stained images;

[0248] For each vinegar-stained image, the first characteristic distances between the vinegar-stained image and other vinegar-stained images are averaged to obtain the intra-class characteristic distance corresponding to the vinegar-stained image.

[0249] In one or more embodiments, a method for calculating intra-class feature distance is introduced. As can be seen from the above embodiments, the image feature distance between the vinegar-stained image and each other vinegar-stained image can be obtained based on the image features of P vinegar-stained images. In one case, the image feature distance between the vinegar-stained image and each other vinegar-stained image can be directly used as the intra-class feature distance between the two.

[0250] Specifically, the distance between two image features can be defined as the spatial point distance. The image feature is represented as an M-dimensional feature vector, that is, an image feature includes M element features. Therefore, the intra-class feature distance corresponding to the vinegar-stained image can be calculated as follows:

[0251]

[0252] Among them, Dis i F represents the intra-class feature distance corresponding to the i-th vinegar-stained image. i F represents the image features of the i-th vinegar-stained image. j represents the image feature of the jth vinegar-stained image. x represents the xth element feature in the image feature. M represents the total number of element features included in the image feature (i.e., the length of the image feature). P represents the total number of vinegar-stained images.

[0253] It can be seen that for the i-th vinegar-stained image, the i-th vinegar-stained image corresponds to the x-th element feature in the image feature (i.e., F i (x)), and the xth element feature in the image feature corresponding to the jth vinegar-stained image (i.e., F j (x)) to obtain the difference of the x-th element feature (i.e., F i (x)-F j (x)). Then square the difference of the x-th element feature to obtain the element feature distance of the x-th element feature (i.e., (F i (x)-F j (x) 2 ). Thus, M element feature distances can be obtained. Based on this, the M element feature distances are first summed up (ie, ), and then take the square root of the summed result (i.e., ) to obtain the first characteristic distance between the i-th vinegar-stained image and the j-th vinegar-stained image. Based on this, the first characteristic distances between the i-th vinegar-stained image and each other vinegar-stained image are averaged to obtain the intra-class characteristic distance corresponding to the i-th vinegar-stained image (i.e., Dis i ).

[0254] It should be noted that each vinegar-stained image uses the above method to calculate its corresponding intra-class feature distance, which will not be described in detail here.

[0255] Again, in the embodiment of the present application, a method for calculating the intra-class feature distance is provided. In the above method, since the image feature belongs to a high-dimensional feature, and the high-dimensional feature contains discriminative semantic information. Therefore, the intra-class feature distance calculated using the image feature can better identify the semantic difference between the vinegar-dyed image and other vinegar-dyed images, thereby improving the accuracy of discriminating the difference between images of the same category.

[0256] Optionally, in the above Figure 3 On the basis of the corresponding embodiments, in another optional embodiment provided by the embodiment of the present application, the image features of the physiological saline image and the image features of each vinegar-stained image are calculated to obtain the inter-class feature distance between each vinegar-stained image and the physiological saline image, which may specifically include:

[0257] Performing feature distance calculation on the image features of the physiological saline image and the image features of each vinegar-stained image to obtain the image feature distance between each vinegar-stained image and the physiological saline image;

[0258] Performing feature distance calculation on the histogram features of the physiological saline image and the histogram features of each vinegar-stained image to obtain the histogram feature distance between each vinegar-stained image and the physiological saline image;

[0259] For each vinegar-stained image, the image feature distance between the vinegar-stained image and the saline image, and the histogram feature distance between the vinegar-stained image and the saline image are weighted summed to obtain the inter-class feature distance between the vinegar-stained image and the saline image.

[0260] In one or more embodiments, another method for calculating the inter-class feature distance is introduced. It can be seen from the above embodiments that after obtaining P vinegar-stained images, on the one hand, it is necessary to calculate the image feature distance between each vinegar-stained image and the physiological saline image. On the other hand, it is necessary to calculate the histogram feature distance between each vinegar-stained image and the physiological saline image. Based on this, the inter-class feature distance between the vinegar-stained image and the physiological saline image can be calculated in the following manner:

[0261] Dis i0 =α1·DisHIST i0 +β1·DisCNN i0 ;Formula (6)

[0262] Among them, Dis i0 Represents the inter-class feature distance between the i-th vinegar-stained image and the saline image. DisHIST i0 Represents the histogram feature distance between the i-th vinegar-stained image and the saline image. DisCNN i0represents the image feature distance between the i-th vinegar-stained image and the saline image. α1 and β1 both represent weighting coefficients, which usually need to be set manually, for example, α1 = 0.5, β1 = 0.5.

[0263] For the i-th vinegar-stained image, the image feature distance between the i-th vinegar-stained image and the saline image (i.e., DisCNN i0 ), and the histogram feature distance between the i-th vinegar-stained image and the saline image (i.e., DisHIST i0 ) is weighted summed to obtain the inter-class feature distance between the i-th vinegar-stained image and the saline image (i.e., Dis i0 ).

[0264] Specifically, for ease of understanding, see Figure 8 , Figure 8 This is another schematic diagram of the structure of the biological tissue image detection method in the embodiment of the present application. As shown in the figure, taking the second image set including biological tissue image 1, biological tissue image 2, biological tissue image 3, biological tissue image 4, biological tissue image 5, biological tissue image 6 and biological tissue image 7 as an example, these biological tissue images are used as the input of the second image classification model, and the occlusion probability distribution of each biological tissue image is obtained through the second image classification model, and then the occlusion classification result of the biological tissue image can be determined based on the occlusion probability distribution. The occlusion classification result can be an "occlusion category" or a "non-occlusion category". Among them, biological tissue image 1, biological tissue image 3 and biological tissue image 4 all belong to the "occlusion category", so these three biological tissue images can be eliminated. And biological tissue images 5, biological tissue image 2, biological tissue image 6 and biological tissue image 7 belonging to the "non-occlusion category" are used as the first image set.

[0265] Based on this, biological tissue image 5, biological tissue image 2, biological tissue image 6 and biological tissue image 7 are used as inputs of the first image classification model, and the staining probability distribution of each biological tissue image is obtained through the first image classification model, and then the staining classification results of the biological tissue images can be determined based on the staining probability distribution. Among them, biological tissue image 5 is a physiological saline image, biological tissue image 2 and biological tissue image 6 are both vinegar-stained images, and biological tissue image 7 is an iodine-stained image. At the same time, the image features of each biological tissue image can also be obtained through the backbone network of the first image classification model, that is, image feature F0, image feature F1, image feature F2 and image feature F3.

[0266] On the one hand, it is necessary to use image feature F0, image feature F1 and image feature F2 to calculate feature distance to obtain image feature distance of vinegar-stained image 1 (ie, biological tissue image 2) and image feature distance of vinegar-stained image 2 (ie, biological tissue image 6).

[0267] On the other hand, it is necessary to extract the histogram features of the physiological saline image and each vinegar-stained image respectively to obtain the histogram feature f0 of the physiological saline image, the histogram feature f1 of the vinegar-stained image 1, and the histogram feature f2 of the vinegar-stained image 2. The feature distance is calculated using the histogram feature f0, the histogram feature f1, and the histogram feature f2 to obtain the histogram feature distance of the vinegar-stained image 1 (i.e., the biological tissue image 2) and the histogram feature distance of the vinegar-stained image 2 (i.e., the biological tissue image 6).

[0268] Thus, the comprehensive feature distance of vinegar-stained image 1 is calculated based on the image feature distance and histogram feature distance of vinegar-stained image 1. The comprehensive feature distance of vinegar-stained image 2 is calculated based on the image feature distance and histogram feature distance of vinegar-stained image 2. Therefore, the vinegar-stained image with a larger comprehensive feature distance is selected from vinegar-stained image 1 and vinegar-stained image 2 as the target vinegar-stained image. Finally, the target detection model is called to detect the target vinegar-stained image to obtain the prediction result of the biopsy area.

[0269] Again, in the embodiment of the present application, another method for calculating the inter-class feature distance is provided. In the above method, the image feature belongs to a high-dimensional feature, and the high-dimensional feature contains discriminative semantic information. The histogram feature belongs to a low-dimensional feature, and the low-dimensional feature is highly interpretable. Therefore, the inter-class feature distance calculated by using the image feature and the histogram feature can measure the differences between images of different categories, which not only takes into account the amount of information and the parseability, but also improves the robustness of the scheme.

[0270] Optionally, in the above Figure 3 On the basis of the corresponding embodiments, in another optional embodiment provided by the embodiment of the present application, the feature distance calculation is performed on the image features of the physiological saline image and the image features of each vinegar-stained image to obtain the image feature distance between each vinegar-stained image and the physiological saline image, which may specifically include:

[0271] For each vinegar-stained image, element distance calculation is performed between each element feature in the image features corresponding to the vinegar-stained image and the element feature at the corresponding position in the image features corresponding to the saline image, to obtain M element feature distances between the vinegar-stained image and the saline image, where M is an integer greater than or equal to 1;

[0272] For each vinegar-stained image, the image distance is calculated for the M element feature distances between the vinegar-stained image and the physiological saline image to obtain the image feature distance between the vinegar-stained image and the physiological saline image;

[0273] The feature distance calculation is performed on the histogram feature of the physiological saline image and the histogram feature of each vinegar-stained image to obtain the histogram feature distance between each vinegar-stained image and the physiological saline image, which may specifically include:

[0274] For each vinegar-stained image, each element feature in the histogram feature corresponding to the vinegar-stained image is calculated for element distance with the element feature at the corresponding position in the histogram feature corresponding to the saline image, to obtain N element feature distances between the vinegar-stained image and the saline image, where N is an integer greater than or equal to 1;

[0275] For each vinegar-stained image, a histogram distance calculation is performed on N element feature distances between the vinegar-stained image and the physiological saline image to obtain a histogram feature distance between the vinegar-stained image and the physiological saline image.

[0276] In one or more embodiments, a method for calculating image feature distance and histogram feature distance is introduced. It can be seen from the aforementioned embodiments that the image feature distance between the vinegar-stained image and the physiological saline image can be obtained based on the image features of the physiological saline image and the image features of the vinegar-stained image. The histogram feature distance between the vinegar-stained image and the physiological saline image can be obtained based on the histogram features of the physiological saline image and the histogram features of the vinegar-stained image. In one case, the result of the weighted summation of the image feature distance and the histogram feature distance between the vinegar-stained image and the physiological saline image can be used as the inter-class feature distance between the two.

[0277] Specifically, the distance between two image features can be defined as the spatial point distance. The image feature is represented as an M-dimensional feature vector, that is, an image feature includes M element features. Therefore, the image feature distance between the vinegar-stained image and the saline image can be calculated as follows:

[0278]

[0279] Among them, DisCNN i0 F represents the image feature distance between the i-th vinegar-stained image and the saline image. i represents the image feature of the i-th vinegar-stained image. F0 represents the image feature of the physiological saline image. x represents the x-th element feature in the image feature. M represents the total number of element features included in the image feature (ie, the length of the image feature).

[0280] It can be seen that for the i-th vinegar-stained image, the i-th vinegar-stained image corresponds to the x-th element feature in the image feature (i.e., F i (x)), and subtract the x-th element feature (i.e., F0(x)) from the image feature corresponding to the saline image to obtain the difference of the x-th element feature (i.e., F i(x)-F0(x)). Then square the difference of the x-th element feature to obtain the element feature distance of the x-th element feature (i.e., (F i (x)-F0(x)) 2 ). Thus, M element feature distances can be obtained. Based on this, the M element feature distances are first summed up (ie, ), and then take the square root of the summed result (i.e., ) to obtain the image feature distance between the i-th vinegar-stained image and the saline image (i.e., DisCNN i0 ).

[0281] The histogram feature is represented as an N-dimensional feature vector, that is, a histogram feature includes N element features. Therefore, the histogram feature distance between the vinegar-stained image and the physiological saline image can be calculated as follows:

[0282]

[0283] Among them, DisHIST i0 represents the histogram feature distance between the i-th vinegar-stained image and the saline image. i represents the histogram feature of the i-th vinegar-stained image. f0 represents the histogram feature of the physiological saline image. x represents the x-th element feature in the image feature. N represents the total number of element features included in the histogram feature (i.e., the length of the histogram feature).

[0284] It can be seen that for the i-th vinegar-stained image, the x-th element feature in the histogram feature corresponding to the i-th vinegar-stained image (i.e., f i (x)), and subtract the x-th element feature (i.e., f0(x)) in the histogram feature corresponding to the saline image to obtain the difference of the x-th element feature (i.e., f i (x)-f0(x)). Then square the difference of the x-th element feature to obtain the element feature distance of the x-th element feature (i.e., (f i (x)-f0(x)) 2 ). Thus, N element characteristic distances can be obtained. Based on this, the N element characteristic distances are first summed up (ie, ), and then take the square root of the summed result (i.e., ) to obtain the histogram feature distance between the i-th vinegar-stained image and the saline image (i.e., DisHIST i0 ).

[0285] It should be noted that the image feature distance and histogram feature distance between each vinegar-stained image and the physiological saline image are calculated in the above manner, which will not be described in detail here.

[0286] Furthermore, in an embodiment of the present application, a method for calculating image feature distance and histogram feature distance is provided. By the above method, the image feature distance and histogram feature distance can be calculated using image features and histogram features, thereby improving the feasibility and operability of the solution.

[0287] Optionally, in the above Figure 3 On the basis of the corresponding embodiments, in another optional embodiment provided by the embodiment of the present application, performing intra-class distance calculation on the image features of each vinegar-stained image and the corresponding image features of other vinegar-stained images to obtain the intra-class feature distance corresponding to each vinegar-stained image may specifically include:

[0288] According to the image features of each vinegar-stained image, the average distance of the image features corresponding to each vinegar-stained image is calculated;

[0289] According to the histogram features of each vinegar-stained image, the average distance of the histogram features corresponding to each vinegar-stained image is calculated;

[0290] For each vinegar-stained image, a weighted sum is performed on the average distance of image features corresponding to the vinegar-stained image and the average distance of histogram features corresponding to the vinegar-stained image to obtain the intra-class feature distance corresponding to the vinegar-stained image.

[0291] In one or more embodiments, another method for calculating the intra-class feature distance is introduced. It can be seen from the above embodiments that after obtaining P vinegar-stained images, on the one hand, the average image feature distance between the vinegar-stained image and the other vinegar-stained images can be obtained based on the image features of the other vinegar-stained images. On the other hand, the average histogram feature distance between the vinegar-stained image and the other vinegar-stained images can be obtained based on the histogram features of the other vinegar-stained images. Based on this, the intra-class feature distance corresponding to the vinegar-stained image can be calculated in the following manner:

[0292] Dis i =α2·DisHIST i +β2·DisCNN i ;Formula (9)

[0293] Among them, Dis i Represents the intra-class feature distance corresponding to the i-th vinegar-stained image. DisHIST i Represents the average distance of the histogram features corresponding to the i-th vinegar-stained image. DisCNN i represents the average distance of image features corresponding to the i-th vinegar-stained image. α2 and β2 both represent weighting coefficients, which usually need to be set manually, for example, α2 = 0.5, β2 = 0.5.

[0294] For the i-th vinegar-stained image, the average distance of image features between the i-th vinegar-stained image and other vinegar-stained images (i.e., DisCNN i ), and the average distance between the histogram features of the i-th vinegar-stained image and each other vinegar-stained image (i.e., DisHIST i ) is weighted summed to obtain the intra-class feature distance between the i-th vinegar-stained image and each other vinegar-stained image (i.e., Dis i ).

[0295] It should be noted that each vinegar-stained image uses the above method to calculate its corresponding intra-class feature distance, which will not be described in detail here.

[0296] Again, in the embodiment of the present application, another method for calculating the intra-class feature distance is provided. Through the above method, the image feature belongs to the high-dimensional feature, and the high-dimensional feature contains discriminative semantic information. The histogram feature belongs to the low-dimensional feature, and the low-dimensional feature is highly interpretable. Therefore, the intra-class feature distance calculated by using the image feature and the histogram feature can measure the differences between images of the same category, which not only takes into account the amount of information and parseability, but also improves robustness.

[0297] Optionally, in the above Figure 3 On the basis of the corresponding embodiments, in another optional embodiment provided by the embodiment of the present application, according to the image features of each vinegar-stained image, calculating the average distance of the image features corresponding to each vinegar-stained image may specifically include:

[0298] For each vinegar-stained image, element distance calculation is performed between each element feature in the image features corresponding to the vinegar-stained image and the element features at the corresponding position in the image features corresponding to other vinegar-stained images, so as to obtain M element feature distances between each vinegar-stained image and other vinegar-stained images, where M is an integer greater than or equal to 1;

[0299] For each vinegar-stained image, performing image distance calculation on M element feature distances between the vinegar-stained image and other vinegar-stained images to obtain a first feature distance between the vinegar-stained image and other vinegar-stained images;

[0300] For each vinegar-stained image, averaging the first characteristic distances between the vinegar-stained image and other vinegar-stained images to obtain an average image characteristic distance corresponding to the vinegar-stained image;

[0301] According to the histogram features of each vinegar-stained image, the average distance of the histogram features corresponding to each vinegar-stained image is calculated, which may specifically include:

[0302] For each vinegar-stained image, each element feature in the histogram feature corresponding to the vinegar-stained image is calculated with the element features at the corresponding position in the histogram features corresponding to other vinegar-stained images to obtain N element feature distances between each vinegar-stained image and other vinegar-stained images, where N is an integer greater than or equal to 1;

[0303] For each vinegar-stained image, a histogram distance calculation is performed on N element characteristic distances between the vinegar-stained image and other vinegar-stained images to obtain a second characteristic distance between the vinegar-stained image and other vinegar-stained images;

[0304] For each vinegar-stained image, the second characteristic distances between the vinegar-stained image and other vinegar-stained images are averaged to obtain the histogram characteristic average distance corresponding to the vinegar-stained image.

[0305] In one or more embodiments, a method for calculating the average distance of image features and the average distance of histogram features is introduced. It can be seen from the aforementioned embodiments that the average distance of image features between the vinegar-stained image and other vinegar-stained images can be obtained based on the image features of P vinegar-stained images. The average distance of histogram features between the vinegar-stained image and other vinegar-stained images can be obtained based on the histogram features of the P vinegar-stained images. In one case, the result of the weighted summation of the average distance of image features and the average distance of histogram features corresponding to the vinegar-stained image can be used as the inter-class feature distance between the two.

[0306] Specifically, the distance between two image features can be defined as the spatial point distance. The image feature is represented as an M-dimensional feature vector, that is, an image feature includes M element features. Therefore, the average distance of the image features corresponding to the vinegar-stained image can be calculated as follows:

[0307]

[0308] Among them, DisCNN i F represents the average distance of image features corresponding to the i-th vinegar-stained image. i F represents the image features of the i-th vinegar-stained image. j represents the image feature of the jth vinegar-stained image. x represents the xth element feature in the image feature. M represents the total number of element features included in the image feature (i.e., the length of the image feature). P represents the total number of vinegar-stained images.

[0309] It can be seen that for the i-th vinegar-stained image, the i-th vinegar-stained image corresponds to the x-th element feature in the image feature (i.e., F i (x)), and the xth element feature in the image feature corresponding to the jth vinegar-stained image (i.e., F j (x)) to obtain the difference of the x-th element feature (i.e., Fi (x)-F j (x)). Then square the difference of the x-th element feature to obtain the element feature distance of the x-th element feature (i.e., (F i (x)-F j (x) 2 ). Thus, M element feature distances can be obtained. Based on this, the M element feature distances are first summed up (ie, ), and then take the square root of the summed result (i.e., ) to obtain the first characteristic distance between the i-th vinegar-stained image and the j-th vinegar-stained image. Based on this, the first characteristic distances between the i-th vinegar-stained image and each other vinegar-stained image are averaged to obtain the average image characteristic distance corresponding to the i-th vinegar-stained image (i.e., DisCNN i ).

[0310] The histogram feature is represented as an N-dimensional feature vector, that is, a histogram feature includes N element features. Therefore, the average distance of the histogram feature corresponding to the vinegar-stained image can be calculated as follows:

[0311]

[0312] Among them, DisHIST i represents the average distance of the histogram features corresponding to the i-th vinegar-stained image. i (x) represents the histogram feature of the i-th vinegar-stained image. j (x) represents the histogram feature of the jth vinegar-stained image. x represents the xth element feature in the image feature. N represents the total number of element features included in the histogram feature (i.e., the length of the histogram feature). P represents the total number of vinegar-stained images.

[0313] It can be seen that for the i-th vinegar-stained image, the x-th element feature in the histogram feature corresponding to the i-th vinegar-stained image (i.e., f i (x)), corresponding to the xth element feature in the histogram feature of the jth vinegar-stained image (i.e., f j (x)) to obtain the difference of the x-th element feature (i.e., f i (x)-f j (x)). Then square the difference of the x-th element feature to obtain the element feature distance of the x-th element feature (i.e., (f i (x)-f j (x) 2 ). Thus, N element characteristic distances can be obtained. Based on this, the N element characteristic distances are first summed up (ie, ), and then take the square root of the summed result (i.e., ) to obtain the second characteristic distance between the i-th vinegar-stained image and the j-th vinegar-stained image. Based on this, the second characteristic distances between the i-th vinegar-stained image and each other vinegar-stained image are averaged to obtain the histogram feature average distance corresponding to the i-th vinegar-stained image (i.e., DisHIST i ).

[0314] It should be noted that each vinegar-stained image uses the above method to calculate the corresponding image feature average distance and histogram feature average distance, which will not be described in detail here.

[0315] Furthermore, in an embodiment of the present application, a method for calculating the average distance of image features and the average distance of histogram features is provided. By the above method, the average distance of image features and the average distance of histogram features can be calculated using image features and histogram features, thereby improving the feasibility and operability of the solution.

[0316] Optionally, in the above Figure 3 On the basis of the corresponding embodiments, another optional embodiment provided by the embodiment of the present application may further include:

[0317] Gray-scaling the physiological saline image to obtain a physiological saline grayscale image, wherein each pixel in the physiological saline grayscale image corresponds to a brightness value;

[0318] Based on the physiological saline grayscale image corresponding to the physiological saline image, respectively counting the number of pixels corresponding to each preset brightness value;

[0319] Constructing a histogram feature of the physiological saline image according to the number of pixels of the physiological saline grayscale image at each preset brightness value;

[0320] It may also include:

[0321] For each vinegar-dyed image, grayscale processing is performed on the vinegar-dyed image to obtain a vinegar-dyed grayscale image, wherein each pixel point in the vinegar-dyed grayscale image corresponds to a brightness value;

[0322] For each vinegar-dyed image, based on the vinegar-dyed grayscale image, the number of pixels corresponding to each preset brightness value is counted respectively;

[0323] For each vinegar-dyed image, a histogram feature of the vinegar-dyed image is constructed according to the number of pixels of the vinegar-dyed grayscale image at each preset brightness value.

[0324] In one or more embodiments, a method for constructing a histogram feature corresponding to a biological tissue image is introduced. As can be seen from the above embodiments, for a physiological saline image and a vinegar-stained image, their corresponding histogram features can be calculated respectively.

[0325] Specifically, first, the color image (i.e., the saline image and the vinegar-stained image) needs to be grayscaled using the red, green, blue (RGB) mean value method. Based on this, the grayscale image (i.e., the saline grayscale image and the vinegar-stained grayscale image) can be obtained in the following manner:

[0326] GrayImage(x,y)=floor{[Image R (x,y)+Image G (x,y)+Image B (x,y)] / 3}; Formula (12)

[0327] GrayImage represents a grayscale image (i.e., a saline grayscale image or a vinegar-stained grayscale image). x represents the horizontal coordinate of the pixel. y represents the vertical coordinate of the pixel. GrayImage(x,y) represents the brightness value of the pixel in the grayscale image. Image represents a color image (i.e., a saline image or a vinegar-stained image). Image R Represents the red channel corresponding to the color image, Image R (x,y) represents the pixel corresponding to the red channel of the color image. G Represents the green channel corresponding to the color image, Image G (x,y) represents the pixel corresponding to the green channel of the color image. B Represents the blue channel corresponding to the color image, Image B (x,y) represents the pixel corresponding to the blue channel of the color image. Floor means rounding down.

[0328] Based on this, the number of pixels corresponding to each preset brightness value can be counted based on the grayscale image (ie, the salt water grayscale image or the vinegar-stained grayscale image). The counting method is as follows:

[0329]

[0330] Where f(i) represents the number of pixels corresponding to the i-th preset brightness value. width represents the width of the grayscale image. x represents the horizontal coordinate of the pixel. height represents the height of the grayscale image. y represents the vertical coordinate of the pixel. GrayImage(x,y) represents the brightness value of the pixel in the grayscale image.

[0331] For example, assuming f(0)=200, f(1)=0, f(2)=150, the corresponding histogram features include (200, 0, 150). Thus, the histogram features of the physiological saline image and the histogram features of each aceto-stained image can be constructed.

[0332] It can be understood that, taking i∈[0,255] as an example, it corresponds to 256 preset brightness values. In one case, all 256 preset brightness values ​​are used as element features in the histogram feature, that is, N=256. In practical applications, N preset brightness values ​​can also be selected from the 256 preset brightness values ​​as element features in the histogram feature, for example, N=128.

[0333] Secondly, in the embodiment of the present application, a method for constructing a histogram feature corresponding to a biological tissue image is provided. Through the above method, the histogram feature can intuitively reflect the distribution of brightness values ​​of the biological tissue image, thereby increasing the interpretability of the feature.

[0334] Optionally, in the above Figure 3 On the basis of the corresponding embodiments, in another optional embodiment provided by the embodiment of the present application, after calling the target detection model to detect the target vinegar-stained image and obtaining the image detection result of the target vinegar-stained image, the following may also be included:

[0335] If the image detection result indicates that the target vinegar-stained image does not have a biopsy area, a failure prompt message is displayed through the operation interface, or the image detection result is sent to the terminal so that the terminal displays a failure prompt message;

[0336] If the image detection result is used to indicate that a biopsy area exists in the target aceto-stained image, at least one biopsy area included in the target aceto-stained image is displayed through the operation interface, or the image detection result is sent to the terminal so that the terminal displays at least one biopsy area included in the target aceto-stained image.

[0337] In one or more embodiments, a method for automatically generating a test report is introduced. As can be seen from the above embodiments, if the image detection result is used to indicate that the target vinegar-stained image does not have a biopsy area, then a failure prompt message is displayed in the operation interface provided by the terminal, for example, "No qualified image was detected this time." If the image detection result is used to indicate that the target vinegar-stained image has a biopsy area, then the corresponding biopsy area is displayed in the operation interface provided by the terminal.

[0338] Specifically, for ease of understanding, see Fig. 9 , Fig. 9 FIG. 1 is a schematic diagram of an operation interface for displaying a detection report based on image detection results in an embodiment of the present application. Fig. 9 As shown in Figure (A), the test report displays the patient's basic information, including but not limited to name, age, height, weight, test time, and preliminary conclusion. At the same time, the target acetic acid-stained image is also displayed on the operation interface, and three biopsy areas (i.e., the areas circled in white) are displayed in the target acetic acid-stained image. When the image switching control indicated by A1 is clicked, the following is displayed: Fig. 9The operation interface shown in Figure (B) shows the automatically identified iodine-stained image. When the image switching control indicated by A2 is clicked, the image can be switched back to Fig. 9 The operation interface is shown in Figure (A).

[0339] It should be noted that Fig. 9 The operation interface shown is only a schematic diagram. And because the iodine-stained image is not an image that must be collected during the inspection process, that is, the application of the iodine-stained image is not universal. Therefore, it may not be displayed. Fig. 9 Iodine-stained image shown in the middle (B).

[0340] Secondly, in the embodiment of the present application, a method for automatically generating a test report is provided. Through the above method, based on the image detection results, it can help doctors to efficiently detect the lesion condition to provide an effective lesion area. On the one hand, it can help areas lacking high-quality medical resources to improve the medical level and improve the overall diagnostic accuracy. On the other hand, it can solve certain medical resources. Since the method provided in this application can standardize the operating procedures, there is no need to rely too much on the personal ability of the doctor.

[0341] In combination with the above embodiments, for ease of understanding, please refer to Fig.10 , Fig.10 This is a schematic diagram of the overall process of the biological tissue image detection method in the embodiment of the present application. As shown in the figure, steps S1 to S14 can refer to the aforementioned embodiment and will not be described in detail here.

[0342] The biological tissue image detection device in this application is described in detail below. Fig.11 , Fig.11 This is a schematic diagram of an embodiment of a biological tissue image detection device in an embodiment of the present application. The biological tissue image detection device 30 includes:

[0343] An acquisition module 310 is used to acquire a first image set to be identified, wherein the first image set includes at least two biological tissue images;

[0344] The recognition module 320 is used to call the first image classification model to recognize each biological tissue image in the first image set, and obtain a staining classification result of each biological tissue image and an image feature of each biological tissue image, wherein the staining classification result is used to indicate the staining type to which the biological tissue image belongs;

[0345] The acquisition module 310 is further used to acquire a physiological saline image and P vinegar-stained images from the first image set according to the staining classification result of each biological tissue image, wherein the physiological saline image and the vinegar-stained image are both biological tissue images, and P is an integer greater than or equal to 1;

[0346] A determination module 330 is used to determine P inter-class feature distances and P intra-class feature distances according to the image features of the physiological saline image and the image features of each vinegar-stained image, wherein the P inter-class feature distances include the inter-class feature distance between each vinegar-stained image and the physiological saline image, and the P intra-class feature distances include the intra-class feature distance corresponding to each vinegar-stained image;

[0347] The determination module 330 is further used to determine the comprehensive feature distance corresponding to each vinegar-stained image according to the P inter-class feature distances and the P intra-class feature distances, wherein the comprehensive feature distance is used to characterize the difference significance of the vinegar-stained images;

[0348] The determination module 330 is further used to obtain the maximum comprehensive feature distance from the comprehensive feature distances corresponding to each vinegar-stained image, and use the vinegar-stained image corresponding to the maximum comprehensive feature distance as the target vinegar-stained image;

[0349] The detection module 340 is used to call the target detection model to detect the target vinegar-stained image to obtain an image detection result of the target vinegar-stained image, wherein the image detection result is used to determine the biopsy area in the target vinegar-stained image.

[0350] In an embodiment of the present application, a biological tissue image detection device is provided. With the above device, the intra-class feature distance of the vinegar-stained image and the inter-class feature distance between the vinegar-stained image and the physiological saline image are introduced to jointly measure the difference significance of the vinegar-stained image. Thus, the vinegar-stained image with the largest difference can be selected as the vinegar-stained image at the time of the maximum disease reaction, and the vinegar-stained image is used as the target vinegar-stained image for predicting the biopsy area. The above method fully takes into account the differences in the reaction time of the lesion area between different patients, which is conducive to improving the accuracy and effectiveness of the biopsy area prediction.

[0351] Optionally, in the above Fig.11 On the basis of the corresponding embodiment, in another embodiment of the biological tissue image detection device 30 provided in the embodiment of the present application,

[0352] An acquisition module 310, specifically used for a second image set to be identified, wherein the second image set includes at least two biological tissue images;

[0353] Calling the second image classification model to identify each biological tissue image in the second image set, and obtaining an occlusion classification result of each biological tissue image, wherein the occlusion classification result is used to indicate an occlusion condition of the biological tissue image;

[0354] According to the occlusion classification result of each biological tissue image, the biological tissue images whose occlusion classification results belong to the no-occlusion category in the second image set are used as the biological tissue images in the first image set.

[0355] In an embodiment of the present application, a biological tissue image detection device is provided. By using the above device, occluded images that do not meet the biopsy area prediction input can be automatically excluded, and biological tissue images occluded by cotton swabs or other instruments can be avoided as input, thereby facilitating the improvement of the accuracy of subsequent detection.

[0356] Optionally, in the above Fig.11 On the basis of the corresponding embodiment, in another embodiment of the biological tissue image detection device 30 provided in the embodiment of the present application, the biological tissue image detection device 30 further includes a training module 350;

[0357] The acquisition module 310 is further used to acquire a first image sample set, wherein the first image sample set includes at least one first image sample, and each first image sample corresponds to an occlusion classification label;

[0358] The acquisition module 310 is further configured to acquire, for each first image sample in the first image sample set, an occlusion probability distribution corresponding to each first image sample based on the second image classification model;

[0359] The acquisition module 310 is further used to calculate a first loss value based on the occlusion probability distribution and the occlusion classification label corresponding to each first image sample;

[0360] The training module 350 is used to update the model parameters of the second image classification model using the first loss value.

[0361] In an embodiment of the present application, a biological tissue image detection device is provided. The above device is used to train a sample set with occlusion classification labels using a deep classification network (i.e., a second image classification model). In this way, the occlusion situation of the biological tissue images collected autonomously by the doctor can be identified, thereby achieving the purpose of automated detection.

[0362] Optionally, in the above Fig.11 On the basis of the corresponding embodiment, in another embodiment of the biological tissue image detection device 30 provided in the embodiment of the present application,

[0363] The recognition module 320 is specifically configured to obtain, for each biological tissue image in the first image set, image features of each biological tissue image through a backbone network included in the first image classification model;

[0364] For each biological tissue image, a pooling feature of each biological tissue image is obtained through a pooling layer included in the first image classification model;

[0365] For each biological tissue image, a pooled feature is used to obtain a staining probability distribution of each biological tissue image through a fully connected network included in the first image classification model;

[0366] According to the staining probability distribution of each biological tissue image, the staining classification result of each biological tissue image is determined.

[0367] In the embodiment of the present application, a biological tissue image detection device is provided. By using the above device, the first image classification model can not only identify the staining classification result of the biological tissue image, but also extract the image features of the biological tissue image to learn and obtain the high-dimensional features of the biological tissue image. Thus, a corresponding basis is provided for subsequent feature processing, thereby improving the feasibility and operability of the solution.

[0368] Optionally, in the above Fig.11 On the basis of the corresponding embodiment, in another embodiment of the biological tissue image detection device 30 provided in the embodiment of the present application,

[0369] The identification module 320 is specifically used to compare the probability values ​​included in the staining probability distribution of each biological tissue image to obtain the maximum probability value in the staining probability distribution of each biological tissue image;

[0370] For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the first element feature position, the staining classification result of the biological tissue image is determined as a non-staining category;

[0371] For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the second element feature position, the staining classification result of the biological tissue image is determined as a physiological saline category, wherein the biological tissue image belonging to the physiological saline category is a physiological saline image;

[0372] For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the third element feature position, the staining classification result of the biological tissue image is determined as the vinegar staining category, wherein the biological tissue image belonging to the vinegar staining category is a vinegar staining image.

[0373] In an embodiment of the present application, a biological tissue image detection device is provided. Using the above device, based on the staining probability distribution output by the first image classification model, the staining category to which the biological tissue image belongs can be determined. Then, physiological saline images, vinegar-stained images, and iodine-stained images can be automatically distinguished without the need for relevant personnel to operate, thereby saving labor costs.

[0374] Optionally, in the above Fig.11 On the basis of the corresponding embodiment, in another embodiment of the biological tissue image detection device 30 provided in the embodiment of the present application,

[0375] The identification module 320 is specifically used to compare the probability values ​​included in the staining probability distribution of each biological tissue image to obtain the maximum probability value in the staining probability distribution of each biological tissue image;

[0376] For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the first element feature position, the staining classification result of the biological tissue image is determined as a non-staining category;

[0377] For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the second element feature position, the staining classification result of the biological tissue image is determined as a physiological saline category, wherein the biological tissue image belonging to the physiological saline category is a physiological saline image;

[0378] For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the third element feature position, the staining classification result of the biological tissue image is determined as a vinegar staining category, wherein the biological tissue image belonging to the vinegar staining category is a vinegar staining image;

[0379] For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the fourth element characteristic position, the staining classification result of the biological tissue image is determined as the iodine staining category, wherein the biological tissue image belonging to the iodine staining category is an iodine staining image.

[0380] In an embodiment of the present application, a biological tissue image detection device is provided. Using the above device, based on the staining probability distribution output by the first image classification model, the staining category to which the biological tissue image belongs can be determined. Then, physiological saline images, vinegar-stained images, and iodine-stained images can be automatically distinguished without the need for relevant personnel to operate, thereby saving labor costs.

[0381] Optionally, in the above Fig.11 On the basis of the corresponding embodiment, in another embodiment of the biological tissue image detection device 30 provided in the embodiment of the present application,

[0382] The acquisition module 310 is further used to acquire a second image sample set, wherein the second image sample set includes at least one second image sample, and each second image sample corresponds to a staining classification label;

[0383] The acquisition module 310 is further configured to acquire, for each second image sample in the second image sample set, a staining probability distribution corresponding to each second image sample based on the first image classification model;

[0384] The acquisition module 310 is further used to calculate a second loss value based on the staining probability distribution and the staining classification label corresponding to each second image sample;

[0385] The training module 350 is further used to update the model parameters of the first image classification model using the second loss value.

[0386] In an embodiment of the present application, a biological tissue image detection device is provided. The above device is used to train a sample set with a marked staining classification label using a deep classification network (i.e., a first image classification model). Thus, the staining condition of the biological tissue image collected autonomously by the doctor can be identified, thereby achieving the purpose of automated detection.

[0387] Optionally, in the above Fig.11 On the basis of the corresponding embodiment, in another embodiment of the biological tissue image detection device 30 provided in the embodiment of the present application,

[0388] The determination module 330 is specifically used to calculate the inter-class distance of the image features of the physiological saline image and the image features of each vinegar-stained image, so as to obtain the inter-class feature distance between each vinegar-stained image and the physiological saline image;

[0389] Performing intra-class distance calculation on the image features of each vinegar-stained image and the corresponding image features of other vinegar-stained images to obtain the intra-class feature distance corresponding to each vinegar-stained image, wherein the other vinegar-stained images are the remaining vinegar-stained images except the vinegar-stained image in the P vinegar-stained images;

[0390] The determination module 330 is specifically configured to perform weighted summation of the inter-class feature distance and the intra-class feature distance corresponding to each vinegar-stained image to obtain a comprehensive feature distance corresponding to the vinegar-stained image.

[0391] In an embodiment of the present application, a biological tissue image detection device is provided. Using the above device, the inter-class feature distance and the intra-class feature distance of the vinegar-stained image are weighted and summed to obtain the comprehensive feature distance of each other vinegar-stained image. Thus, the image with the most obvious lesions in the vinegar-stained image can be automatically selected as the target vinegar-stained image for biopsy area prediction. Based on the intra-class and inter-class difference comparison, the visibility of the lesions in the vinegar-stained image is measured in a weighted manner, which can increase the robustness and effectiveness of the scheme.

[0392] Optionally, in the above Fig.11 On the basis of the corresponding embodiment, in another embodiment of the biological tissue image detection device 30 provided in the embodiment of the present application,

[0393] The determination module 330 is specifically used to calculate the element distance between each element feature in the image feature corresponding to the vinegar-stained image and the element feature at the corresponding position in the image feature corresponding to the saline image for each vinegar-stained image, so as to obtain M element feature distances between the vinegar-stained image and the saline image, where M is an integer greater than or equal to 1;

[0394] For each vinegar-stained image, image distance calculation is performed on the M element feature distances between the vinegar-stained image and the physiological saline image to obtain the inter-class feature distance between the vinegar-stained image and the physiological saline image.

[0395] In an embodiment of the present application, a biological tissue image detection device is provided. With the above device, since the image features belong to high-dimensional features, and the high-dimensional features contain discriminative semantic information, the inter-class feature distance calculated using the image features can better identify the semantic difference between the physiological saline image and the vinegar-stained image, thereby improving the accuracy of discriminating the difference between images of different categories.

[0396] Optionally, in the above Fig.11 On the basis of the corresponding embodiment, in another embodiment of the biological tissue image detection device 30 provided in the embodiment of the present application,

[0397] The determination module 330 is specifically used to calculate the element distance between each element feature in the image feature corresponding to the vinegar-stained image and the element features at the corresponding position in the image features corresponding to other vinegar-stained images for each vinegar-stained image, so as to obtain M element feature distances between each vinegar-stained image and other vinegar-stained images, wherein M is an integer greater than or equal to 1;

[0398] For each vinegar-stained image, performing image distance calculation on M element feature distances between the vinegar-stained image and other vinegar-stained images to obtain a first feature distance between the vinegar-stained image and other vinegar-stained images;

[0399] For each vinegar-stained image, the first characteristic distances between the vinegar-stained image and other vinegar-stained images are averaged to obtain the intra-class characteristic distance corresponding to the vinegar-stained image.

[0400] In an embodiment of the present application, a biological tissue image detection device is provided. With the above device, since the image features belong to high-dimensional features, and the high-dimensional features contain discriminative semantic information, the intra-class feature distance calculated using the image features can better identify the semantic difference between the vinegar-stained image and other vinegar-stained images, thereby improving the accuracy of discriminating the difference between images of the same category.

[0401] Optionally, in the above Fig.11On the basis of the corresponding embodiment, in another embodiment of the biological tissue image detection device 30 provided in the embodiment of the present application,

[0402] The determination module 330 is specifically used to calculate the feature distance between the image feature of the physiological saline image and the image feature of each vinegar-stained image, so as to obtain the image feature distance between each vinegar-stained image and the physiological saline image;

[0403] Performing feature distance calculation on the histogram features of the physiological saline image and the histogram features of each vinegar-stained image to obtain the histogram feature distance between each vinegar-stained image and the physiological saline image;

[0404] For each vinegar-stained image, the image feature distance between the vinegar-stained image and the saline image, and the histogram feature distance between the vinegar-stained image and the saline image are weighted summed to obtain the inter-class feature distance between the vinegar-stained image and the saline image.

[0405] In an embodiment of the present application, a biological tissue image detection device is provided. Using the above device, the image features belong to high-dimensional features, and the high-dimensional features contain discriminative semantic information. The histogram features belong to low-dimensional features, and the low-dimensional features are highly interpretable. Therefore, the inter-class feature distance calculated by using the image features and the histogram features can measure the differences between images of different categories, which not only takes into account the amount of information and the interpretability, but also improves the robustness of the scheme.

[0406] Optionally, in the above Fig.11 On the basis of the corresponding embodiment, in another embodiment of the biological tissue image detection device 30 provided in the embodiment of the present application,

[0407] The determination module 330 is specifically used to calculate the element distance between each element feature in the image feature corresponding to the vinegar-stained image and the element feature at the corresponding position in the image feature corresponding to the saline image for each vinegar-stained image, so as to obtain M element feature distances between the vinegar-stained image and the saline image, where M is an integer greater than or equal to 1;

[0408] For each vinegar-stained image, the image distance is calculated for the M element feature distances between the vinegar-stained image and the physiological saline image to obtain the image feature distance between the vinegar-stained image and the physiological saline image;

[0409] The determination module 330 is specifically used to calculate the element distance between each element feature in the histogram feature corresponding to the vinegar-stained image and the element feature at the corresponding position in the histogram feature corresponding to the saline image for each vinegar-stained image, so as to obtain N element feature distances between the vinegar-stained image and the saline image, where N is an integer greater than or equal to 1;

[0410] For each vinegar-stained image, a histogram distance calculation is performed on N element feature distances between the vinegar-stained image and the physiological saline image to obtain a histogram feature distance between the vinegar-stained image and the physiological saline image.

[0411] In an embodiment of the present application, a biological tissue image detection device is provided. By using the above device, the image feature and histogram feature can be used to calculate the image feature distance and the histogram feature distance, thereby improving the feasibility and operability of the solution.

[0412] Optionally, in the above Fig.11 On the basis of the corresponding embodiment, in another embodiment of the biological tissue image detection device 30 provided in the embodiment of the present application,

[0413] The determination module 330 is specifically used to calculate the average distance of the image features corresponding to each vinegar-stained image according to the image features of each vinegar-stained image;

[0414] According to the histogram features of each vinegar-stained image, the average distance of the histogram features corresponding to each vinegar-stained image is calculated;

[0415] For each vinegar-stained image, a weighted sum is performed on the average distance of image features corresponding to the vinegar-stained image and the average distance of histogram features corresponding to the vinegar-stained image to obtain the intra-class feature distance corresponding to the vinegar-stained image.

[0416] In an embodiment of the present application, a biological tissue image detection device is provided. Using the above device, the image features are high-dimensional features, and the high-dimensional features contain discriminative semantic information. The histogram features are low-dimensional features, and the low-dimensional features are highly interpretable. Therefore, the intra-class feature distance calculated by using the image features and the histogram features can measure the differences between images of the same category, which not only takes into account the amount of information and interpretability, but also improves robustness.

[0417] Optionally, in the above Fig.11 On the basis of the corresponding embodiment, in another embodiment of the biological tissue image detection device 30 provided in the embodiment of the present application,

[0418] The determination module 330 is specifically used to calculate the element distance between each element feature in the image feature corresponding to the vinegar-stained image and the element features at the corresponding position in the image features corresponding to other vinegar-stained images for each vinegar-stained image, so as to obtain M element feature distances between each vinegar-stained image and other vinegar-stained images, wherein M is an integer greater than or equal to 1;

[0419] For each vinegar-stained image, performing image distance calculation on M element feature distances between the vinegar-stained image and other vinegar-stained images to obtain a first feature distance between the vinegar-stained image and other vinegar-stained images;

[0420] For each vinegar-stained image, averaging the first characteristic distances between the vinegar-stained image and other vinegar-stained images to obtain an average image characteristic distance corresponding to the vinegar-stained image;

[0421] The determination module 330 is specifically used to calculate the element distance between each element feature in the histogram feature corresponding to the vinegar-stained image and the element features at the corresponding position in the histogram features corresponding to other vinegar-stained images for each vinegar-stained image, so as to obtain N element feature distances between each vinegar-stained image and other vinegar-stained images, wherein N is an integer greater than or equal to 1;

[0422] For each vinegar-stained image, a histogram distance calculation is performed on N element characteristic distances between the vinegar-stained image and other vinegar-stained images to obtain a second characteristic distance between the vinegar-stained image and other vinegar-stained images;

[0423] For each vinegar-stained image, the second characteristic distances between the vinegar-stained image and other vinegar-stained images are averaged to obtain the histogram characteristic average distance corresponding to the vinegar-stained image.

[0424] In an embodiment of the present application, a biological tissue image detection device is provided. By using the above device, the image feature and the histogram feature can be used to calculate the average distance of the image feature and the average distance of the histogram feature, thereby improving the feasibility and operability of the solution.

[0425] Optionally, in the above Fig.11 On the basis of the corresponding embodiment, in another embodiment of the biological tissue image detection device 30 provided in the embodiment of the present application, the biological tissue image detection device 30 further includes a processing module 360;

[0426] A processing module 360 ​​is used to perform grayscale processing on the physiological saline image to obtain a physiological saline grayscale image, wherein each pixel in the physiological saline grayscale image corresponds to a brightness value;

[0427] The processing module 360 ​​is further used to count the number of pixels corresponding to each preset brightness value based on the physiological saline grayscale image corresponding to the physiological saline image;

[0428] The processing module 360 ​​is further used to construct a histogram feature of the physiological saline image according to the number of pixels of the physiological saline grayscale image at each preset brightness value;

[0429] The processing module 360 ​​is further used to perform grayscale processing on each vinegar-stained image to obtain a vinegar-stained grayscale image, wherein each pixel in the vinegar-stained grayscale image corresponds to a brightness value;

[0430] The processing module 360 ​​is further used to count the number of pixels corresponding to each preset brightness value for each vinegar-dyed image based on the vinegar-dyed grayscale image;

[0431] The processing module 360 ​​is further configured to construct a histogram feature of each vinegar-dyed image according to the number of pixels of the vinegar-dyed grayscale image at each preset brightness value.

[0432] In an embodiment of the present application, a biological tissue image detection device is provided. By using the above device, the brightness value distribution of the biological tissue image can be intuitively reflected by using the histogram feature, thereby increasing the interpretability of the feature.

[0433] Optionally, in the above Fig.11 On the basis of the corresponding embodiment, in another embodiment of the biological tissue image detection device 30 provided in the embodiment of the present application, the biological tissue image detection device 30 further includes a display module 370;

[0434] The display module 370 is used to call the target detection model to detect the target vinegar-stained image, and after obtaining the image detection result of the target vinegar-stained image, if the image detection result is used to indicate that the target vinegar-stained image does not have a biopsy area, display a failure prompt message through the operation interface, or send the image detection result to the terminal so that the terminal displays a failure prompt message;

[0435] The display module 370 is further used to display at least one biopsy area included in the target aceto-stained image through an operation interface if the image detection result is used to indicate that a biopsy area exists in the target aceto-stained image, or to send the image detection result to the terminal so that the terminal displays at least one biopsy area included in the target aceto-stained image.

[0436] In an embodiment of the present application, a biological tissue image detection device is provided. By using the above device, based on the image detection results, it can help doctors efficiently detect the lesion condition to provide an effective lesion area. On the one hand, it can help areas lacking high-quality medical resources to improve the medical level and improve the overall diagnostic accuracy. On the other hand, it can solve certain medical resources. Since the method provided in this application can standardize the operating procedures, there is no need to rely too much on the personal ability of the doctor.

[0437] Fig.124 is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. The computer device 400 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 422 (for example, one or more processors) and a memory 432, and one or more storage media 430 (for example, one or more mass storage devices) storing application programs 442 or data 444. Among them, the memory 432 and the storage medium 430 can be short-term storage or permanent storage. The program stored in the storage medium 430 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the computer device. Furthermore, the central processing unit 422 can be configured to communicate with the storage medium 430 to execute a series of instruction operations in the storage medium 430 on the computer device 400.

[0438] The computer device 400 may also include one or more power supplies 426, one or more wired or wireless network interfaces 450, one or more input and output interfaces 458, and / or one or more operating systems 441, such as Windows Server 2000, Windows Server 2003, Windows Server 2003E, Windows Server 2003R ... TM , Mac OS X TM , Unix TM , Linux TM , FreeBSD TM etc.

[0439] The steps performed by the computer device in the above embodiment can be based on the Fig.12 The computer device structure shown.

[0440] A computer-readable storage medium is also provided in an embodiment of the present application, on which a computer program is stored. When the computer program is executed by a processor, the steps of the methods described in the above embodiments are implemented.

[0441] A computer program product is also provided in an embodiment of the present application, including a computer program, which, when executed by a processor, implements the steps of the methods described in the above embodiments.

[0442] It is understandable that in the specific implementation of the present application, related data such as biological tissue images and patient information are involved. When the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0443] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0444] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0445] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0446] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0447] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a server or terminal device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store computer programs.

[0448] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting biological tissue images, characterized in that: include: Acquire a first image set to be identified, wherein the first image set includes at least two biological tissue images, and the biological tissue images belong to a non-occlusion category; Calling a first image classification model to identify each biological tissue image in the first image set, and obtaining a staining classification result of each biological tissue image and an image feature of each biological tissue image, wherein the staining classification result is used to indicate a staining type to which the biological tissue image belongs; According to the staining classification result of each biological tissue image, a physiological saline image and P vinegar-stained images are obtained from the first image set, wherein the physiological saline image and the vinegar-stained image both belong to the biological tissue image, and P is an integer greater than or equal to 1; According to the image features of the physiological saline image and the image features of each vinegar-stained image, P inter-class feature distances and P intra-class feature distances are determined, wherein the P inter-class feature distances include the inter-class feature distance between each vinegar-stained image and the physiological saline image, the inter-class feature distance is the image feature distance between the vinegar-stained image and the physiological saline image, and the P intra-class feature distances include the intra-class feature distance between each vinegar-stained image and other vinegar-stained images, the intra-class feature distance is the image feature distance between each vinegar-stained image and other vinegar-stained images; For each vinegar-stained image, a weighted sum is performed according to the inter-class feature distance corresponding to the vinegar-stained image and the intra-class feature distance corresponding to each vinegar-stained image, so as to obtain a comprehensive feature distance corresponding to each vinegar-stained image, wherein the comprehensive feature distance is used to characterize the difference significance of the vinegar-stained images; Acquire a maximum comprehensive feature distance from the comprehensive feature distances corresponding to each vinegar-stained image, and use the vinegar-stained image corresponding to the maximum comprehensive feature distance as a target vinegar-stained image; The target detection model is called to detect the target aceto-stained image to obtain an image detection result of the target aceto-stained image, wherein the image detection result is used to determine a biopsy area in the target aceto-stained image.

2. The detection method according to claim 1, characterized in that: The step of obtaining a first set of images to be identified includes: a second image set to be identified, wherein the second image set includes at least two biological tissue images; Calling a second image classification model to identify each biological tissue image in the second image set, and obtaining an occlusion classification result of each biological tissue image, wherein the occlusion classification result is used to indicate an occlusion condition of the biological tissue image; According to the occlusion classification result of each biological tissue image, the biological tissue images whose occlusion classification results belong to the no-occlusion category in the second image set are used as the biological tissue images in the first image set.

3. The detection method according to claim 2, characterized in that: The method further comprises: Acquire a first image sample set, wherein the first image sample set includes at least one first image sample, and each first image sample corresponds to an occlusion classification label; For each first image sample in the first image sample set, acquiring an occlusion probability distribution corresponding to each first image sample based on the second image classification model; Calculate a first loss value based on the occlusion probability distribution corresponding to each first image sample and the occlusion classification label; The first loss value is used to update the model parameters of the second image classification model.

4. The detection method according to claim 1, characterized in that: The calling of the first image classification model to identify each biological tissue image in the first image set to obtain a staining classification result of each biological tissue image and an image feature of each biological tissue image includes: For each biological tissue image in the first image set, acquiring image features of each biological tissue image through a backbone network included in the first image classification model; With respect to the image features of each biological tissue image, obtaining the pooling features of each biological tissue image through the pooling layer included in the first image classification model; With respect to the pooled features of each biological tissue image, obtaining the staining probability distribution of each biological tissue image through the fully connected network included in the first image classification model; The staining classification result of each biological tissue image is determined according to the staining probability distribution of each biological tissue image.

5. The detection method according to claim 4, characterized in that: Determining the staining classification result of each biological tissue image according to the staining probability distribution of each biological tissue image includes: Comparing the probability values ​​included in the staining probability distribution of each biological tissue image to obtain the maximum probability value in the staining probability distribution of each biological tissue image; For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the first element feature position, the staining classification result of the biological tissue image is determined as a non-staining category; For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the second element feature position, the staining classification result of the biological tissue image is determined as a physiological saline category, wherein the biological tissue image belonging to the physiological saline category is the physiological saline image; For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the third element feature position, the staining classification result of the biological tissue image is determined as the vinegar-stained category, wherein the biological tissue image belonging to the vinegar-stained category is the vinegar-stained image.

6. The detection method according to claim 4, characterized in that: Determining the staining classification result of each biological tissue image according to the staining probability distribution of each biological tissue image includes: Comparing the probability values ​​included in the staining probability distribution of each biological tissue image to obtain the maximum probability value in the staining probability distribution of each biological tissue image; For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the first element feature position, the staining classification result of the biological tissue image is determined as a non-staining category; For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the second element feature position, the staining classification result of the biological tissue image is determined as a physiological saline category, wherein the biological tissue image belonging to the physiological saline category is the physiological saline image; For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the third element feature position, the staining classification result of the biological tissue image is determined as a vinegar staining category, wherein the biological tissue image belonging to the vinegar staining category is the vinegar staining image; For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the fourth element characteristic position, the staining classification result of the biological tissue image is determined as an iodine staining category, wherein the biological tissue image belonging to the iodine staining category is an iodine staining image.

7. The detection method according to any one of claims 1 to 6, characterized in that: The method further comprises: Acquire a second image sample set, wherein the second image sample set includes at least one second image sample, and each second image sample corresponds to a staining classification label; For each second image sample in the second image sample set, acquiring a staining probability distribution corresponding to each second image sample based on the first image classification model; Calculating a second loss value based on the staining probability distribution corresponding to each second image sample and the staining classification label; The model parameters of the first image classification model are updated using the second loss value.

8. The detection method according to claim 1, characterized in that: The determining of P inter-class feature distances and P intra-class feature distances according to the image features of the physiological saline image and the image features of each vinegar-stained image includes: Performing inter-class distance calculation on the image features of the physiological saline image and the image features of each vinegar-stained image to obtain the inter-class feature distance between each vinegar-stained image and the physiological saline image; An intra-class distance calculation is performed on the image features of each vinegar-stained image and the corresponding image features of other vinegar-stained images to obtain the intra-class feature distance corresponding to each vinegar-stained image, wherein the other vinegar-stained images are the remaining vinegar-stained images among the P vinegar-stained images except the vinegar-stained image.

9. The detection method according to claim 8, characterized in that: The performing inter-class distance calculation on the image features of the physiological saline image and the image features of each vinegar-stained image to obtain the inter-class feature distance between each vinegar-stained image and the physiological saline image includes: For each vinegar-stained image, performing element distance calculation between each element feature in the image features corresponding to the vinegar-stained image and the element features at the corresponding position in the image features corresponding to the physiological saline image, to obtain M element feature distances between the vinegar-stained image and the physiological saline image, where M is an integer greater than or equal to 1; For each vinegar-stained image, image distance calculation is performed on the M element feature distances between the vinegar-stained image and the physiological saline image to obtain the inter-class feature distance between the vinegar-stained image and the physiological saline image.

10. The detection method according to claim 8, characterized in that: The intra-class distance calculation is performed on the image features of each vinegar-stained image and the corresponding image features of other vinegar-stained images to obtain the intra-class feature distance corresponding to each vinegar-stained image, including: For each of the vinegar-stained images, element distance calculation is performed between each element feature in the image features corresponding to the vinegar-stained image and the element features at the corresponding position in the image features corresponding to the other vinegar-stained images, so as to obtain M element feature distances between each of the vinegar-stained images and the other vinegar-stained images, wherein M is an integer greater than or equal to 1; For each of the vinegar-stained images, performing image distance calculation on M element feature distances between the vinegar-stained image and each of the other vinegar-stained images to obtain a first feature distance between the vinegar-stained image and each of the other vinegar-stained images; For each of the vinegar-stained images, the first characteristic distances between the vinegar-stained image and each of the other vinegar-stained images are averaged to obtain the intra-class characteristic distance corresponding to the vinegar-stained image.

11. The detection method according to claim 8, characterized in that: The performing inter-class distance calculation on the image features of the physiological saline image and the image features of each vinegar-stained image to obtain the inter-class feature distance between each vinegar-stained image and the physiological saline image includes: Performing feature distance calculation on the image features of the physiological saline image and the image features of each vinegar-stained image to obtain the image feature distance between each vinegar-stained image and the physiological saline image; Performing feature distance calculation on the histogram features of the physiological saline image and the histogram features of each vinegar-stained image to obtain a histogram feature distance between each vinegar-stained image and the physiological saline image; For each vinegar-stained image, a weighted sum is performed on the image feature distance between the vinegar-stained image and the physiological saline image, and the histogram feature distance between the vinegar-stained image and the physiological saline image to obtain an inter-class feature distance between the vinegar-stained image and the physiological saline image.

12. The detection method according to claim 11, characterized in that: The performing feature distance calculation on the image features of the physiological saline image and the image features of each vinegar-stained image to obtain the image feature distance between each vinegar-stained image and the physiological saline image includes: For each vinegar-stained image, performing element distance calculation between each element feature in the image features corresponding to the vinegar-stained image and the element features at the corresponding position in the image features corresponding to the physiological saline image, to obtain M element feature distances between the vinegar-stained image and the physiological saline image, where M is an integer greater than or equal to 1; For each vinegar-stained image, performing image distance calculation on M element feature distances between the vinegar-stained image and the physiological saline image to obtain an image feature distance between the vinegar-stained image and the physiological saline image; The performing feature distance calculation on the histogram feature of the physiological saline image and the histogram feature of each vinegar-stained image to obtain the histogram feature distance between each vinegar-stained image and the physiological saline image includes: For each vinegar-stained image, performing element distance calculation on each element feature in the histogram feature corresponding to the vinegar-stained image and the element feature at the corresponding position in the histogram feature corresponding to the physiological saline image, to obtain N element feature distances between the vinegar-stained image and the physiological saline image, where N is an integer greater than or equal to 1; For each vinegar-stained image, a histogram distance calculation is performed on N element feature distances between the vinegar-stained image and the physiological saline image to obtain a histogram feature distance between the vinegar-stained image and the physiological saline image.

13. The detection method according to claim 8, characterized in that: The intra-class distance calculation is performed on the image features of each vinegar-stained image and the corresponding image features of other vinegar-stained images to obtain the intra-class feature distance corresponding to each vinegar-stained image, including: According to the image features of each vinegar-stained image, an average distance of the image features corresponding to each vinegar-stained image is calculated; According to the histogram features of each vinegar-stained image, an average distance of the histogram features corresponding to each vinegar-stained image is calculated; For each vinegar-stained image, a weighted sum is performed on the average distance of image features corresponding to the vinegar-stained image and the average distance of histogram features corresponding to the vinegar-stained image to obtain the intra-class feature distance corresponding to the vinegar-stained image.

14. The detection method according to claim 13, characterized in that: The step of calculating the average distance of the image features corresponding to each vinegar-stained image according to the image features of each vinegar-stained image includes: For each of the vinegar-stained images, calculating the element distance between each element feature in the image features corresponding to the vinegar-stained image and the element features at the corresponding position in the image features corresponding to other vinegar-stained images, to obtain M element feature distances between each of the vinegar-stained images and the other vinegar-stained images, wherein M is an integer greater than or equal to 1; For each of the vinegar-stained images, performing image distance calculation on M element feature distances between the vinegar-stained image and each of the other vinegar-stained images to obtain a first feature distance between the vinegar-stained image and each of the other vinegar-stained images; For each of the vinegar-stained images, averaging the first characteristic distances between the vinegar-stained image and each of the other vinegar-stained images to obtain an average image characteristic distance corresponding to the vinegar-stained image; The step of calculating the average distance of the histogram features corresponding to each vinegar-stained image according to the histogram features of each vinegar-stained image comprises: For each of the vinegar-stained images, element distance calculation is performed between each element feature in the histogram feature corresponding to the vinegar-stained image and the element features at the corresponding position in the histogram features corresponding to other vinegar-stained images, so as to obtain N element feature distances between each of the vinegar-stained images and the other vinegar-stained images, wherein N is an integer greater than or equal to 1; For each of the vinegar-stained images, a histogram distance calculation is performed on N element characteristic distances between the vinegar-stained image and each of the other vinegar-stained images to obtain a second characteristic distance between the vinegar-stained image and each of the other vinegar-stained images; For each of the vinegar-stained images, the second characteristic distances between the vinegar-stained image and each of the other vinegar-stained images are averaged to obtain the histogram characteristic average distance corresponding to the vinegar-stained image.

15. The detection method according to any one of claims 11 to 14, characterized in that: The method further comprises: grayscale the physiological saline image to obtain a physiological saline grayscale image, wherein each pixel in the physiological saline grayscale image corresponds to a brightness value; Based on the physiological saline grayscale image corresponding to the physiological saline image, respectively counting the number of pixel points corresponding to each preset brightness value; Constructing a histogram feature of the physiological saline image according to the number of pixels of the physiological saline grayscale image at each preset brightness value; The method further comprises: For each of the vinegar-dyed images, grayscale processing is performed on the vinegar-dyed image to obtain a vinegar-dyed grayscale image, wherein each pixel in the vinegar-dyed grayscale image corresponds to a brightness value; For each of the vinegar-dyed images, based on the vinegar-dyed grayscale image, respectively counting the number of pixels corresponding to each preset brightness value; For each vinegar-dyed image, a histogram feature of the vinegar-dyed image is constructed according to the number of pixels of the vinegar-dyed grayscale image at each preset brightness value.

16. The detection method according to claim 1, characterized in that: After calling the target detection model to detect the target vinegar-stained image and obtaining the image detection result of the target vinegar-stained image, the method further includes: If the image detection result indicates that the target vinegar-stained image does not have a biopsy area, a failure prompt message is displayed through an operation interface, or the image detection result is sent to a terminal so that the terminal displays the failure prompt message; If the image detection result is used to indicate that a biopsy area exists in the target aceto-stained image, at least one biopsy area included in the target aceto-stained image is displayed through the operation interface, or the image detection result is sent to the terminal so that the terminal displays at least one biopsy area included in the target aceto-stained image.

17. A biological tissue image detection device, characterized in that: include: An acquisition module, configured to acquire a first image set to be identified, wherein the first image set includes at least two biological tissue images, and the biological tissue images belong to a non-occluded category; an identification module, used for calling a first image classification model to identify each biological tissue image in the first image set, and obtaining a staining classification result of each biological tissue image and an image feature of each biological tissue image, wherein the staining classification result is used to indicate the staining type to which the biological tissue image belongs; The acquisition module is further used to acquire a physiological saline image and P vinegar-stained images from the first image set according to the staining classification result of each biological tissue image, wherein the physiological saline image and the vinegar-stained image both belong to the biological tissue image, and P is an integer greater than or equal to 1; a determination module, configured to determine P inter-class feature distances and P intra-class feature distances according to the image features of the physiological saline image and the image features of each vinegar-stained image, wherein the P inter-class feature distances include the inter-class feature distance between each vinegar-stained image and the physiological saline image, the inter-class feature distance is the image feature distance between the vinegar-stained image and the physiological saline image, and the P intra-class feature distances include the intra-class feature distance between each vinegar-stained image and other vinegar-stained images, the intra-class feature distance is the image feature distance between each vinegar-stained image and other vinegar-stained images; The determination module is further used to perform weighted summation on each vinegar-stained image according to the P inter-class feature distances corresponding to the vinegar-stained image and the intra-class feature distance corresponding to each vinegar-stained image, so as to obtain a comprehensive feature distance corresponding to each vinegar-stained image, wherein the comprehensive feature distance is used to characterize the difference significance of the vinegar-stained images; The determination module is further used to obtain a maximum comprehensive characteristic distance from the comprehensive characteristic distances corresponding to each vinegar-stained image, and use the vinegar-stained image corresponding to the maximum comprehensive characteristic distance as a target vinegar-stained image; The detection module is used to call the target detection model to detect the target vinegar-stained image to obtain an image detection result of the target vinegar-stained image, wherein the image detection result is used to determine the biopsy area in the target vinegar-stained image.

18. The device according to claim 17, characterized in that The device comprises: An acquisition module, specifically used for a second image set to be identified, wherein the second image set includes at least two biological tissue images; Calling a second image classification model to identify each biological tissue image in the second image set, and obtaining an occlusion classification result of each biological tissue image, wherein the occlusion classification result is used to indicate an occlusion condition of the biological tissue image; According to the occlusion classification result of each biological tissue image, the biological tissue images whose occlusion classification results belong to the no-occlusion category in the second image set are used as the biological tissue images in the first image set.

19. The device according to claim 18, characterized in that The device also includes a training module; The acquisition module is further used to acquire a first image sample set, wherein the first image sample set includes at least one first image sample, and each first image sample corresponds to an occlusion classification label; The acquisition module is further used to acquire, for each first image sample in the first image sample set, an occlusion probability distribution corresponding to the each first image sample based on the second image classification model; The acquisition module is further used to calculate a first loss value based on the occlusion probability distribution corresponding to each first image sample and the occlusion classification label; A training module is used to update the model parameters of the second image classification model using the first loss value.

20. The device according to claim 17, characterized in that The device comprises: an identification module, specifically configured to obtain, for each biological tissue image in the first image set, image features of each biological tissue image through a backbone network included in the first image classification model; With respect to the image features of each biological tissue image, obtaining the pooling features of each biological tissue image through the pooling layer included in the first image classification model; With respect to the pooled features of each biological tissue image, obtaining the staining probability distribution of each biological tissue image through the fully connected network included in the first image classification model; The staining classification result of each biological tissue image is determined according to the staining probability distribution of each biological tissue image.

21. The device according to claim 20, characterized in that The device comprises: An identification module, specifically used to compare the probability values ​​included in the staining probability distribution of each biological tissue image, and obtain the maximum probability value in the staining probability distribution of each biological tissue image; For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the first element feature position, the staining classification result of the biological tissue image is determined as a non-staining category; For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the second element feature position, the staining classification result of the biological tissue image is determined as a physiological saline category, wherein the biological tissue image belonging to the physiological saline category is the physiological saline image; For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the third element feature position, the staining classification result of the biological tissue image is determined as the vinegar-stained category, wherein the biological tissue image belonging to the vinegar-stained category is the vinegar-stained image.

22. The device according to claim 20, characterized in that The device comprises: An identification module, specifically used to compare the probability values ​​included in the staining probability distribution of each biological tissue image, and obtain the maximum probability value in the staining probability distribution of each biological tissue image; For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the first element feature position, the staining classification result of the biological tissue image is determined as a non-staining category; For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the second element feature position, the staining classification result of the biological tissue image is determined as a physiological saline category, wherein the biological tissue image belonging to the physiological saline category is the physiological saline image; For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the third element feature position, the staining classification result of the biological tissue image is determined as a vinegar staining category, wherein the biological tissue image belonging to the vinegar staining category is the vinegar staining image; For each biological tissue image, if the maximum probability value in the staining probability distribution of the biological tissue image corresponds to the fourth element characteristic position, the staining classification result of the biological tissue image is determined as an iodine staining category, wherein the biological tissue image belonging to the iodine staining category is an iodine staining image.

23. The device according to any one of claims 17 to 22, characterized in that The device also includes: The acquisition module is further used to acquire a second image sample set, wherein the second image sample set includes at least one second image sample, and each second image sample corresponds to a staining classification label; The acquisition module is further used to acquire, for each second image sample in the second image sample set, a staining probability distribution corresponding to each second image sample based on the first image classification model; The acquisition module is further used to calculate a second loss value based on the staining probability distribution corresponding to each second image sample and the staining classification label; The training module is also used to update the model parameters of the first image classification model using the second loss value.

24. The device according to claim 17, characterized in that The device comprises: A determination module, specifically used to calculate the inter-class distance of the image features of the physiological saline image and the image features of each vinegar-stained image, to obtain the inter-class feature distance between each vinegar-stained image and the physiological saline image; An intra-class distance calculation is performed on the image features of each vinegar-stained image and the corresponding image features of other vinegar-stained images to obtain the intra-class feature distance corresponding to each vinegar-stained image, wherein the other vinegar-stained images are the remaining vinegar-stained images among the P vinegar-stained images except the vinegar-stained image.

25. The device according to claim 24, characterized in that The device comprises: The determination module is specifically configured to calculate the element distance between each element feature in the image features corresponding to the vinegar-stained image and the element feature at the corresponding position in the image features corresponding to the physiological saline image for each vinegar-stained image, so as to obtain M element feature distances between the vinegar-stained image and the physiological saline image, where M is an integer greater than or equal to 1; For each vinegar-stained image, image distance calculation is performed on the M element feature distances between the vinegar-stained image and the physiological saline image to obtain the inter-class feature distance between the vinegar-stained image and the physiological saline image.

26. The device according to claim 24, characterized in that The device comprises: The determination module is specifically used to calculate the element distance between each element feature in the image feature corresponding to the vinegar-stained image and the element features at the corresponding position in the image features corresponding to other vinegar-stained images for each vinegar-stained image, so as to obtain M element feature distances between each vinegar-stained image and other vinegar-stained images, wherein M is an integer greater than or equal to 1; For each vinegar-stained image, performing image distance calculation on M element feature distances between the vinegar-stained image and other vinegar-stained images to obtain a first feature distance between the vinegar-stained image and other vinegar-stained images; For each vinegar-stained image, the first characteristic distances between the vinegar-stained image and other vinegar-stained images are averaged to obtain the intra-class characteristic distance corresponding to the vinegar-stained image.

27. The device according to claim 24, characterized in that The device comprises: A determination module, specifically used to perform feature distance calculation on the image features of the physiological saline image and the image features of each vinegar-stained image, to obtain the image feature distance between each vinegar-stained image and the physiological saline image; Performing feature distance calculation on the histogram features of the physiological saline image and the histogram features of each vinegar-stained image to obtain a histogram feature distance between each vinegar-stained image and the physiological saline image; For each vinegar-stained image, a weighted sum is performed on the image feature distance between the vinegar-stained image and the physiological saline image, and the histogram feature distance between the vinegar-stained image and the physiological saline image to obtain an inter-class feature distance between the vinegar-stained image and the physiological saline image.

28. The device according to claim 27, characterized in that The device comprises: The determination module is specifically configured to calculate the element distance between each element feature in the image features corresponding to the vinegar-stained image and the element feature at the corresponding position in the image features corresponding to the physiological saline image for each vinegar-stained image, so as to obtain M element feature distances between the vinegar-stained image and the physiological saline image, where M is an integer greater than or equal to 1; For each vinegar-stained image, performing image distance calculation on M element feature distances between the vinegar-stained image and the physiological saline image to obtain an image feature distance between the vinegar-stained image and the physiological saline image; A determination module is specifically used to calculate the element distance between each element feature in the histogram feature corresponding to the vinegar-stained image and the element feature at the corresponding position in the histogram feature corresponding to the physiological saline image for each vinegar-stained image, so as to obtain N element feature distances between the vinegar-stained image and the physiological saline image, where N is an integer greater than or equal to 1; For each vinegar-stained image, a histogram distance calculation is performed on N element feature distances between the vinegar-stained image and the physiological saline image to obtain a histogram feature distance between the vinegar-stained image and the physiological saline image.

29. The device according to claim 24, characterized in that The device comprises: A determination module, specifically used to calculate an average distance of image features corresponding to each vinegar-stained image according to the image features of each vinegar-stained image; According to the histogram features of each vinegar-stained image, an average distance of the histogram features corresponding to each vinegar-stained image is calculated; For each vinegar-stained image, a weighted sum is performed on the average distance of image features corresponding to the vinegar-stained image and the average distance of histogram features corresponding to the vinegar-stained image to obtain the intra-class feature distance corresponding to the vinegar-stained image.

30. The device according to claim 29, characterized in that The device comprises: The determination module is specifically used to calculate the element distance between each element feature in the image feature corresponding to the vinegar-stained image and the element features at the corresponding position in the image features corresponding to other vinegar-stained images for each vinegar-stained image, so as to obtain M element feature distances between each vinegar-stained image and the other vinegar-stained images, wherein M is an integer greater than or equal to 1; For each of the vinegar-stained images, performing image distance calculation on M element feature distances between the vinegar-stained image and each of the other vinegar-stained images to obtain a first feature distance between the vinegar-stained image and each of the other vinegar-stained images; For each of the vinegar-stained images, averaging the first characteristic distances between the vinegar-stained image and each of the other vinegar-stained images to obtain an average image characteristic distance corresponding to the vinegar-stained image; The determination module is specifically used to calculate the element distance between each element feature in the histogram feature corresponding to the vinegar-stained image and the element features at the corresponding position in the histogram features corresponding to other vinegar-stained images for each vinegar-stained image, so as to obtain N element feature distances between each vinegar-stained image and the other vinegar-stained images, wherein N is an integer greater than or equal to 1; For each of the vinegar-stained images, a histogram distance calculation is performed on N element characteristic distances between the vinegar-stained image and each of the other vinegar-stained images to obtain a second characteristic distance between the vinegar-stained image and each of the other vinegar-stained images; For each of the vinegar-stained images, the second characteristic distances between the vinegar-stained image and each of the other vinegar-stained images are averaged to obtain the histogram characteristic average distance corresponding to the vinegar-stained image.

31. The device according to any one of claims 27 to 30, characterized in that The device also includes a processing module; A processing module, configured to grayscale the physiological saline image to obtain a physiological saline grayscale image, wherein each pixel in the physiological saline grayscale image corresponds to a brightness value; The processing module is further used to count the number of pixel points corresponding to each preset brightness value based on the physiological saline grayscale image corresponding to the physiological saline image; The processing module is further used to construct a histogram feature of the physiological saline image according to the number of pixels of the physiological saline grayscale image for each preset brightness value; The processing module is further used to perform grayscale processing on each vinegar-dyed image to obtain a vinegar-dyed grayscale image, wherein each pixel in the vinegar-dyed grayscale image corresponds to a brightness value; The processing module is further used to count the number of pixel points corresponding to each preset brightness value for each vinegar-dyed image based on the vinegar-dyed grayscale image; The processing module is further used to construct a histogram feature of the vinegar-stained image according to the number of pixels of the vinegar-stained grayscale image at each preset brightness value for each vinegar-stained image.

32. The device according to claim 17, characterized in that The device also includes a display module; A display module is used for calling a target detection model to detect a target vinegar-stained image, and after obtaining an image detection result of the target vinegar-stained image, if the image detection result indicates that the target vinegar-stained image does not have a biopsy area, displaying a failure prompt message through an operation interface, or sending the image detection result to a terminal so that the terminal displays the failure prompt message; The display module is further used to display at least one biopsy area included in the target aceto-stained image through the operation interface if the image detection result is used to indicate that there is a biopsy area in the target aceto-stained image, or to send the image detection result to the terminal so that the terminal displays at least one biopsy area included in the target aceto-stained image.

33. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the detection method according to any one of claims 1 to 16 are implemented.

34. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the detection method according to any one of claims 1 to 16 are implemented.

35. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the detection method according to any one of claims 1 to 16 are implemented.

Citation Information

Patent Citations

  • Multi-feature deep neural network-based cervical biopsy region identification method and apparatus

    CN108388841A

  • Pathological image processing method and model training method and device based on deep learning

    CN111462036A

  • Pathological picture dyed tissue area detection method, detection system and prognosis state analysis system

    CN113628199A

  • Digital pathological image processing method and device, electronic equipment and storage medium

    CN113763370A

  • Method for predicting focus area in histopathological image

    CN115272196A