Biopsy region priority determination method, device, and apparatus, and storage medium

Abstract

The application discloses a biopsy region priority determination method and device, equipment and a storage medium, and relates to the technical field of artificial intelligence. The method comprises the following steps: acquiring a first image and a second image obtained by using two different methods for the same physiological part; determining the mapping region of n biopsy regions in the first image in the second image according to the position mapping relationship of the physiological part in the first image and the second image; determining the difference information corresponding to the n biopsy regions respectively according to the image content of the n biopsy regions and the mapping regions corresponding to the n biopsy regions respectively; and determining the priority corresponding to the n biopsy regions respectively according to the difference information corresponding to the n biopsy regions respectively. The application automatically determines the priority of the biopsy region through the computer equipment, can avoid determining the priority of the biopsy region by using artificial means, improves the priority determination efficiency of the biopsy region, and reduces the labor cost for determining the biopsy region.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of artificial intelligence, and particularly relate to a biopsy region priority determination method and device, equipment and a storage medium. BACKGROUND

[0002] In medical activities, doctors need to take part of the living tissue of the physiological part of the patient for pathological examination (i.e. biopsy) to exclude the possibility of illness. The position of the living tissue taken by the doctor can be considered as a biopsy region.

[0003] In related technologies, biopsy and determination of biopsy regions are usually performed manually. The order of biopsy regions that need to be biopsied (i.e. the priority of biopsy regions) is also determined by experts in the medical field according to clinical experience.

[0004] However, in the above related technology, the determination of the priority of the biopsy region is too dependent on manual work and is low in efficiency. SUMMARY

[0005] Embodiments of the present application provide a biopsy region priority determination method, device, equipment and storage medium. The technical solution is as follows:

[0006] According to an aspect of an embodiment of the present application, a biopsy region priority determination method is provided, the method comprising:

[0007] obtaining a first image and a second image obtained by using two different ways for the same physiological part;

[0008] determining, according to a position mapping relationship of the physiological part in the first image and the second image, mapping regions of n biopsy regions in the first image in the second image, n being an integer greater than 1;

[0009] determining, according to the n biopsy regions and image contents of the mapping regions corresponding to the n biopsy regions respectively, difference information corresponding to the n biopsy regions respectively, the difference information being used to represent feature differences in image contents between the biopsy regions and the mapping regions corresponding to the biopsy regions respectively;

[0010] determining, according to the difference information corresponding to the n biopsy regions respectively, priorities corresponding to the n biopsy regions respectively, the priorities being used to represent probabilities of pathological changes of the biopsy regions.

[0011] According to an aspect of an embodiment of the present application, a biopsy region priority determination device is provided, the device comprising:

[0012] An image acquisition module is configured to acquire a first image and a second image obtained by using two different manners for the same physiological part;

[0013] A region determination module is configured to determine, according to a location mapping relationship of the physiological part in the first image and the second image, mapping regions of the n biopsy regions in the second image, n being an integer greater than 1;

[0014] A difference determination module is configured to determine, according to the n biopsy regions and image contents of the mapping regions corresponding to the n biopsy regions respectively, difference information corresponding to the n biopsy regions respectively, the difference information being used to represent feature differences in image contents between the biopsy regions and the mapping regions corresponding to the biopsy regions respectively.

[0015] A priority determination module is configured to determine, according to the difference information corresponding to the n biopsy regions respectively, priorities corresponding to the n biopsy regions respectively, the priorities being used to represent probabilities of lesions of the biopsy regions.

[0016] According to an aspect of an embodiment of the present application, a computer device is provided, the computer device comprising a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the above method.

[0017] According to an aspect of an embodiment of the present application, a computer readable storage medium is provided, the readable storage medium storing a computer program, the computer program being loaded and executed by a processor to implement the above method.

[0018] According to an aspect of an embodiment of the present application, a computer program product is provided, the computer program product comprising a computer program stored in a computer readable storage medium. A processor of a computer device reads the computer program from the computer readable storage medium, and the processor executes the computer program, so that the computer device executes the above method.

[0019] The technical scheme provided by the embodiment of the present application can have the following beneficial effects:

[0020] By obtaining first and second images of the same physiological part using different methods, n biopsy regions are determined from the first image. Based on the positional mapping relationship of the physiological part in the first and second images, a corresponding mapping region in the second image is determined for each of the n biopsy regions. Based on the n biopsy regions in the first image and their corresponding mapping regions in the second image, difference information representing the feature differences between the biopsy regions and the mapping regions is determined for each of the n biopsy regions. Based on this difference information, the priority of each of the n biopsy regions is determined. By acquiring two different images, the method provided in this application embodiment can determine the priority information of the biopsy regions. That is, the priority information of the biopsy regions is used to represent the disease probability of the biopsy regions; biopsy regions with higher priority have a higher disease probability and are therefore given a higher priority for biopsy. Therefore, the technical solution provided in this application embodiment automatically determines the priority of biopsy regions using computer equipment, avoiding manual determination of biopsy region priority, improving the efficiency of biopsy region priority determination, and reducing the labor cost of determining biopsy regions. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the implementation environment of a solution provided in one embodiment of this application;

[0022] Figure 2 This is a block diagram of a method for determining the priority of biopsy regions based on colposcopy images, provided in one embodiment of this application;

[0023] Figure 3 This is a flowchart of a method for determining the priority of a biopsy region according to an embodiment of this application;

[0024] Figure 4 This is a flowchart of a method for determining the priority of a biopsy region provided in another embodiment of this application;

[0025] Figure 5 This is a schematic diagram of an image classification model provided in one embodiment of this application;

[0026] Figure 6 This is a block diagram of a method for calculating feature differences provided in one embodiment of this application;

[0027] Figure 7 This is a flowchart of a method for determining the priority of a biopsy region provided in another embodiment of this application;

[0028] Figure 8 This is a schematic diagram of a cervical os detection network provided in one embodiment of this application;

[0029] Figure 9 This is a schematic diagram of a biopsy region detection model provided in one embodiment of this application;

[0030] Figure 10 is a block diagram of a method for determining a mapping region according to an embodiment of the present application;

[0031] Figure 11 is a schematic diagram of determining a coordinate system according to an embodiment of the present application;

[0032] Figure 12 is a schematic diagram of sub-image interception according to an embodiment of the present application;

[0033] Figure 13 is a block diagram of a method for determining a priority of a biopsy region according to an embodiment of the present application;

[0034] Figure 14 is a block diagram of an apparatus for determining a priority of a biopsy region according to an embodiment of the present application;

[0035] Figure 15 is a block diagram of an apparatus for determining a priority of a biopsy region according to another embodiment of the present application;

[0036] Figure 16 is a structural block diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical scheme and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0038] Before introducing the technical scheme of the present application, some background technical knowledge related to the present application will be introduced and explained. The following related technologies can be combined with the technical scheme of the embodiments of the present application in any way as optional schemes, which all belong to the protection scope of the embodiments of the present application. The embodiments of the present application include at least part of the following contents.

[0039] Artificial intelligence (AI) is to use digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which tries to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to design and implement principles and methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0040] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include natural language processing and machine learning / deep learning.

[0041] Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in recognizing and measuring targets, and then performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0042] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learn-by-doing.

[0043] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, and smart customer service. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0044] The solutions provided in this application involve technologies such as computer vision and machine learning in artificial intelligence, which are specifically illustrated through the following embodiments.

[0045] Before introducing the technical solutions of this application, some terms involved in this application will be explained. The following related explanations are optional solutions and can be arbitrarily combined with the technical solutions of the embodiments of this application, all of which fall within the protection scope of the embodiments of this application. The embodiments of this application include at least some of the following contents.

[0046] Deep learning: The concept of deep learning originated from research on artificial neural networks. A multilayer perceptron with multiple hidden layers is a type of deep learning architecture. Deep learning discovers distributed feature representations of data by combining low-level features to form more abstract high-level representations of attribute categories or features.

[0047] Convolutional Neural Networks (CNNs) are a type of deep feedforward neural network that incorporates convolutional computations. They are one of the representative algorithms in deep learning. CNNs possess representation learning capabilities, enabling translation-invariant classification of input information according to their hierarchical structure. Generally, the basic structure of a CNN consists of two layers: a feature extraction layer, where the input of each neuron is connected to the local receptive field of the previous layer, extracting local features. Once these local features are extracted, their positional relationships with other features are determined; and a feature mapping layer, where each computational layer consists of multiple feature maps, each a plane where all neurons have equal weights. A CNN comprises an input layer, convolutional layers, activation functions, pooling layers, and fully connected layers.

[0048] Deep Residual Networks (DRNs) are currently the most widely used CNN feature extraction networks, used to address gradient explosion or vanishing gradients caused by excessively deep network layers.

[0049] Biopsy area: This refers to the area in medical procedures where a biopsy is performed. A biopsy, or simply biopsy, involves taking a sample of diseased tissue from a patient for pathological examination to assist clinicians in determining the disease. For example, a cervical biopsy involves taking a small piece or several pieces of tissue from the cervix for pathological examination. It is a relatively routine examination method in modern medical practice, providing a basis for subsequent diagnosis.

[0050] Colposcope images: A colposcope is a gynecological clinical diagnostic instrument and one type of gynecological endoscope. It is suitable for diagnosing various cervical diseases and genital lesions, and is also an important method for the early diagnosis of male and female sexual diseases. For example, the acetic acid-stained images and saline images obtained from the processing of the cervix in this application can be considered colposcope images.

[0051] Cervical biopsy: A cervical biopsy is a live tissue examination of the cervix, that is, taking a small piece or several pieces of tissue from the cervix for pathological examination to confirm the diagnosis.

[0052] Sensitivity: The proportion of images of precancerous lesions that are detected as precancerous lesions out of all images of precancerous lesions.

[0053] Specificity: The proportion of all normal images that are detected as normal.

[0054] Image category: The category to which the image content belongs, i.e., normal or precancerous lesions.

[0055] Detection Network / Model: A mathematical model obtained by machine learning techniques after learning from labeled sample data (images - the correspondence between target positions in the image). The parameters of the mathematical model are obtained during the learning and training process. During detection, the parameters of the mathematical model are loaded and the detection position of the target in the input sample is calculated.

[0056] Classification network / model: A mathematical model obtained by machine learning techniques after learning from labeled sample data (image-category correspondence). The parameters of the mathematical model are obtained during the learning and training process. When recognizing and predicting, the parameters of the mathematical model are loaded and the probability of the input sample belonging to each category is calculated.

[0057] EMD (earth mover's distances): Measures the distance between two distributions. Given two distributions, one can be viewed as a spatially distributed mound of earth, and the other as a spatially distributed hole. EMD measures the minimum amount of work required to fill these holes with the earth. For example, an image can be viewed as composed of three components: hue, saturation, and brightness. The histogram of each component is a distribution. Different images correspond to different histograms, therefore the distance between images can be represented by histogram distance, which can then be calculated using EMD.

[0058] ResNet (Residual Neural Network): Traditional convolutional or fully connected networks suffer from information loss and degradation during information transmission, leading to vanishing or exploding gradients, making deep networks untrainable. ResNet addresses this issue to some extent by directly routing input information to the output, preserving information integrity. The entire network only needs to learn the difference between the input and output, simplifying the learning objective and reducing the difficulty.

[0059] It should be noted that this application may display a prompt interface, pop-up window, or output voice prompt information before and during the collection of user data. This prompt interface, pop-up window, or voice prompt information is used to inform the user that their data is being collected. This ensures that the application only begins the steps for collecting user data after receiving confirmation from the user regarding the prompt interface or pop-up window; otherwise (i.e., without confirmation from the user), the steps for collecting user data end, meaning no user data is collected. In other words, all user data collected in this application is collected with the user's consent and authorization, and the collection, use, and processing of related user data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. Specifically, the acquisition of the first and second images involved in the embodiments of this application is carried out with the user's (patient's) consent and authorization, and the collection, use, and processing of related images must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0060] Please refer to Figure 1 This diagram illustrates an implementation environment for a solution provided in one embodiment of this application. The implementation environment may include: a terminal device 10 and a server 20.

[0061] Terminal device 10 includes, but is not limited to, mobile phones, tablets, smart voice interaction devices, game consoles, wearable devices, multimedia playback devices, PCs (Personal Computers), in-vehicle terminals, smart home appliances, and other electronic devices. The client for the target application can be installed on terminal device 10.

[0062] In this embodiment, the target application described above can be used to determine the priority of biopsy areas. Typically, this application is an application for determining the priority of biopsy areas. Of course, in addition to applications for determining the priority of biopsy areas, other types of applications can also perform the priority determination of biopsy areas. For example, Virtual Reality (VR) applications, Augmented Reality (AR) applications, simulation applications, etc., are not limited in this embodiment. In addition, different applications determine the physiological sites for biopsy area priority, which can be the cervix, or other physiological sites such as the lungs or stomach. Optionally, the terminal device 10 runs a client of the above-mentioned application.

[0063] Server 20 is used to provide backend services for the client of the target application in terminal device 10. For example, server 20 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms, but it is not limited to these.

[0064] Terminal device 10 and server 20 can communicate with each other via a network. This network can be a wired network or a wireless network.

[0065] The method provided in this application embodiment can be executed by a computer device in each step. The computer device can be any electronic device capable of data storage and processing. For example, the computer device can be... Figure 1 Server 20 in the middle can be Figure 1 The terminal device 10 can also be another device other than the terminal device 10 and the server 20.

[0066] Please refer to Figure 2 The diagram illustrates a block diagram of a method for prioritizing biopsy regions based on colposcopy images, provided in one embodiment of this application.

[0067] In related technologies, the automatic diagnosis of precancerous lesions of the cervix based on colposcopy images is still in its early stages and remains at the level of predicting the probability of disease. Colposcopy biopsy point prediction often only outputs the location of the biopsy point without providing priority. That is, there is a determination of the biopsy area, but no way to determine the priority of the biopsy area. On the one hand, only the prediction of biopsy points (in the embodiments of this application, the biopsy point can be considered as the biopsy area) is provided, and there is no priority difference between the prediction results. If more biopsy points are predicted than the doctor's judgment, the doctor may not perform biopsies according to the prediction results, which may lead to missing the biopsy sampling of the most serious lesions. On the other hand, the severity of the biopsy area needs to be compared with the corresponding position of the acetic acid stained image and the saline image, and this comparison is done manually. Related technologies lack effective automatic location matching and automatic feature comparison schemes.

[0068] In some embodiments, the physiological site is the cervix. A colposcope is used to obtain acetic acid-stained images and saline images by processing the cervix with either acetic acid or saline solution. As shown in the block diagram of the cervical biopsy region priority determination method in Figure 200, a saline image and an acetic acid-stained image of the cervix obtained through colposcopy are acquired. The acetic acid-stained image is processed by a biopsy region detection model and a cervical os detection network to obtain predicted biopsy regions and predicted cervical os. The saline image is processed by a cervical os prediction and detection network to obtain predicted cervical os. The positional relationship between the biopsy point in the acetic acid-stained image and the cervical os in the acetic acid-stained image is calculated. Based on this positional relationship, a mapping region corresponding to the predicted biopsy region in the acetic acid-stained image is matched in the saline image. The predicted biopsy region in the acetic acid-stained image and the matched biopsy mapping region in the saline image are extracted from the acetic acid-stained image and the saline image, respectively, to form acetic acid-stained sub-images and saline sub-images. An image classification model is used to predict the features of acetic acid-stained and saline-stained biopsy images, and the features output from the last convolutional layer are taken as the features of the sub-image. When calculating the feature differences between acetic acid-stained and saline-stained biopsy images, a one-to-many approach is adopted, calculating the feature differences between each acetic acid-stained biopsy image and all saline-stained biopsy images, and taking the average as the feature difference of that acetic acid-stained biopsy image. Optionally, the EMD distance between the features of paired acetic acid-stained and saline-stained biopsy images is calculated. A larger EMD distance indicates a greater difference between the acetic acid-stained biopsy image and the matched saline-stained biopsy image. A larger feature difference indicates a greater difference between the acetic acid-stained biopsy image and the saline-stained biopsy image, meaning a greater variation in the predicted result for that biopsy region, and thus a higher biopsy priority. Of course, this is not limited to calculating the EMD distance; other reasonable methods for calculating feature differences are also within the scope of protection of this application.

[0069] The technical solution provided in this application can be applied to the biopsy area prediction scheme of colposcopy, and the biopsy area has priority indication, allowing clinicians to perform more appropriate biopsies based on the patient's condition and biopsy point priority. On one hand, the technical solution provided in this application proposes using feature distance to quantify the differences between acetic acid-stained images and saline images, overcoming the robustness challenges posed by different device image color temperatures and contrasts. On the other hand, the technical solution provided in this application proposes a matching scheme for the corresponding positions of biopsy points in acetic acid-stained images and saline images. The biopsy point represents the area of ​​epithelial change after acetic acid staining, and position matching is a prerequisite for change comparison. Furthermore, the technical solution provided in this application improves the extraction capability of acetic acid-stained epithelial features by using a CNN to train saline and acetic acid-stained images for classification tasks, and then using the trained CNN for feature learning. The technical solution provided in this application can quickly and effectively provide doctors and specialists with references for biopsy areas for hospitals and private clinics. Besides identifying cervical precancerous lesions and predicting the priority of biopsy areas, different detection and identification tasks can be achieved by changing different types of datasets, thereby enabling the determination of the priority of biopsy areas for different physiological sites. Furthermore, the technical solution provided in this application embodiment can improve the accuracy of biopsy area priority determination. Since the priority of the biopsy area is calculated entirely automatically, the relative error is small. Compared to errors in biopsy area priority determination caused by varying physician skill levels, the technical solution provided in this application embodiment relatively improves the accuracy of biopsy area priority determination.

[0070] In this application, the specific type of physiological site is not limited, nor is the processing method for the physiological site limited. It can be an image obtained by acetic acid staining or an image obtained by treating the physiological site with saline solution. Furthermore, besides a colposcope, different medical endoscopes can be used to obtain images of the corresponding physiological sites, such as an endoscope to obtain images of the stomach, or an endoscope to obtain images of the intestines. Of course, this application does not limit the processing method for the physiological site; images obtained using other processing methods, such as iodine staining, can also be used.

[0071] Please refer to Figure 3 This document illustrates a flowchart of a method for determining the priority of a biopsy region according to an embodiment of this application. The entity executing each step of this method may be... Figure 1 The terminal device 10 or server 20 in the implementation environment of the illustrated scheme. In the following method embodiments, for ease of description, only the execution subject of each step is described as a "computer device". The method may include at least one of the following steps (310-340):

[0072] Step 310: Obtain a first image and a second image obtained using two different methods for the same physiological site.

[0073] Physiological site: A physiological site of a living organism or a non-living organism. Optionally, the object of the physiological site includes, but is not limited to, at least one of humans, animals, and plants. Optionally, the physiological site is at least one of any tissue or organ such as the stomach, lungs, intestines, or heart. Optionally, the physiological site is a complete tissue or organ or a part of a tissue or organ. In this application embodiment, the cervix is ​​mainly used as an example of a physiological site for the following explanation. The method for determining the priority of biopsy areas of other physiological sites is the same as the method for determining the priority of biopsy areas of the cervix, and will not be repeated here.

[0074] In some embodiments, the first image corresponds to a first processing method for a physiological part, and the second image corresponds to a second processing method for a physiological part. The first processing method and the second processing method are different processing methods.

[0075] In some embodiments, the first processing method is to change the original display style of the physiological part. Optionally, the first processing method is to treat with acetic acid or iodine, and the first image is an image obtained by treating the physiological part with acetic acid or iodine. The technical solution provided in this application embodiment can effectively extract the white epithelial features of the physiological part by obtaining a second image of the physiological part, and these white epithelial features can characterize the pathological condition of the area to a certain extent.

[0076] In some embodiments, the second processing method is a method that does not change the original display style of the physiological part. Optionally, the second processing method is to use physiological saline treatment, and the second image is an image obtained by treating the physiological part with physiological saline. The technical solution provided in this application embodiment, by using physiological saline to treat the physiological part, can make the representation of the physiological part clearer and more valuable for comparison without changing the original morphology of the physiological part.

[0077] This application does not limit the image acquisition method, and it can be at least one of medical endoscopes or exoscopes, including but not limited to colposcopes, colonoscopes, gastroscopes, cystoscopes, laparoscopes, and microscopes. In some embodiments, taking a microscope as an example, the acquisition method can be a pathological examination (autopsy) of a non-living physiological part, such as taking a physiological part sample and obtaining a first image and a second image of the same physiological part sample through two processing methods using a microscope. According to the technical solution provided in this application, the area most likely to be a lesion can be predicted, and further examination can be carried out.

[0078] Step 320: Based on the positional mapping relationship of physiological sites in the first and second images, determine the mapping regions of n biopsy areas in the first image in the second image, where n is an integer greater than 1.

[0079] In some embodiments, step 320 is preceded by determining n biopsy regions in the first image. In some embodiments, the first image obtained using the first processing method is used to determine n biopsy regions in the first image. Taking an acetic acid-stained image as an example, when acetic acid is used to treat physiological sites, the reaction between acetic acid and the physiological sites results in acetic acid-white epithelium on the physiological parts of the sites. The presence of acetic acid-white epithelium in these sites indicates a potential lesion. Therefore, based on this characteristic of acetic acid-stained images, n biopsy regions can be determined from the acetic acid-stained image.

[0080] In some embodiments, the physiological components have different locations in the first image and the second image, and there is a certain mapping relationship between them. Based on the positions of the n biopsy regions in the first image, the mapped regions of the n biopsy regions in the second image are determined.

[0081] In some embodiments, taking the cervix as an example where the acquisition method is colposcopy and the physiological site is the cervix, since the cervix needs to be processed twice to obtain a first image and a second image respectively, it cannot be guaranteed that the angles of the two images obtained by colposcopy are completely consistent. However, even if the acquisition angles of the two images are not completely consistent, the relative position and relative size of the two images can be determined by using a reference object. In some embodiments, taking the cervical os as an example, since the position of the cervical os is relatively fixed, the relative position of the biopsy area in the second image can be deduced from the position of the biopsy area relative to the cervical os in the first image and the position of the cervical os in the second image. The size of the biopsy area in the second image can be determined based on the relative size of the cervical os in the first image and the cervical os in the second image. For other physiological sites, please refer to the explanation of the cervix.

[0082] Step 330: Based on the image content of the n biopsy regions and their corresponding mapping regions, determine the difference information corresponding to the n biopsy regions. The difference information is used to characterize the feature differences between the biopsy regions and their corresponding mapping regions in the image content.

[0083] Mapped region: This is the region on the second image that corresponds to the biopsy region in the first image. Optionally, for each biopsy region in the first image, there is a corresponding mapped region. Optionally, if there are n biopsy regions in the first image, then there are n corresponding mapped regions in the second image.

[0084] In some embodiments, the positions of n mapped regions in the second image need to be determined based on the positions of n biopsy regions in the first image. In other embodiments, the n mapped regions in the second image are directly determined using a biopsy region detection model, which is a neural network model.

[0085] In some embodiments, the difference information corresponding to the n biopsy regions in the first image is determined based on the n biopsy regions in the first image and the mapping regions in the second image corresponding to the n biopsy regions.

[0086] In some embodiments, the image content of the same biopsy region corresponding to the first image and the second image is not the same. Optionally, the difference information of the i-th biopsy region is determined based on the image content of the i-th biopsy region among the n biopsy regions in the first image and the image content of the n biopsy regions in the second image, where i is a positive integer not greater than n. That is, by comparing the image content of the i-th biopsy region in the first image with the image content of the n biopsy regions in the second image, the difference information of the i-th biopsy region is obtained, and then the difference information corresponding to each of the n biopsy regions is obtained.

[0087] In some embodiments, features corresponding to the image content are extracted using a neural network model. In some embodiments, the feature differences between the i-th biopsy region and other regions are determined based on the features of the image content of the i-th biopsy region among the n biopsy regions in the first image and the features corresponding to the image content of the n biopsy regions in the second image.

[0088] Step 340: Based on the difference information corresponding to each of the n biopsy regions, determine the priority of each of the n biopsy regions. The priority is used to characterize the lesion probability of the biopsy region.

[0089] The difference information includes, but is not limited to, numerical and vector representations. In some embodiments, taking numerical differences as an example, the biopsy areas with the largest differences are considered to have the highest priority, indicating the highest probability of lesions and should be biopsied first. The biopsy areas with the smallest differences are considered to have the lowest priority, indicating the lowest probability of lesions and may not be prioritized for biopsy. The difference information characterizes the difference between this area and other areas, thereby characterizing the likelihood of lesions in that area. Optionally, the larger the difference, the greater the likelihood of lesions in that area.

[0090] The technical solution provided in this application uses first and second images of the same physiological part obtained through different methods. It identifies n biopsy regions from the first image and determines the corresponding mapping regions in the second image based on the positional mapping relationship of the physiological part in the first and second images. Based on the n biopsy regions in the first image and the corresponding mapping regions in the second image, it determines the difference information representing the feature differences between the biopsy regions and the mapping regions. Based on the difference information corresponding to each of the n biopsy regions, it determines the priority of each biopsy region. By acquiring two different images, the method provided in this application can determine the priority information of the biopsy regions. That is, the priority information of the biopsy regions represents the probability of disease occurrence in the biopsy regions; biopsy regions with higher priority have a higher probability of disease occurrence and are therefore given a higher priority for biopsy. Therefore, the technical solution provided in this application automatically determines the priority of biopsy regions through computer equipment, avoiding manual determination of biopsy region priority, improving the efficiency of biopsy region priority determination, and reducing the labor cost of determining biopsy regions.

[0091] Please refer to Figure 4 This illustrates a flowchart of a method for prioritizing biopsy regions according to another embodiment of this application. The entity executing each step of this method may be... Figure 1 The terminal device 10 or server 20 in the implementation environment of the illustrated scheme. In the following method embodiments, for ease of description, only the execution subject of each step is described as a "computer device". The method may include at least one of the following steps (310-340):

[0092] Step 310: Obtain a first image and a second image obtained using two different methods for the same physiological site.

[0093] Step 320: Based on the positional mapping relationship of physiological sites in the first and second images, determine the mapping regions of n biopsy areas in the first image in the second image, where n is an integer greater than 1.

[0094] In some embodiments, physiological sites are present in both the first and second images, but due to different acquisition angles, the positions of these physiological sites differ between the first and second images. Therefore, there is a positional mapping relationship between the physiological sites in the first and second images. Taking the cervix as an example, the positional mapping relationship of the cervix in the first and second images can be determined using the cervical opening as a reference.

[0095] In some embodiments, after determining n biopsy regions in the first image, the mapping regions of the n biopsy regions in the first image in the second image are determined according to the positional mapping relationship of physiological sites in the first and second images.

[0096] Step 331: Extract the image content of n biopsy regions from the first image to obtain n first sub-images. The i-th biopsy region in the n biopsy regions corresponds to the i-th first sub-image in the n first sub-images, where i is a positive integer less than or equal to n.

[0097] In some embodiments, after determining n biopsy regions in the first image, image content of n biopsy regions is extracted from the first image according to their positions. Taking n as 3 as an example, image content of 3 biopsy regions is extracted from the first image to obtain 3 first sub-images.

[0098] Step 332: Extract the image content of the mapped regions corresponding to the n biopsy regions from the second image to obtain n second sub-images.

[0099] In some embodiments, a first sub-image is determined based on the image content of the biopsy region in a first image, and a second sub-image is determined based on the image content of the mapped region of the biopsy region in a second image. This application embodiment does not limit the number of first and second sub-images.

[0100] In some embodiments, taking n=3 as an example, after extracting 3 first sub-images from the first image, 3 second sub-images are determined based on the mapping region in the second image corresponding to the biopsy region in the first image.

[0101] In some embodiments, step 332-1 (not shown in the figure) is included before step 333.

[0102] Step 332-1: Obtain feature information of the first sub-image and feature information of the second sub-image through the feature extraction network of the image classification model; wherein, the image classification model is used to classify whether the biopsy area is a lesion.

[0103] In some embodiments, the feature information is a feature vector in vector form. In some embodiments, the feature extraction network is a network for extracting image features; optionally, the feature extraction network is ResNet50. In some embodiments, ResNet50 is used to obtain the feature vector of the first sub-image.

[0104] In some embodiments, the image classification model is a model trained to correctly classify images. Optionally, the image classification model is a neural network model.

[0105] In some embodiments, the training process of the image classification model is as follows:

[0106] The feature extraction network is used to obtain the feature information corresponding to the first image training sample and the second image training sample obtained by two different methods for the same physiological site. The feature fusion network of the image classification model is used to fuse the feature information corresponding to the first image training sample and the second image training sample to obtain fused feature information. The classification network of the image classification model is used to obtain multiple lesion detection categories corresponding to the fused feature information and the probabilities corresponding to different lesion detection categories. The parameters of the image classification model are adjusted according to the multiple lesion detection categories corresponding to the fused feature information and the probabilities corresponding to different lesion detection categories, as well as the first label corresponding to the first image sample, which includes the first lesion detection category.

[0107] In some embodiments, the first image training sample is an acetic acid-stained image, and the second image training sample is a saline image. In some embodiments, the image classification model is used to classify cervical precancerous lesions from acetic acid-stained and saline images of the same examination, unlike other classification models. In some embodiments, the input to the image classification model is multiple time-series images from the same examination, including one saline image and multiple acetic acid-stained images. Specifically, four acetic acid-stained images at 60s, 90s, 120s, and 150s are used.

[0108] In some embodiments, such as Figure 5As shown, the image classification model consists of three parts: a backbone convolutional network or feature extraction network 500. Optionally, the feature extraction network is ResNet50. ResNet50 as the backbone for feature extraction does not include average pooling, 1000-dfc, and softmas. That is, the feature extraction network extracts the feature information from the first training sample and the second training sample. Optionally, the number of second training samples is not limited to one. Optionally, the feature information is a feature vector. A feature fusion layer or feature fusion network 510, typically a concatenate layer, concatenates the CNN features of multiple images. According to the feature fusion network of the image classification model, multiple feature information is fused to obtain fused feature information. Optionally, the fused feature information is a feature vector. Finally, a fully connected (FC) layer, or classification network 520, outputs whether a lesion is present. In some embodiments, the fully connected layer is a classifier; by operating on the parameters of the fully connected layer, the probabilities of different categories can be obtained. In some embodiments, the categories include normal and lesion. Optionally, the fused feature vector is processed by the classification network 520 to obtain the probability of belonging to the normal category and the probability of belonging to the lesion category.

[0109] In the image classification model provided in this application embodiment, each input during the training process needs to include a first image and a second image. The order of the number of first images can be changed according to data augmentation requirements to improve the classification ability of the image classification model, thereby improving the feature extraction ability of CNN.

[0110] Step 333: For the i-th biopsy region, obtain the difference information corresponding to the i-th biopsy region based on the feature differences between the i-th first sub-image and the n second sub-images.

[0111] In some embodiments, feature information corresponding to the first sub-image and the second sub-image is obtained respectively. Based on the feature information, the feature differences between the i-th first sub-image and the n-th second sub-image are determined, and the difference information corresponding to the i-th biopsy region is obtained.

[0112] In some embodiments, step 333 includes steps 333-1 to 333-2 (not shown in the figure).

[0113] Step 333-1: Obtain the difference between the feature information of the i-th first sub-image and the feature information of the n second sub-images respectively, and obtain n difference results.

[0114] In some embodiments, the feature information is a feature vector. A feature extraction network is used to obtain the feature vectors corresponding to each sub-image.

[0115] Step 333-2: Based on the n difference results, determine the difference information corresponding to the i-th biopsy region.

[0116] In some embodiments, for the i-th biopsy region, the acetic acid staining image features need to be compared with the features of all saline smear images, and the average value is taken as the feature difference of the acetic acid staining image. Taking the mean squared error as an example, the feature difference calculation is performed on the i-th acetic acid staining image:

[0117]

[0118] Where n represents the number of biopsy regions, and m represents the number of elements in the sub-image feature. F i Let f represent the feature vector of the i-th acetylene sub-image. i This represents the saline ion image corresponding to the j-th acetylcholine chromatogram. Of course, besides calculating the mean square error, the squared difference of features, root mean square, etc., can also be calculated; this application does not limit this calculation.

[0119] In some embodiments, such as Figure 6 As shown, 60 represents the n first sub-images corresponding to the n biopsy regions, and 61 represents the n second sub-images corresponding to the n biopsy regions. Optionally, the feature information corresponding to the first sub-image and the second sub-image is obtained through an image classification model. Optionally, taking the first sub-image corresponding to the first biopsy region as an example, the feature difference between the feature information F1 of the first sub-image and the feature information f1, f2, and f3 corresponding to the n second sub-images is calculated, thereby obtaining the difference information corresponding to the first biopsy region, and then obtaining the difference information corresponding to the n biopsy regions.

[0120] Step 340: Based on the difference information corresponding to each of the n biopsy regions, determine the priority of each of the n biopsy regions. The priority is used to characterize the lesion probability of the biopsy region.

[0121] The technical solution provided in this application proposes a function for prioritizing biopsy regions, providing doctors with parameters for selecting biopsy regions. Furthermore, the technical solution provided in this application also proposes using feature distance to quantify the differences between the first and second images. Since the extracted high-dimensional features have the ability to distinguish between images unaffected by color temperature and contrast, it can overcome the challenges to robustness posed by the color temperature and contrast of images from different devices.

[0122] Please refer to Figure 7 This illustrates a flowchart of a method for prioritizing biopsy regions according to another embodiment of this application. The entity executing each step of this method may be...Figure 1 The terminal device 10 or server 20 in the implementation environment of the illustrated scheme. In the following method embodiments, for ease of description, only the execution subject of each step is described as a "computer device". The method may include at least one of the following steps (310-340):

[0123] Step 310: Obtain a first image and a second image obtained using two different methods for the same physiological site.

[0124] Step 321: Detect the physiological parts contained in the first image to obtain a first detection box containing the physiological parts.

[0125] In some embodiments, a physiological site detection network is used to detect the cervix in the first and second images. In some embodiments, the physiological site detection network is also trained using samples and manually or automatically labeled data, enabling it to label the regions containing physiological sites (i.e., the first and second detection boxes) within the images. In some embodiments, the physiological site is the cervix, so a cervical os detection network is used to detect the cervix in the first and second images, such as... Figure 8 As shown, a cervical os detection network 800 is used to annotate the cervix in the image to obtain cervical os annotation information. In some embodiments, if the physiological part is not the cervix, a other physiological part detection network is used to detect other physiological parts in the first image and the second image.

[0126] Step 322: Detect the physiological regions contained in the second image to obtain a second detection box containing the physiological regions. A cervical os detection network is used to detect the cervical os region in acetic acid-stained images and saline images; a typical network is Yolo V3.

[0127] In some embodiments, step 322-1 (not shown in the figure) is included before step 323.

[0128] Step 322-1: Process the first image using the biopsy region detection model to obtain n biopsy regions in the first image; wherein, the loss function of the biopsy region detection model during training includes the looseness loss, which is used to measure the positional relationship between biopsy regions.

[0129] In some embodiments, the biopsy region detection model is a model for detecting biopsy regions in an image. Optionally, this model is also trained using sample images and manually / automatically labeled biopsy regions to output the correct biopsy regions for the input image. Figure 9As shown, the biopsy region detection model 900 is used to annotate the biopsy region in the first image to obtain biopsy point annotation information / biopsy region annotation information.

[0130] In this embodiment, a biopsy region detection model is directly used to detect the biopsy region in the first image, which firstly reduces the time spent determining the biopsy region, and secondly improves the accuracy of the determined biopsy region.

[0131] Biopsy region detection models are used to detect biopsy regions in acetic acid-stained images, with typical networks such as YOLOv3. In some embodiments, since biopsy regions need to be relatively dispersed, region looseness is introduced into the loss function during training, where γ is a looseness control parameter.

[0132] Loss = Loss det +γ·Loss dis

[0133] Where Loss represents the total loss function, Loss dis This indicates the loss of looseness. det The loss function is the original loss function of the biopsy region detection model, typically such as YOLOv3, Loss. det as follows:

[0134] Loss det =Loss xy +Loss wh +Loss cls +Loss conf

[0135] Among them, Loss xy This represents the loss value of the detection task in the xy coordinate system. wh This represents the loss value for the detection task in width and height. cls Loss represents the loss value of the detection task in classification. conf This represents the loss value of the detection task in predicting probabilities.

[0136] Loss dis The looseness of the predicted biopsy regions is measured by subtracting the product of the sum of the distances between any two points in the predicted biopsy regions and the total number of combinations of those two points, multiplied by the length of the image diagonal, from 1, as shown below. The higher the looseness between the predicted biopsy regions, the lower the loss. dis The lower.

[0137]

[0138] Where m represents the number of biopsy areas, x i This represents the coordinate of the i-th biopsy region on the x-axis, x jThis represents the y-coordinate of the i-th biopsy region on the y-axis. i The x-coordinate of the j-th biopsy region is represented by the x-coordinate, and the y-coordinate by the x-coordinate. j Let represent the coordinates of the j-th biopsy region on the y-axis.

[0139] The technical solution provided in this application introduces a looseness loss function to determine a new loss function. Generally, biopsies are performed in the most severely affected areas of the physiological site, and biopsies are performed on different lesion areas, but multiple biopsies cannot be performed on the same severely affected area. Therefore, increasing the regional looseness allows the predicted biopsy areas to be as dispersed as possible, avoiding multiple predictions for the same lesion area. Thus, by introducing a looseness loss, the determined biopsy areas are more dispersed, reflecting reality.

[0140] Step 323: Based on the relative positional relationship between the n biopsy regions in the first image and the first detection box, and the coordinate mapping relationship between the first detection box and the second detection box, determine the mapping region of the n biopsy regions in the second image, where n is an integer greater than 1.

[0141] In some embodiments, taking the cervix as an example, the positional relationship between the biopsy area in the acetic acid-stained image and the cervical os (first detection box) in the acetic acid-stained image is calculated. Based on this positional relationship, the region corresponding to the predicted biopsy area in the acetic acid-stained image is matched in the saline image—the matched biopsy area, or mapping.

[0142] In some embodiments, a mapping rule is calculated based on the cervical os detection results (first detection box) of the acetic acid-stained image and the cervical os detection results (second detection box) of the saline image. The image of the acetic acid-stained image is then mapped according to the calculated mapping rule to obtain the biopsy area corresponding to the saline image.

[0143] like Figure 10 As shown, based on the predicted biopsy area and the cervical os detection box on the acetic acid stained image 1000, the mapping area of ​​the biopsy area on the saline image 1010 is determined.

[0144] In some embodiments, step 323 includes steps 323-1 to 323-3 (not shown in the figures).

[0145] Step 323-1: For the i-th biopsy region among the n biopsy regions, determine the positional deviation between the i-th biopsy region and the first detection frame.

[0146] Step 323-2: Based on the position information of the second detection frame and the position information of the first detection frame, determine the size ratio of the second detection frame to the first detection frame.

[0147] Step 323-3: Based on the size ratio of the second detection box to the first detection box, the positional deviation between the i-th biopsy region and the first detection box, and the positional information of the second detection box, determine the mapping region of the i-th biopsy region in the second image.

[0148] In some embodiments, the detection result is assumed to be in the format (x, y, w, h), representing the coordinates of the center point of the detection box in the image coordinate system and the width and height of the detection box, respectively. The image coordinate system has its origin at the lower left corner of the image, with the x-axis horizontal and the y-axis vertical. Additionally, a target coordinate system is established with the center of the cervical os detection box as its origin, the x-axis horizontal, and the y-axis vertical. Figure 11 As shown, based on the acetic acid staining image coordinate system 1110, the acetic acid staining target coordinate system 1120 is established, and based on the physiological saline image coordinate system 1130, the physiological saline target coordinate system 1140 is established.

[0149] In other embodiments, a first sub-image and a second sub-image are cropped based on the position of the biopsy region in the first image and the position of the mapped region in the second image. Optionally, sub-images are formed by cropping from the two images respectively, based on the biopsy region detected in the acetic acid image and the region matched in the saline image. Figure 12 As shown, based on the first image and the second image, multiple first sub-images 1200 and multiple second sub-images 1210 are determined.

[0150] In some embodiments, the coordinates of the detection result of the i-th biopsy region in the acetic acid-stained image coordinate system are defined as biopsy. ori_aa_i The corresponding coordinates in the target coordinate system of the acetic acid staining image are biopsy aa_i The calculation method is as follows:

[0151] biopsy aa_i (x)=biopsy ori_aa_i (x)-center ori_aa (x)

[0152] biopsy aa_i (y) = biopsy ori_aa_i (y)-center ori_aa (y)

[0153] Among them, center ori_aa Let be the coordinates of the cervical os detection box in the acetic acid-stained image coordinate system. Similarly, define 'center'. ori_ns The coordinates of the cervical os detection box in the saline image are given in the saline image coordinate system. (biopsy) ori_aa_i (x)-center ori_aa(x) represents the positional deviation in the x-direction between the i-th biopsy region and the first detection frame. ori_aa_i (y)-center ori_aa (y) represents the positional deviation in the y-direction between the i-th biopsy region and the first detection frame.

[0154] biopsy ns_i (x)=biopsy aa_i (x)·ratio x +center ori_ns

[0155] biopsy ns_i (y) = biopsy aa_i (y)·ratio x +center ori_ns

[0156] Among them, biopsy ns_i Let be the coordinates of the center of the i-th biopsy region in the saline image coordinate system.

[0157] ratio x =width ns / width aa

[0158] ratio y =height ns / height aa

[0159] Where, ratio x and ratio y The ratios are the horizontal and vertical side lengths of the cervical os detection box in the saline image and the acetic acid-stained image, respectively. x The ratio represents the size ratio of the second detection box to the first detection box in the x-direction. y This indicates the size ratio of the second detection frame to the first detection frame in the y-direction.

[0160] Step 330: Based on the image content of the n biopsy regions and their corresponding mapping regions, determine the difference information corresponding to the n biopsy regions. The difference information is used to characterize the feature differences between the biopsy regions and their corresponding mapping regions in the image content.

[0161] Step 340: Based on the difference information corresponding to each of the n biopsy regions, determine the priority of each of the n biopsy regions. The priority is used to characterize the lesion probability of the biopsy region.

[0162] The technical solution provided in this application determines the mapping region in the second image corresponding to the biopsy region in the first image by utilizing the relative positional relationship and coordinate mapping relationship between the first detection box including the physiological part in the first image and the second detection box including the physiological part in the second image. Therefore, the determination of the mapping region is more reasonable and conforms to the actual situation.

[0163] The technical solution provided in this application proposes a scheme to match the biopsy region of a first image with the mapping region of a second image. Simultaneously, it extracts features from the first image using a feature extraction network of an image classification model. The pre-trained image classification model has good feature extraction capabilities; therefore, the feature information extracted by the feature extraction network based on the image classification model can better reflect the image features of the first image, which is beneficial for subsequent feature difference calculations. Thus, the priority of the determined biopsy region is more accurate. Furthermore, the technical solution provided in this application, through positional deviation and size ratio, determines a mapping region that better reflects the actual situation. Therefore, with a more accurate determination of the mapping region, the final priority determination of the biopsy region is also more precise.

[0164] Taking cervical images as an example, the technical solution provided in this application proposes a matching scheme between the biopsy points in acetic acid-stained images and the corresponding positions in saline images. The biopsy points represent the areas of epithelial change after acetic acid staining, and matching the positions is a prerequisite for comparing these changes. Simultaneously, a multi-input network backbone training scheme is used to facilitate CNN training for classification tasks on saline and acetic acid-stained image sequences, improving the ability to extract acetic acid whitening epithelial features from a single acetic acid-stained image. Furthermore, the solution proposes incorporating target dispersion into the training loss function, aligning with practical applications.

[0165] Please refer to Figure 13 This diagram illustrates a block diagram of a method for determining the priority of a biopsy region according to an embodiment of this application. The entity executing each step of this method may be... Figure 1 The terminal device 10 or server 20 in the implementation environment of the illustrated scheme. In the following method embodiments, for ease of description, only the execution subject of each step is described as a "computer device". The method may include at least one of the following steps (S1 to S12):

[0166] S1, acquire acetic acid staining images and saline images of the cervix.

[0167] S2, detect the cervix contained in the acetic acid stained image to obtain the cervix detection box in the acetic acid stained image containing the cervix.

[0168] In some embodiments, a cervical os detection network is used to detect the cervix included in the acetic acid staining image.

[0169] S3, detect the cervix contained in the saline image, and obtain the cervical detection box of the saline image containing the cervix.

[0170] S4. The acetic acid stained image is processed by the biopsy region detection model to obtain n biopsy regions in the acetic acid stained image, where n is a positive integer greater than 1.

[0171] S5, for the i-th biopsy region among n biopsy regions, determine the positional deviation between the i-th biopsy region and the cervical os detection frame in the acetic acid staining image.

[0172] S6. Based on the position information of the cervical os detection frame in the saline image and the cervical os detection frame in the acetic acid image, determine the size ratio of the saline image detection frame to the acetic acid image cervical os detection frame.

[0173] S7. Based on the size ratio of the saline cervical os detection frame to the acetic acid smear cervical os detection frame, the positional deviation between the i-th biopsy region and the acetic acid smear cervical os detection frame, and the positional information of the saline cervical os detection frame, determine the mapping region of the i-th biopsy region in the saline image.

[0174] S8. Extract the image content of n biopsy regions from the acetic acid staining image to obtain n acetic acid staining sub-images. The i-th biopsy region in the n biopsy regions corresponds to the i-th sub-image in the n acetic acid staining sub-images, where i is a positive integer less than or equal to n.

[0175] S9. Extract the image content of the mapped regions corresponding to n biopsy areas from the saline image to obtain n saline sub-images.

[0176] S10: Through the feature extraction network of the image classification model, the difference between the feature information of the i-th acetic acid saccharide image and the feature information of the n physiological saline saccharide images is obtained, resulting in n difference results.

[0177] S11, Based on the n difference results, determine the difference information corresponding to the i-th biopsy region.

[0178] In some embodiments, the EMD distance between paired acetic acid stained image and saline stained image features is calculated.

[0179] S12, based on the difference information corresponding to each of the n biopsy regions, determine the priority of each of the n biopsy regions. The priority is used to characterize the lesion probability of the biopsy region.

[0180] The technical solution provided in this application helps doctors and patients efficiently detect precancerous cervical lesions. Due to its very high coverage and accuracy, this solution can accurately detect a significant number of effective precancerous cervical lesion areas. It can also help improve the medical level and overall diagnostic accuracy in areas lacking high-quality medical resources. Furthermore, it greatly frees up medical resources. The technical solution provided in this application can be operated by junior doctors, allowing senior doctors or specialists to be removed from the front line of colposcopy examinations. Simultaneously, the recognition capability continuously evolves, leading to a virtuous cycle. The technical solution provided in this application further enhances the model's recognition capability by continuously collecting sample data and increasing the training dataset of the offline model.

[0181] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0182] Please refer to Figure 14 This diagram illustrates a block diagram of a biopsy region priority determination apparatus according to an embodiment of this application. The apparatus has the functionality to implement the method example described above; this functionality can be implemented in hardware or by hardware executing corresponding software. The apparatus can be the computer device described above, or it can be located within a computer device. Figure 14 As shown, the device 1400 may include: an image acquisition module 1410, a region determination module 1420, a difference determination module 1430, and a priority determination module 1440.

[0183] The image acquisition module 1410 is used to acquire a first image and a second image obtained by two different methods for the same physiological part.

[0184] The region determination module 1420 is used to determine the mapping region of n biopsy regions in the first image in the second image based on the position mapping relationship of the physiological parts in the first image and the second image, where n is an integer greater than 1.

[0185] The difference determination module 1430 is used to determine the difference information corresponding to the n biopsy regions based on the image content of the n biopsy regions and the corresponding mapping regions, respectively. The difference information is used to characterize the feature differences between the biopsy regions and their corresponding mapping regions in the image content.

[0186] The priority determination module 1440 is used to determine the priority of each of the n biopsy regions based on the difference information corresponding to each of the n biopsy regions. The priority is used to characterize the lesion probability of the biopsy region.

[0187] In some embodiments, such asFigure 15 As shown, the difference determination module 1430 includes a sub-image acquisition unit 1432 and a difference determination unit 1434.

[0188] The sub-image acquisition unit 1432 is used to extract the image content of the n biopsy regions from the first image to obtain n first sub-images. The i-th biopsy region in the n biopsy regions corresponds to the i-th first sub-image in the n first sub-images, where i is a positive integer less than or equal to n.

[0189] The sub-image acquisition unit 1432 is further configured to extract the image content of the mapping regions corresponding to the n biopsy regions from the second image to obtain n second sub-images.

[0190] The difference determination unit 1434 is used to obtain the difference information corresponding to the i-th biopsy region based on the feature differences between the i-th first sub-image and the n second sub-images for the i-th biopsy region.

[0191] In some embodiments, the difference determination unit 1434 is used to obtain the difference between the feature information of the i-th first sub-image and the feature information of the n second sub-images, respectively, to obtain n difference results.

[0192] The difference determination unit 1434 is further configured to determine the difference information corresponding to the i-th biopsy region based on the n difference results.

[0193] In some embodiments, the difference determination unit 1434 is further configured to obtain feature information of the first sub-image and feature information of the second sub-image through the feature extraction network of the image classification model; wherein, the image classification model is used to classify whether the biopsy area is a lesion.

[0194] In some embodiments, the training process of the image classification model is as follows: The feature extraction network is used to obtain feature information corresponding to a first image training sample and a second image training sample obtained using two different methods for the same physiological site; the feature fusion network of the image classification model is used to fuse the feature information corresponding to the first image training sample and the second image training sample to obtain fused feature information; the classification network of the image classification model is used to obtain multiple lesion detection categories corresponding to the fused feature information and the probabilities corresponding to different lesion detection categories; the parameters of the image classification model are adjusted based on the multiple lesion detection categories corresponding to the fused feature information, the probabilities corresponding to different lesion detection categories, and the first label corresponding to the first image sample, including the first lesion detection category.

[0195] In some embodiments, such as Figure 15 As shown, the region determination module 1420 includes a detection box determination unit 1422 and a region determination unit 1424.

[0196] The detection frame determination unit 1422 is used to detect the physiological parts contained in the first image to obtain a first detection frame containing the physiological parts.

[0197] The detection frame determination unit 1422 is further configured to detect the physiological part contained in the second image to obtain a second detection frame containing the physiological part.

[0198] The region determination unit 1424 is used to determine the mapping region of the n biopsy regions in the first image in the second image based on the relative positional relationship between the n biopsy regions in the first image and the first detection box, and the coordinate mapping relationship between the first detection box and the second detection box.

[0199] In some embodiments, the region determination unit 1424 is configured to determine the positional deviation between the i-th biopsy region and the first detection frame for the i-th biopsy region among the n biopsy regions.

[0200] The region determination unit 1424 is further configured to determine the size ratio of the second detection frame to the first detection frame based on the position information of the second detection frame and the position information of the first detection frame.

[0201] The region determination unit 1424 is further configured to determine the mapping region of the i-th biopsy region in the second image based on the size ratio of the second detection box to the first detection box, the positional deviation between the i-th biopsy region and the first detection box, and the positional information of the second detection box.

[0202] In some embodiments, the region determination module 1420 is further configured to process the first image using a biopsy region detection model to obtain n biopsy regions in the first image; wherein the loss function of the biopsy region detection model during training includes a looseness loss, which is used to measure the positional relationship between the biopsy regions.

[0203] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0204] Figure 16 A structural block diagram of a computer device provided in an exemplary embodiment of this application is shown.

[0205] Typically, computer device 1600 includes a processor 1601 and a memory 1602.

[0206] Processor 1601 may include one or more processing cores, such as a quad-core processor or a 16-core processor. Processor 1601 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1601 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1601 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 1601 may also include an AI processor for handling computational operations related to machine learning.

[0207] The memory 1602 may include one or more computer-readable storage media, which may be tangible and non-transitory. The memory 1602 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1602 stores a computer program that is loaded and executed by the processor 1601 to implement the glue defect determination method provided in the above-described method embodiments.

[0208] Those skilled in the art will understand that Figure 16The structure shown does not constitute a limitation on the computer device 1600, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0209] In an exemplary embodiment, a computer-readable storage medium is also provided, the storage medium storing a computer program that, when executed by a processor, implements a method for determining the priority of an upper biopsy region.

[0210] Optionally, the computer-readable storage medium may include: ROM (Read-Only Memory), RAM (Random Access Memory), SSD (Solid State Drives), or optical disc, etc. The random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0211] In an exemplary embodiment, a computer program product is also provided, the computer program product including a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, causing the computer device to perform the above-described biopsy region priority determination method.

[0212] It should be understood that "multiple" as used herein refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the step numbers described herein are merely illustrative of one possible execution order. In some other embodiments, the steps may not be executed in numerical order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the illustration. This application does not limit this.

[0213] The above description is merely an exemplary embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for determining the priority of a biopsy region, characterized in that, The method includes: Acquire a first image and a second image obtained using two different methods for the same physiological site; The physiological parts contained in the first image are detected to obtain a first detection box containing the physiological parts; the physiological parts contained in the second image are detected to obtain a second detection box containing the physiological parts; based on the relative positional relationship between the n biopsy regions in the first image and the first detection box, and the coordinate mapping relationship between the first detection box and the second detection box, the mapping region of the n biopsy regions in the first image in the second image is determined, where n is an integer greater than 1; Based on the image content of the n biopsy regions and their corresponding mapping regions, the difference information corresponding to the n biopsy regions is determined. The difference information is used to characterize the feature differences between the biopsy regions and their corresponding mapping regions in the image content. Based on the difference information corresponding to the n biopsy regions, the priority of each of the n biopsy regions is determined, and the priority is used to characterize the lesion probability of the biopsy region.

2. The method according to claim 1, characterized in that, The step of determining the difference information corresponding to each of the n biopsy regions based on the image content of the n biopsy regions and the corresponding mapped regions includes: The image content of the n biopsy regions is extracted from the first image to obtain n first sub-images. The i-th biopsy region in the n biopsy regions corresponds to the i-th first sub-image in the n first sub-images, where i is a positive integer less than or equal to n. The image content of the mapped regions corresponding to the n biopsy regions is extracted from the second image to obtain n second sub-images; For the i-th biopsy region, the difference information corresponding to the i-th biopsy region is obtained based on the feature differences between the i-th first sub-image and the n second sub-images.

3. The method according to claim 2, characterized in that, The step of obtaining the difference information corresponding to the i-th biopsy region based on the feature differences between the i-th first sub-image and the n second sub-images includes: The differences between the feature information of the i-th first sub-image and the feature information of the n second sub-images are obtained respectively, resulting in n difference results; Based on the n difference results, the difference information corresponding to the i-th biopsy region is determined.

4. The method according to claim 3, characterized in that, The method further includes: The feature extraction network of the image classification model is used to obtain the feature information of the first sub-image and the feature information of the second sub-image; wherein, the image classification model is used to classify whether the biopsy area is a lesion.

5. The method according to claim 4, characterized in that, The training process of the image classification model is as follows: The feature extraction network is used to obtain feature information corresponding to the first image training sample and the second image training sample obtained by two different methods for the same physiological part. Based on the feature fusion network of the image classification model, the feature information corresponding to the first image training sample and the second image training sample are fused to obtain fused feature information. Based on the classification network of the image classification model, multiple lesion detection categories corresponding to the fused feature information and the probabilities corresponding to different lesion detection categories are obtained. Based on the multiple lesion detection categories corresponding to the fused feature information and the probabilities corresponding to different lesion detection categories, as well as the first label corresponding to the first image sample including the first lesion detection category, the parameters of the image classification model are adjusted.

6. The method according to claim 1, characterized in that, The step of determining the mapping region of the n biopsy regions in the first image in the second image based on the relative positional relationship between the n biopsy regions in the first image and the first detection box, and the coordinate mapping relationship between the first detection box and the second detection box, includes: For the i-th biopsy region among the n biopsy regions, determine the positional deviation between the i-th biopsy region and the first detection frame, where i is a positive integer less than or equal to n; Based on the position information of the second detection frame and the position information of the first detection frame, the size ratio of the second detection frame to the first detection frame is determined; Based on the size ratio of the second detection box to the first detection box, the positional deviation between the i-th biopsy region and the first detection box, and the positional information of the second detection box, the mapping region of the i-th biopsy region in the second image is determined.

7. The method according to claim 1, characterized in that, The method further includes: The first image is processed using a biopsy region detection model to obtain n biopsy regions in the first image; The loss function of the biopsy region detection model during training includes a looseness loss, which is used to measure the positional relationship between the biopsy regions.

8. A priori determination device for biopsy areas, characterized in that, The device includes: The image acquisition module is used to acquire a first image and a second image obtained using two different methods for the same physiological site. The region determination module is used to detect the physiological parts contained in the first image to obtain a first detection box containing the physiological parts; to detect the physiological parts contained in the second image to obtain a second detection box containing the physiological parts; and to determine the mapping region of the n biopsy regions in the first image in the second image based on the relative positional relationship between the n biopsy regions in the first image and the first detection box, and the coordinate mapping relationship between the first detection box and the second detection box, where n is an integer greater than 1. The difference determination module is used to determine the difference information corresponding to the n biopsy regions based on the image content of the n biopsy regions and the corresponding mapping regions, respectively. The difference information is used to characterize the feature differences between the biopsy regions and their corresponding mapping regions in the image content. The priority determination module is used to determine the priority of each of the n biopsy regions based on the difference information corresponding to each of the n biopsy regions. The priority is used to characterize the lesion probability of the biopsy region.

9. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing a computer program that is loaded and executed by the processor to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is loaded and executed by a processor to implement the method as described in any one of claims 1 to 7.

11. A computer program product, characterized in that, The computer program product includes a computer program that is loaded and executed by a processor to implement the method as described in any one of claims 1 to 7.

Images