A method for recognizing cervical cancer pathological images with small samples

Abstract

The present application provides a cervical cancer pathology image recognition method. The method obtains cervical cancer pathology information of a target object, then uses a preset cervical cancer pathology image recognition model to recognize the cervical cancer pathology image to identify a characteristic area, and finally displays the cervical cancer pathology image recognition result on a target terminal, thereby realizing automatic and rapid recognition of pathological characteristic areas in the cervical cancer pathology image, and realizing intuitive display of the recognition result, effectively improving the efficiency of cervical cancer pathology diagnosis and recognition.

Description

Technical Field

[0001] The present application relates to data processing technology, and in particular to a method for recognizing cervical cancer pathological images under small samples. Background Art

[0002] Pathological analysis of cervical cancer is a crucial link in the diagnosis and treatment of cervical cancer. In recent years, significant progress and development have been made in technology, methods and applications.

[0003] In the prior art, biopsy or cytology is usually performed on the patient, including scraping, cone biopsy, etc., to obtain tissue samples from the patient's cervix. Then, the collected tissue samples are fixed, dehydrated, embedded, sliced ​​into slides, and stained to show different structures and features. Finally, they are scanned by a digital pathology scanner, and the scanned digital pathology images can be uploaded to the hospital information system through the network to facilitate doctors to view the digital pathology images. However, for digital pathology images with ultra-high pixel resolution, their size is very large, and the efficiency of doctors viewing them is low. Summary of the invention

[0004] The present application provides a method for recognizing cervical cancer pathological images under a small sample, so as to solve the technical problem that doctors have low efficiency in viewing digital pathological images.

[0005] In a first aspect, the present application provides a method for recognizing cervical cancer pathological images, comprising:

[0006] Acquire cervical cancer pathology information of the target object, wherein the cervical cancer pathology information includes identity information and cervical cancer pathology images acquired by a digital pathology scanner;

[0007] The cervical cancer pathology image is identified by using a preset cervical cancer pathology image recognition model to identify a characteristic area, wherein the characteristic area is a partial image area in the cervical cancer pathology image, the preset cervical cancer pathology image recognition model is a neural network model trained based on a preset cervical cancer pathology image data set, and the cervical cancer pathology image samples in the preset cervical cancer pathology image data set include an original cervical cancer pathology image and a characteristic identifier annotated on the original cervical cancer pathology image, and the characteristic identifier is used to characterize that the corresponding area on the original cervical cancer pathology image is in an abnormal state;

[0008] The cervical cancer pathology image recognition result is displayed on the target terminal, and the cervical cancer pathology image recognition result includes the cervical cancer pathology image and a feature identification frame, and the feature identification frame is used to identify the feature area on the cervical cancer pathology image. The target terminal is a terminal device determined according to the identity identification information.

[0009] Optionally, displaying the cervical cancer pathology image recognition result on the target terminal includes:

[0010] Displaying the cervical cancer pathology image on the target terminal, and displaying the feature identification frame in the feature area of ​​the cervical cancer pathology image, wherein the display position of the geometric center point of the feature identification frame on the target terminal coincides with the geometric center point on the display of the target terminal;

[0011] A feature prompt mark is displayed on the feature mark frame, so that when the feature prompt mark is triggered, a feature magnified image is displayed on the target terminal, and the feature magnified image is an enlarged image of the feature area.

[0012] Optionally, if there are multiple feature areas, all feature areas and feature identification boxes corresponding to each feature area are included in the initial display image of the display, and the geometric center point on the display coincides with the geometric center point of the feature identification box corresponding to the feature area with the largest area among the multiple feature areas.

[0013] Optionally, before the cervical cancer pathology image is recognized by using a preset cervical cancer pathology image recognition model, the method further includes:

[0014] The original cervical cancer pathology image set P = {p1, p2, ..., p t ,…,p n}, where p t is the t-th original cervical cancer pathology image in the original cervical cancer pathology image set P, and n is the number of original cervical cancer pathology images in the original cervical cancer pathology image set P;

[0015] Obtain the image resolution set F corresponding to the original cervical cancer pathology image set P = {f1, f2, ..., f t ,…,f n}, wherein the f t For the p t The corresponding resolution;

[0016] If the f t >f k , then the p is downsampled by the downsampling coefficient s t Downsampling is performed to convert the p t Updated to the image p after the downsampling process t ′ , and generate an updated original cervical cancer pathology image set P ′ , wherein the f kTo preset the image resolution threshold, the downsampling coefficient s is determined according to Formula 1, which is:

[0017]

[0018] In response to the feature identification instruction, an updated original cervical cancer pathology image set P is obtained. ′ The feature identifiers on each original cervical cancer pathology image are obtained, and corresponding cervical cancer pathology image samples are formed to generate the preset cervical cancer pathology image dataset.

[0019] Optionally, in response to the feature identification instruction, obtaining an updated original cervical cancer pathology image set P ′ The feature identifiers on each original cervical cancer pathology image in the image are used to form corresponding cervical cancer pathology image samples, including:

[0020] The original cervical cancer pathology image set P ′ The original cervical cancer pathology images are gray-scaled and the gray-scaled original cervical cancer pathology images are segmented into a set of image blocks. Wherein, m is the number of image blocks in a first direction and a second direction of the image, and the first direction is orthogonal to the second direction;

[0021] In response to acting on the updated original cervical cancer pathology image set P ′ A contour drawing instruction on each original cervical cancer pathology image in the image block set W is used to determine an abnormal region set from the image block set W, wherein the image blocks in the abnormal region set are image blocks within the abnormal region contour line range and image blocks having intersections with the abnormal region contour line, and the abnormal region contour line is a contour line generated in response to the contour drawing instruction;

[0022] According to Formula 2, the feature point coordinates (c x ,c y ), the formula 2 is:

[0023]

[0024] Among them, x ij is the horizontal coordinate of the pixel numbered i in the first direction and j in the second direction in the image block, ij The gray value of the pixel with sequence number i in the first direction and sequence number j in the second direction in the image block, y ij is the ordinate of the pixel numbered i in the first direction and j in the second direction in the image block;

[0025] If the feature point coordinates (c x ,c y ) is outside the abnormal region contour line, then the adjacent image blocks of the corresponding image block are merged into the abnormal region set to generate an extended abnormal region set;

[0026] The feature identifier is generated according to the set of extended abnormal regions, and a corresponding cervical cancer pathology image sample is formed. The feature identifier is used to identify the extended abnormal region.

[0027] Optionally, the neural network model is a graph neural network, and the graph neural network includes multiple layers of main neighborhood aggregation layers, wherein the output of the current layer of the main neighborhood aggregation layer in the multiple layers of the main neighborhood aggregation layer is the input of the next layer of the main neighborhood aggregation layer, and the node feature of the g-th node output by the next layer of the main neighborhood aggregation layer is The determination is made according to Formula 3, which is:

[0028]

[0029] in, is the node feature of the g-th node output by the main neighborhood aggregation layer of the current layer, M is the perceptron prediction layer, E is the edge between any nodes, is the node feature of the adjacent nodes of the g-th node on edge E, U is the node aggregation operation, is the tensor product.

[0030] Optionally, after displaying the cervical cancer pathology image recognition result on the target terminal, the method further includes:

[0031] Acquire a recognition result feedback instruction acting on the target terminal, wherein the recognition result feedback instruction includes a recognition result confirmation instruction and a recognition result modification instruction, wherein the recognition result confirmation instruction is used to confirm the cervical cancer pathology image recognition result, and the recognition result modification instruction is used to perform at least one modification operation of position adjustment, range adjustment, and deletion of the feature identification frame on the cervical cancer pathology image recognition result;

[0032] If the recognition result feedback instruction is the recognition result modification instruction, generating a cervical cancer pathology image recognition modification result according to the recognition result modification instruction, and adding the cervical cancer pathology image recognition modification result to the preset cervical cancer pathology image dataset to generate an updated cervical cancer pathology image dataset;

[0033] The preset cervical cancer pathology image recognition model is updated and trained using the updated cervical cancer pathology image dataset.

[0034] In a second aspect, the present application provides a cervical cancer pathology image recognition device, characterized in that:

[0035] An acquisition module, used to acquire cervical cancer pathology information of a target object, wherein the cervical cancer pathology information includes identity information and a cervical cancer pathology image acquired by a digital pathology scanner;

[0036] A processing module, used to identify the cervical cancer pathology image by using a preset cervical cancer pathology image recognition model to identify a characteristic area, wherein the characteristic area is a partial image area in the cervical cancer pathology image, the preset cervical cancer pathology image recognition model is a neural network model trained based on a preset cervical cancer pathology image data set, the cervical cancer pathology image samples in the preset cervical cancer pathology image data set include an original cervical cancer pathology image and a characteristic identifier annotated on the original cervical cancer pathology image, and the characteristic identifier is used to indicate that the corresponding area on the original cervical cancer pathology image is in an abnormal state;

[0037] A display module is used to display the cervical cancer pathology image recognition result on the target terminal, the cervical cancer pathology image recognition result includes the cervical cancer pathology image and a feature identification frame, the feature identification frame is used to identify the feature area on the cervical cancer pathology image, and the target terminal is a terminal device determined according to the identity identification information.

[0038] Optionally, the display module is specifically used for:

[0039] Displaying the cervical cancer pathology image on the target terminal, and displaying the feature identification frame in the feature area of ​​the cervical cancer pathology image, wherein the display position of the geometric center point of the feature identification frame on the target terminal coincides with the geometric center point on the display of the target terminal;

[0040] A feature prompt mark is displayed on the feature mark frame, so that when the feature prompt mark is triggered, a feature magnified image is displayed on the target terminal, and the feature magnified image is an enlarged image of the feature area.

[0041] Optionally, if there are multiple feature areas, all feature areas and feature identification boxes corresponding to each feature area are included in the initial display image of the display, and the geometric center point on the display coincides with the geometric center point of the feature identification box corresponding to the feature area with the largest area among the multiple feature areas.

[0042] Optionally, the acquisition module is further used to acquire an original cervical cancer pathology image set P = {p1, p2, ..., p t ,…,p n}, where pt is the t-th original cervical cancer pathology image in the original cervical cancer pathology image set P, and n is the number of original cervical cancer pathology images in the original cervical cancer pathology image set P;

[0043] The acquisition module is also used to acquire the image resolution set F corresponding to the original cervical cancer pathology image set P = {f1, f2, ..., f t ,…,f n}, wherein the f t For the p t The corresponding resolution;

[0044] If the f t >f k , then the p is downsampled by the downsampling coefficient s t Downsampling is performed to convert the p t Updated to the image p after the downsampling process t ′ , and generate an updated original cervical cancer pathology image set P ′ , wherein the f k To preset the image resolution threshold, the downsampling coefficient s is determined according to Formula 1, which is:

[0045]

[0046] The acquisition module is also used to respond to the feature identification instruction to acquire the updated original cervical cancer pathology image set P ′ The feature identifiers on each original cervical cancer pathology image are obtained, and corresponding cervical cancer pathology image samples are formed to generate the preset cervical cancer pathology image dataset.

[0047] Optionally, the processing module is specifically used to:

[0048] The original cervical cancer pathology image set P ′ The original cervical cancer pathology images are gray-scaled and the gray-scaled original cervical cancer pathology images are segmented into a set of image blocks. Wherein, m is the number of image blocks in a first direction and a second direction of the image, and the first direction is orthogonal to the second direction;

[0049] In response to acting on the updated original cervical cancer pathology image set P ′A contour drawing instruction on each original cervical cancer pathology image in the image block set W is used to determine an abnormal region set from the image block set W, wherein the image blocks in the abnormal region set are image blocks within the abnormal region contour line range and image blocks having intersections with the abnormal region contour line, and the abnormal region contour line is a contour line generated in response to the contour drawing instruction;

[0050] According to Formula 2, the feature point coordinates (c x ,c y ), the formula 2 is:

[0051]

[0052] Among them, x ij is the horizontal coordinate of the pixel numbered i in the first direction and j in the second direction in the image block, ij The gray value of the pixel with sequence number i in the first direction and sequence number j in the second direction in the image block, y ij is the ordinate of the pixel numbered i in the first direction and j in the second direction in the image block;

[0053] If the feature point coordinates (c x ,c y ) is outside the abnormal region contour line, then the adjacent image blocks of the corresponding image block are merged into the abnormal region set to generate an extended abnormal region set;

[0054] The feature identifier is generated according to the set of extended abnormal regions, and a corresponding cervical cancer pathology image sample is formed. The feature identifier is used to identify the extended abnormal region.

[0055] Optionally, the neural network model is a graph neural network, and the graph neural network includes multiple layers of main neighborhood aggregation layers, wherein the output of the current layer of the main neighborhood aggregation layer in the multiple layers of the main neighborhood aggregation layer is the input of the next layer of the main neighborhood aggregation layer, and the node feature of the g-th node output by the next layer of the main neighborhood aggregation layer is Determine according to Formula 3, which is:

[0056]

[0057] in, is the node feature of the g-th node output by the main neighborhood aggregation layer of the current layer, M is the perceptron prediction layer, E is the edge between any nodes, is the node feature of the adjacent nodes of the g-th node on edge E, U is the node aggregation operation, is the tensor product.

[0058] Optionally, the acquisition module is further used to acquire a recognition result feedback instruction acting on the target terminal, the recognition result feedback instruction includes a recognition result confirmation instruction and a recognition result modification instruction, wherein the recognition result confirmation instruction is used to confirm the cervical cancer pathology image recognition result, and the recognition result modification instruction is used to perform at least one modification operation of position adjustment, range adjustment and deletion of the feature identification frame on the cervical cancer pathology image recognition result;

[0059] If the recognition result feedback instruction is the recognition result modification instruction, generating a cervical cancer pathology image recognition modification result according to the recognition result modification instruction, and adding the cervical cancer pathology image recognition modification result to the preset cervical cancer pathology image dataset to generate an updated cervical cancer pathology image dataset;

[0060] The preset cervical cancer pathology image recognition model is updated and trained using the updated cervical cancer pathology image dataset.

[0061] In a third aspect, the present application provides an electronic device, including:

[0062] processor; and,

[0063] A memory, configured to store executable instructions of the processor;

[0064] The processor is configured to perform any possible method described in the first aspect by executing the executable instructions.

[0065] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement any possible method described in the first aspect.

[0066] The present application provides a cervical cancer pathology image recognition method, which obtains cervical cancer pathology information of a target object, then uses a preset cervical cancer pathology image recognition model to recognize the cervical cancer pathology image to identify the characteristic area, and finally displays the cervical cancer pathology image recognition result on the target terminal, thereby realizing automatic and rapid recognition of pathological characteristic areas in the cervical cancer pathology image, and realizing intuitive display of the recognition results, thereby effectively improving the efficiency of cervical cancer pathology diagnosis and recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0068] Figure 1 is a flowchart of a method for recognizing cervical cancer pathology images according to an exemplary embodiment of the present application;

[0069] Figure 2 is a flowchart of a method for recognizing cervical cancer pathology images according to another exemplary embodiment of the present application;

[0070] Figure 3 is a schematic structural diagram of a cervical cancer pathology image recognition device according to an exemplary embodiment of the present application;

[0071] Figure 4 It is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present application.

[0072] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0073] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0074] Figure 1 FIG. 1 is a flow chart of a method for recognizing cervical cancer pathology images according to an exemplary embodiment of the present application. Figure 1 As shown, the method provided in this embodiment includes:

[0075] S101. Obtain cervical cancer pathology information of a target object.

[0076] Specifically, it can be to obtain the cervical cancer pathology information of the target object, wherein the cervical cancer pathology information can include identity information and cervical cancer pathology images obtained by a digital pathology scanner. Specifically, the process of obtaining cervical cancer pathology images can be that the patient undergoes a biopsy or cytology examination in the hospital, including scraping, cone biopsy, etc., to obtain tissue samples from the patient's cervix. Then, the collected tissue samples are fixed, dehydrated, embedded, and sliced ​​into slides, and stained to display different structures and features, and finally scanned by a digital pathology scanner, and the scanned digital pathology images can be uploaded to the hospital information system through the network, or imported into a special pathology image analysis software for storage and management.

[0077] S102: Using a preset cervical cancer pathology image recognition model to recognize the cervical cancer pathology image to identify characteristic areas.

[0078] Specifically, a preset cervical cancer pathology image recognition model may be used to identify cervical cancer pathology images to identify characteristic areas, wherein the characteristic areas are partial image areas in cervical cancer pathology images, the preset cervical cancer pathology image recognition model is a neural network model trained based on a preset cervical cancer pathology image data set, and the cervical cancer pathology image samples in the preset cervical cancer pathology image data set include original cervical cancer pathology images and characteristic identifiers annotated on the original cervical cancer pathology images, and the characteristic identifiers are used to characterize that the corresponding areas on the original cervical cancer pathology images are in an abnormal state.

[0079] S103: Displaying the cervical cancer pathology image recognition result on the target terminal.

[0080] Specifically, the cervical cancer pathology image recognition result is displayed on the target terminal, and the cervical cancer pathology image recognition result includes the cervical cancer pathology image and the feature identification frame, and the feature identification frame is used to identify the feature area on the cervical cancer pathology image, wherein the target terminal is a terminal device determined according to the identity identification information. Optionally, the target terminal can be a terminal device corresponding to the attending physician who diagnoses the target object.

[0081] Optionally, a cervical cancer pathology image is displayed on the target terminal, and a feature identification frame is displayed in the feature area of ​​the cervical cancer pathology image. The display position of the geometric center point of the feature identification frame on the target terminal coincides with the geometric center point on the display of the target terminal, so that the area with pathological features is displayed to the center of the display to improve the doctor's viewing efficiency and effectively avoid the doctor from repeatedly dragging and zooming on the cervical cancer pathology image to find the area with pathological features. In addition, a feature prompt mark can be displayed on the feature identification frame, so that when the feature prompt mark is triggered, a feature magnified image is displayed on the target terminal, and the feature magnified image is an enlarged image of the feature area, so that the doctor can quickly zoom in on the area with pathological features.

[0082] Furthermore, if there are multiple feature areas, all feature areas and feature identification boxes corresponding to each feature area are included in the initial display image of the display, and the geometric center point on the display coincides with the geometric center point of the feature identification box corresponding to the feature area with the largest area among the multiple feature areas. Therefore, during the initial display, the doctor can first obtain the overall distribution, and the feature area with the largest area is displayed in the center, so that the doctor can quickly view the area with the highest priority, further improving the doctor's viewing efficiency.

[0083] In this embodiment, the cervical cancer pathology information of the target object is obtained, and then the cervical cancer pathology image is recognized using a preset cervical cancer pathology image recognition model to identify the characteristic area. Finally, the cervical cancer pathology image recognition result is displayed on the target terminal, thereby achieving automatic and rapid recognition of pathological characteristic areas in the cervical cancer pathology image, and achieving an intuitive display of the recognition results, thereby effectively improving the efficiency of cervical cancer pathology diagnosis and recognition.

[0084] Figure 2 FIG. 1 is a flow chart of a method for recognizing cervical cancer pathology images according to another exemplary embodiment of the present application. Figure 2 As shown, the method provided in this embodiment includes:

[0085] S201. Acquire a set of original cervical cancer pathology images through a digital pathology scanner.

[0086] Specifically, the original cervical cancer pathology image set P = {p1, p2, ..., p t ,…,p n}, where p t is the tth original cervical cancer pathology image in the original cervical cancer pathology image set P, and n is the number of original cervical cancer pathology images in the original cervical cancer pathology image set P.

[0087] S202: Obtain an image resolution set corresponding to the original cervical cancer pathology image set P.

[0088] Then, the image resolution set F corresponding to the original cervical cancer pathology image set P is obtained. t ,…,f n}, where f t For p t The corresponding resolution.

[0089] S203, performing downsampling processing by using a downsampling coefficient to update the image to an image after the downsampling processing, and generating an updated original cervical cancer pathology image set.

[0090] If f t >f k , then the downsampling coefficient s is used to reduce p t Downsampling is performed to convert p t Updated to the downsampled image p t ′ , and generate an updated original cervical cancer pathology image set P ′ , where f k To preset the image resolution threshold, the downsampling coefficient s is determined according to Formula 1, which is:

[0091]

[0092] It is worth noting that, through the selection method of the above-mentioned downsampling coefficient, dynamic adjustment can be made according to the specific resolution of the cervical cancer pathology image, which can take into account the limited sensitivity of the human eye to high-frequency details and avoid excessive loss of effective information due to excessive downsampling.

[0093] S204, in response to the feature identification instruction, obtaining an updated original cervical cancer pathology image set P ′ The feature identifiers on each original cervical cancer pathology image in the image are obtained, and corresponding cervical cancer pathology image samples are formed to generate a preset cervical cancer pathology image dataset.

[0094] In this step, in response to the feature identification instruction, an updated original cervical cancer pathology image set P is obtained. ′ The feature identifiers on each original cervical cancer pathology image in the image are obtained, and corresponding cervical cancer pathology image samples are formed to generate a preset cervical cancer pathology image dataset.

[0095] Specifically, the original cervical cancer pathology image set P ′ The original cervical cancer pathology images are gray-scaled and the gray-scaled original cervical cancer pathology images are segmented into a set of image blocks. Wherein, m is the number of image blocks in the first direction and the second direction of the image, and the first direction is orthogonal to the second direction.

[0096] In response to acting on the updated original cervical cancer pathology image set P ′ The contour drawing instructions are executed on each original cervical cancer pathology image to determine the abnormal area set from the image block set W. The image blocks in the abnormal area set are the image blocks within the abnormal area contour line range and the image blocks that have intersections with the abnormal area contour line. The abnormal area contour line is the contour line generated in response to the contour drawing instruction.

[0097] Then, according to Formula 2, the feature point coordinates of the image blocks in the abnormal region set that have intersections with the abnormal region contour line are calculated (c x ,c y ), Formula 2 is:

[0098]

[0099] Among them, x ij is the horizontal coordinate of the pixel numbered i in the first direction and j in the second direction in the image block, h ijThe gray value of the pixel with sequence number i in the first direction and sequence number j in the second direction in the image block, y ij is the ordinate of the pixel numbered i in the first direction and j in the second direction in the image block;

[0100] If the feature point coordinates (c x ,c y ) is outside the abnormal area contour line, the adjacent image blocks of the corresponding image block are merged into the abnormal area set to generate an extended abnormal area set. Finally, a feature identifier is generated according to the extended abnormal area set, and a corresponding cervical cancer pathology image sample is formed. The feature identifier is used to identify the extended abnormal area. It is worth noting that the position distribution of the feature points in the corresponding image block can be used to characterize the distribution of the overall features of the corresponding image block. Then, the relative position relationship between the feature point coordinates and the abnormal area contour line is determined. When the feature point exceeds the abnormal area contour line, it means that the distribution of the overall features of the corresponding image block exceeds the abnormal area contour line. Therefore, the adjacent image blocks of the corresponding image block can be merged into the abnormal area set, thereby ensuring the recognition completeness rate of the abnormal area.

[0101] S205: Training with a preset cervical cancer pathology image dataset to form a preset cervical cancer pathology image recognition model.

[0102] Specifically, a preset cervical cancer pathology image dataset can be used for training to form a preset cervical cancer pathology image recognition model. The neural network model used can be a graph neural network, which includes multiple main neighborhood aggregation layers, wherein the output of the current main neighborhood aggregation layer in the multiple main neighborhood aggregation layers is the input of the next main neighborhood aggregation layer, and the node feature of the gth node output by the next main neighborhood aggregation layer is Determine according to formula 3, which is:

[0103]

[0104] in, is the node feature of the g-th node output by the main neighborhood aggregation layer of the current layer, M is the perceptron prediction layer, E is the edge between any nodes, is the node feature of the adjacent nodes of the g-th node on edge E, U is the node aggregation operation, is the tensor product.

[0105] In addition, in this embodiment, the neural network model used may also be other neural networks, as long as it can ensure that image feature recognition can be performed after training. In this implementation, the type of neural network model trained is not specifically limited.

[0106] S206: Obtain cervical cancer pathology information of the target object.

[0107] Specifically, it can be to obtain the cervical cancer pathology information of the target object, wherein the cervical cancer pathology information can include identity information and cervical cancer pathology images obtained by a digital pathology scanner. Specifically, the process of obtaining cervical cancer pathology images can be that the patient undergoes a biopsy or cytology examination in the hospital, including scraping, cone biopsy, etc., to obtain tissue samples from the patient's cervix. Then, the collected tissue samples are fixed, dehydrated, embedded, and sliced ​​into slides, and stained to display different structures and features, and finally scanned by a digital pathology scanner, and the scanned digital pathology images can be uploaded to the hospital information system through the network, or imported into a special pathology image analysis software for storage and management.

[0108] S207: Using a preset cervical cancer pathology image recognition model to recognize the cervical cancer pathology image to identify characteristic areas.

[0109] Specifically, a preset cervical cancer pathology image recognition model may be used to identify cervical cancer pathology images to identify characteristic areas, wherein the characteristic areas are partial image areas in cervical cancer pathology images, the preset cervical cancer pathology image recognition model is a neural network model trained based on a preset cervical cancer pathology image data set, and the cervical cancer pathology image samples in the preset cervical cancer pathology image data set include original cervical cancer pathology images and characteristic identifiers annotated on the original cervical cancer pathology images, and the characteristic identifiers are used to characterize that the corresponding areas on the original cervical cancer pathology images are in an abnormal state.

[0110] S208. Displaying the cervical cancer pathology image recognition result on the target terminal.

[0111] Specifically, the cervical cancer pathology image recognition result is displayed on the target terminal, and the cervical cancer pathology image recognition result includes the cervical cancer pathology image and the feature identification frame, and the feature identification frame is used to identify the feature area on the cervical cancer pathology image, wherein the target terminal is a terminal device determined according to the identity identification information. Optionally, the target terminal can be a terminal device corresponding to the attending physician who diagnoses the target object.

[0112] Optionally, a cervical cancer pathology image is displayed on the target terminal, and a feature identification frame is displayed in the feature area of ​​the cervical cancer pathology image. The display position of the geometric center point of the feature identification frame on the target terminal coincides with the geometric center point on the display of the target terminal, so that the area with pathological features is displayed to the center of the display to improve the doctor's viewing efficiency and effectively avoid the doctor from repeatedly dragging and zooming on the cervical cancer pathology image to find the area with pathological features. In addition, a feature prompt mark can be displayed on the feature identification frame, so that when the feature prompt mark is triggered, a feature magnified image is displayed on the target terminal, and the feature magnified image is an enlarged image of the feature area, so that the doctor can quickly zoom in on the area with pathological features.

[0113] Furthermore, if there are multiple feature areas, all feature areas and feature identification boxes corresponding to each feature area are included in the initial display image of the display, and the geometric center point on the display coincides with the geometric center point of the feature identification box corresponding to the feature area with the largest area among the multiple feature areas. Therefore, during the initial display, the doctor can first obtain the overall distribution, and the feature area with the largest area is displayed in the center, so that the doctor can quickly view the area with the highest priority, further improving the doctor's viewing efficiency.

[0114] In addition, after the cervical cancer pathology image recognition result is displayed on the target terminal, a recognition result feedback instruction acting on the target terminal can also be obtained. The recognition result feedback instruction includes a recognition result confirmation instruction and a recognition result modification instruction. The recognition result confirmation instruction is used to confirm the cervical cancer pathology image recognition result, and the recognition result modification instruction is used to perform at least one modification operation of position adjustment, range adjustment and deletion of the feature identification box on the cervical cancer pathology image recognition result.

[0115] If the recognition result feedback instruction is a recognition result modification instruction, a cervical cancer pathology image recognition modification result is generated according to the recognition result modification instruction, and the cervical cancer pathology image recognition modification result is added to the preset cervical cancer pathology image dataset to generate an updated cervical cancer pathology image dataset. The preset cervical cancer pathology image recognition model is updated and trained using the updated cervical cancer pathology image dataset, thereby achieving an updated iteration of the cervical cancer pathology image dataset to train a cervical cancer pathology image recognition model with higher recognition accuracy.

[0116] Figure 3 FIG. 1 is a schematic diagram of a cervical cancer pathology image recognition device according to an exemplary embodiment of the present application. Figure 3 As shown, the cervical cancer pathology image recognition device 300 provided in this embodiment includes:

[0117] An acquisition module 310 is used to acquire cervical cancer pathology information of a target object, wherein the cervical cancer pathology information includes identity information and a cervical cancer pathology image acquired by a digital pathology scanner;

[0118] The processing module 320 is used to identify the cervical cancer pathology image by using a preset cervical cancer pathology image recognition model to identify a characteristic area, wherein the characteristic area is a partial image area in the cervical cancer pathology image, the preset cervical cancer pathology image recognition model is a neural network model trained based on a preset cervical cancer pathology image data set, and the cervical cancer pathology image samples in the preset cervical cancer pathology image data set include an original cervical cancer pathology image and a characteristic identifier annotated on the original cervical cancer pathology image, and the characteristic identifier is used to indicate that the corresponding area on the original cervical cancer pathology image is in an abnormal state;

[0119] The display module 330 is used to display the cervical cancer pathology image recognition result on the target terminal, and the cervical cancer pathology image recognition result includes the cervical cancer pathology image and a feature identification frame, and the feature identification frame is used to identify the feature area on the cervical cancer pathology image. The target terminal is a terminal device determined according to the identity identification information.

[0120] Optionally, the display module 330 is specifically used for:

[0121] Displaying the cervical cancer pathology image on the target terminal, and displaying the feature identification frame in the feature area of ​​the cervical cancer pathology image, wherein the display position of the geometric center point of the feature identification frame on the target terminal coincides with the geometric center point on the display of the target terminal;

[0122] A feature prompt mark is displayed on the feature mark frame, so that when the feature prompt mark is triggered, a feature magnified image is displayed on the target terminal, and the feature magnified image is an enlarged image of the feature area.

[0123] Optionally, if there are multiple feature areas, all feature areas and feature identification boxes corresponding to each feature area are included in the initial display image of the display, and the geometric center point on the display coincides with the geometric center point of the feature identification box corresponding to the feature area with the largest area among the multiple feature areas.

[0124] Optionally, the acquisition module 310 is further configured to acquire an original cervical cancer pathology image set P = {p1, p2, ..., p t ,…,p n}, where p tis the t-th original cervical cancer pathology image in the original cervical cancer pathology image set P, and n is the number of original cervical cancer pathology images in the original cervical cancer pathology image set P;

[0125] The acquisition module 310 is further used to acquire an image resolution set F corresponding to the original cervical cancer pathology image set P = {f1, f2, ..., f t ,…,f n}, wherein the f t For the p t The corresponding resolution;

[0126] If the f t >f k , then the p is downsampled by the downsampling coefficient s t Downsampling is performed to convert the p t Updated to the image p after the downsampling process t ′ , and generate an updated original cervical cancer pathology image set P ′ , wherein the f k To preset the image resolution threshold, the downsampling coefficient s is determined according to Formula 1, which is:

[0127]

[0128] The acquisition module 310 is further configured to respond to the feature identification instruction to acquire the updated original cervical cancer pathology image set P ′ The feature identifiers on each original cervical cancer pathology image are obtained, and corresponding cervical cancer pathology image samples are formed to generate the preset cervical cancer pathology image dataset.

[0129] Optionally, the processing module 320 is specifically configured to:

[0130] The original cervical cancer pathology image set P ′ The original cervical cancer pathology images are gray-scaled and the gray-scaled original cervical cancer pathology images are segmented into a set of image blocks. Wherein, m is the number of image blocks in a first direction and a second direction of the image, and the first direction is orthogonal to the second direction;

[0131] In response to acting on the updated original cervical cancer pathology image set P ′A contour drawing instruction on each original cervical cancer pathology image in the image block set W is used to determine an abnormal region set from the image block set W, wherein the image blocks in the abnormal region set are image blocks within the abnormal region contour line range and image blocks having intersections with the abnormal region contour line, and the abnormal region contour line is a contour line generated in response to the contour drawing instruction;

[0132] According to Formula 2, the feature point coordinates (c x ,c y ), the formula 2 is:

[0133]

[0134] Among them, x ij is the horizontal coordinate of the pixel numbered i in the first direction and j in the second direction in the image block, ij The gray value of the pixel with sequence number i in the first direction and sequence number j in the second direction in the image block, y ij is the ordinate of the pixel numbered i in the first direction and j in the second direction in the image block;

[0135] If the feature point coordinates (c x ,c y ) is outside the abnormal region contour line, then the adjacent image blocks of the corresponding image block are merged into the abnormal region set to generate an extended abnormal region set;

[0136] The feature identifier is generated according to the set of extended abnormal regions, and a corresponding cervical cancer pathology image sample is formed. The feature identifier is used to identify the extended abnormal region.

[0137] Optionally, the neural network model is a graph neural network, and the graph neural network includes multiple layers of main neighborhood aggregation layers, wherein the output of the current layer of the main neighborhood aggregation layer in the multiple layers of the main neighborhood aggregation layer is the input of the next layer of the main neighborhood aggregation layer, and the node feature of the g-th node output by the next layer of the main neighborhood aggregation layer is The determination is made according to Formula 3, which is:

[0138]

[0139] in, is the node feature of the g-th node output by the main neighborhood aggregation layer of the current layer, M is the perceptron prediction layer, E is the edge between any nodes, is the node feature of the adjacent nodes of the g-th node on edge E, U is the node aggregation operation, is the tensor product.

[0140] Optionally, the acquisition module 310 is further used to acquire a recognition result feedback instruction acting on the target terminal, wherein the recognition result feedback instruction includes a recognition result confirmation instruction and a recognition result modification instruction, wherein the recognition result confirmation instruction is used to confirm the cervical cancer pathology image recognition result, and the recognition result modification instruction is used to perform at least one modification operation of position adjustment, range adjustment and deletion of the feature identification frame on the cervical cancer pathology image recognition result;

[0141] If the recognition result feedback instruction is the recognition result modification instruction, generating a cervical cancer pathology image recognition modification result according to the recognition result modification instruction, and adding the cervical cancer pathology image recognition modification result to the preset cervical cancer pathology image dataset to generate an updated cervical cancer pathology image dataset;

[0142] The preset cervical cancer pathology image recognition model is updated and trained using the updated cervical cancer pathology image dataset.

[0143] Figure 4 is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present application. Figure 4 As shown, an electronic device 400 provided in this embodiment includes: a processor 401 and a memory 402; wherein:

[0144] The memory 402 is used to store computer programs, and the memory may also be a flash memory.

[0145] The processor 401 is used to execute the execution instructions stored in the memory to implement each step in the above method. For details, please refer to the relevant description in the above method embodiment.

[0146] Optionally, the memory 402 may be independent or integrated with the processor 401 .

[0147] When the memory 402 is a device independent of the processor 401, the electronic device 400 may further include:

[0148] The bus 403 is used to connect the memory 402 and the processor 401 .

[0149] This embodiment further provides a readable storage medium, in which a computer program is stored. When at least one processor of an electronic device executes the computer program, the electronic device executes the methods provided in the above-mentioned various implementation modes.

[0150] This embodiment also provides a program product, which includes a computer program stored in a readable storage medium. At least one processor of the electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device implements the methods provided in the above various embodiments.

[0151] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the claims.

[0152] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for recognizing cervical cancer pathological images, characterized in that: include: Acquire cervical cancer pathology information of the target object, wherein the cervical cancer pathology information includes identity information and cervical cancer pathology images acquired by a digital pathology scanner; The cervical cancer pathology image is identified by using a preset cervical cancer pathology image recognition model to identify a characteristic area, wherein the characteristic area is a partial image area in the cervical cancer pathology image, the preset cervical cancer pathology image recognition model is a neural network model trained based on a preset cervical cancer pathology image data set, and the cervical cancer pathology image samples in the preset cervical cancer pathology image data set include an original cervical cancer pathology image and a characteristic identifier annotated on the original cervical cancer pathology image, and the characteristic identifier is used to characterize that the corresponding area on the original cervical cancer pathology image is in an abnormal state; Displaying a cervical cancer pathology image recognition result on a target terminal, the cervical cancer pathology image recognition result including the cervical cancer pathology image and a feature identification frame, the feature identification frame being used to identify the feature area on the cervical cancer pathology image, the target terminal being a terminal device determined according to the identity identification information; Before the cervical cancer pathology image is recognized by using the preset cervical cancer pathology image recognition model, the method further includes: The original cervical cancer pathology image set P = {p1, p2, ..., p t ,…,p n }, where p t is the t-th original cervical cancer pathology image in the original cervical cancer pathology image set P, and n is the number of original cervical cancer pathology images in the original cervical cancer pathology image set P; Obtain the image resolution set F corresponding to the original cervical cancer pathology image set P = {f1, f2, ..., f t ,…,f n }, wherein the f t For the p t The corresponding resolution; If the f t >f k , then the p is downsampled by the downsampling coefficient s t Downsampling is performed to convert the p t Updated to the image p after the downsampling process t ', and generate an updated original cervical cancer pathology image set P', where the f k To preset the image resolution threshold, the downsampling coefficient s is determined according to Formula 1, which is: In response to the feature identification instruction, acquiring feature identifications on each original cervical cancer pathology image in the updated original cervical cancer pathology image set P', and forming corresponding cervical cancer pathology image samples to generate the preset cervical cancer pathology image data set; The step of obtaining, in response to the feature identification instruction, feature identifications on each original cervical cancer pathology image in the updated original cervical cancer pathology image set P' and forming corresponding cervical cancer pathology image samples includes: Perform grayscale processing on each original cervical cancer pathology image in the updated original cervical cancer pathology image set P', and divide the original cervical cancer pathology images after grayscale processing into a set of image blocks Wherein, m is the number of image blocks in a first direction and a second direction of the image, and the first direction is orthogonal to the second direction; In response to a contour drawing instruction on each original cervical cancer pathology image in the updated original cervical cancer pathology image set P', an abnormal region set is determined from the image block set W, where the image blocks in the abnormal region set are a set of image blocks consisting of image blocks within the abnormal region contour line range and image blocks having intersections with the abnormal region contour line, and the abnormal region contour line is a contour line generated in response to the contour drawing instruction; According to Formula 2, the feature point coordinates (c x ,c y ), the formula 2 is: Among them, x ij is the horizontal coordinate of the pixel numbered i in the first direction and j in the second direction in the image block, ij The gray value of the pixel with sequence number i in the first direction and sequence number j in the second direction in the image block, y ij is the ordinate of the pixel numbered i in the first direction and j in the second direction in the image block; If the feature point coordinates (c x ,c y ) is outside the abnormal region contour line, then the adjacent image blocks of the corresponding image block are merged into the abnormal region set to generate an extended abnormal region set; The feature identifier is generated according to the set of extended abnormal regions, and a corresponding cervical cancer pathology image sample is formed. The feature identifier is used to identify the extended abnormal region.

2. The cervical cancer pathology image recognition method according to claim 1, characterized in that: The displaying of the cervical cancer pathology image recognition result on the target terminal includes: Displaying the cervical cancer pathology image on the target terminal, and displaying the feature identification frame in the feature area of ​​the cervical cancer pathology image, wherein the display position of the geometric center point of the feature identification frame on the target terminal coincides with the geometric center point on the display of the target terminal; A feature prompt mark is displayed on the feature mark frame, so that when the feature prompt mark is triggered, a feature magnified image is displayed on the target terminal, and the feature magnified image is an enlarged image of the feature area.

3. The cervical cancer pathology image recognition method according to claim 2, characterized in that: If there are multiple feature areas, the initial display image of the display includes all the feature areas and the feature identification boxes corresponding to each feature area, and the geometric center point on the display coincides with the geometric center point of the feature identification box corresponding to the feature area with the largest area among the multiple feature areas.

4. The cervical cancer pathology image recognition method according to claim 1, characterized in that: The neural network model is a graph neural network, which includes multiple layers of main neighborhood aggregation layers, wherein the output of the current main neighborhood aggregation layer in the multiple layers of main neighborhood aggregation layers is the input of the next main neighborhood aggregation layer, and the node feature of the g-th node output by the next main neighborhood aggregation layer is The determination is made according to Formula 3, which is: in, is the node feature of the g-th node output by the main neighborhood aggregation layer of the current layer, M is the perceptron prediction layer, E is the edge between any nodes, is the node feature of the adjacent nodes of the g-th node on edge E, U is the node aggregation operation, is the tensor product.

5. The method for recognizing cervical cancer pathological images according to any one of claims 1 to 3, characterized in that: After displaying the cervical cancer pathology image recognition result on the target terminal, it also includes: Acquire a recognition result feedback instruction acting on the target terminal, wherein the recognition result feedback instruction includes a recognition result confirmation instruction and a recognition result modification instruction, wherein the recognition result confirmation instruction is used to confirm the cervical cancer pathology image recognition result, and the recognition result modification instruction is used to perform at least one modification operation of position adjustment, range adjustment, and deletion of the feature identification frame on the cervical cancer pathology image recognition result; If the recognition result feedback instruction is the recognition result modification instruction, generating a cervical cancer pathology image recognition modification result according to the recognition result modification instruction, and adding the cervical cancer pathology image recognition modification result to the preset cervical cancer pathology image dataset to generate an updated cervical cancer pathology image dataset; The preset cervical cancer pathology image recognition model is updated and trained using the updated cervical cancer pathology image dataset.

6. A cervical cancer pathology image recognition device, characterized in that: An acquisition module, used to acquire cervical cancer pathology information of a target object, wherein the cervical cancer pathology information includes identity information and a cervical cancer pathology image acquired by a digital pathology scanner; A processing module, used to identify the cervical cancer pathology image by using a preset cervical cancer pathology image recognition model to identify a characteristic area, wherein the characteristic area is a partial image area in the cervical cancer pathology image, the preset cervical cancer pathology image recognition model is a neural network model trained based on a preset cervical cancer pathology image data set, the cervical cancer pathology image samples in the preset cervical cancer pathology image data set include an original cervical cancer pathology image and a characteristic identifier annotated on the original cervical cancer pathology image, and the characteristic identifier is used to indicate that the corresponding area on the original cervical cancer pathology image is in an abnormal state; A display module, used for displaying a cervical cancer pathology image recognition result on a target terminal, wherein the cervical cancer pathology image recognition result includes the cervical cancer pathology image and a feature identification frame, wherein the feature identification frame is used for identifying the feature area on the cervical cancer pathology image, and the target terminal is a terminal device determined according to the identity identification information; The acquisition module is also used to acquire the original cervical cancer pathology image set P = {p1, p2, ..., p t ,…,p n }, where p t is the t-th original cervical cancer pathology image in the original cervical cancer pathology image set P, and n is the number of original cervical cancer pathology images in the original cervical cancer pathology image set P; The acquisition module is also used to acquire the image resolution set F corresponding to the original cervical cancer pathology image set P = {f1, f2, ..., f t ,…,f n }, wherein the f t For the p t The corresponding resolution; If the f t >f k , then the p is downsampled by the downsampling coefficient s t Downsampling is performed to convert the p t Updated to the image p after the downsampling process t ', and generate an updated original cervical cancer pathology image set P', where the f k To preset the image resolution threshold, the downsampling coefficient s is determined according to Formula 1, which is: The acquisition module is further used to respond to the feature identification instruction, obtain the feature identification on each original cervical cancer pathology image in the updated original cervical cancer pathology image set P', and form corresponding cervical cancer pathology image samples to generate the preset cervical cancer pathology image data set; The processing module is specifically used for: Perform grayscale processing on each original cervical cancer pathology image in the updated original cervical cancer pathology image set P', and divide the original cervical cancer pathology images after grayscale processing into a set of image blocks Wherein, m is the number of image blocks in a first direction and a second direction of the image, and the first direction is orthogonal to the second direction; In response to a contour drawing instruction on each original cervical cancer pathology image in the updated original cervical cancer pathology image set P', an abnormal region set is determined from the image block set W, where the image blocks in the abnormal region set are a set of image blocks consisting of image blocks within the abnormal region contour line range and image blocks having intersections with the abnormal region contour line, and the abnormal region contour line is a contour line generated in response to the contour drawing instruction; According to Formula 2, the feature point coordinates (c x ,c y ), the formula 2 is: Among them, x ij is the horizontal coordinate of the pixel numbered i in the first direction and j in the second direction in the image block, ij The gray value of the pixel with sequence number i in the first direction and sequence number j in the second direction in the image block, y ij is the ordinate of the pixel numbered i in the first direction and j in the second direction in the image block; If the feature point coordinates (c x ,c y ) is outside the abnormal region contour line, then the adjacent image blocks of the corresponding image block are merged into the abnormal region set to generate an extended abnormal region set; The feature identifier is generated according to the set of extended abnormal regions, and a corresponding cervical cancer pathology image sample is formed. The feature identifier is used to identify the extended abnormal region.

7. An electronic device, characterized in that: include: processor; as well as, A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 5 by executing the executable instructions.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 5 when executed by a processor.

Images