Pathological section processing method, computer device and storage medium

Abstract

The present application relates to a pathological slice processing method, a computer device and a storage medium. The method comprises: acquiring images of pathological slices according to acquisition parameters to obtain a plurality of corresponding digital images; naming the digital images according to the acquisition sequence; determining the position of the digital images according to the names of the digital images; splicing the digital images according to the positions of the digital images to obtain corresponding pathological slice images; classifying the pathological slice images using a preset neural network to obtain classification results; the neural network is pre-trained based on the pathological slice images of annotated pathological tissues. The use of this method can improve work efficiency.

Description

[0001] This invention patent application is a divisional application of the Chinese invention patent application with application date of August 28, 2019, application number 201910802010X, and name “Pathological section processing method, computer equipment and storage medium”. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to a pathological image-assisted analysis method, system, computer device and storage medium. Background Art

[0003] With the advent of the Industrial 4.0 era, mechanical automation in all walks of life is advancing rapidly, including in the field of medical imaging. Microscope images, as the gold standard in the field of medical imaging, have always been the direction of research and development in the medical industry. However, the existing pathological section reading method is usually that the user places the stained pathological section on the stage of the microscope, and searches and observes the pathological section by replacing the objective lens of different magnifications to give the result. For example, first, it is necessary to use a 10x objective lens to lock the tissue area, then replace it with a 20x objective lens to find the area of ​​interest from the tissue area, and finally replace it with a 40x objective lens to observe the detailed features of the cell morphology in the area of ​​interest, etc., which leads to a long reading time. In addition, in order to prevent reading errors, it is usually necessary for multiple users to read a pathological section until a unified result position is obtained, which makes the reading time doubled, resulting in reduced reading efficiency. Summary of the invention

[0004] Based on this, it is necessary to provide a pathology section processing method, computer equipment and storage medium that can improve work efficiency in response to the above technical problems.

[0005] A pathological section processing method, the method comprising:

[0006] Acquire images of pathological sections according to acquisition parameters to obtain a plurality of corresponding digital images;

[0007] naming the digitized images according to the order of acquisition;

[0008] determining the position of the digitized image according to the name of the digitized image;

[0009] splicing the digitized images according to their positions to obtain corresponding pathological slice images;

[0010] The pathological slice images are classified using a preset neural network to obtain a classification result; the neural network is pre-trained based on the pathological slice images of annotated pathological tissues.

[0011] In one embodiment, the acquisition parameter includes at least one of an acquisition mode, an acquisition size, and a moving step size;

[0012] The step of acquiring the pathological slices according to the acquisition parameters to obtain the corresponding digital images includes:

[0013] The pathological slices are traversed and collected according to at least one of the collection mode, collection size and moving step length to obtain corresponding digital images.

[0014] In one embodiment, the collection method includes any one or more of an S-type method and a Z-type method.

[0015] In one embodiment, the step of splicing the digitized images according to the positions of the digitized images to obtain corresponding pathological slice images includes:

[0016] Arranging the digitized images according to their positions to obtain images to be spliced;

[0017] The middle row image and the middle column image of the image to be spliced ​​are used as splicing boundaries, and the digitized images in the image to be spliced ​​are spliced ​​according to the splicing boundaries to obtain a pathological slice image corresponding to the pathological slice.

[0018] In one embodiment, the middle row image and the middle column image of the image to be stitched are used as stitching boundaries, and the digitized images in the image to be stitched are stitched according to the stitching boundaries to obtain the pathological slice image corresponding to the pathological slice, including:

[0019] splicing the digitized images in the middle row image and the middle column image to form a cross-shaped image;

[0020] Dividing the image to be stitched according to the boundaries of the cross-shaped image to obtain four stitching areas;

[0021] The digitized images in the stitching areas are stitched together respectively to obtain the pathological slice image corresponding to the pathological slice.

[0022] In one embodiment, the step of respectively splicing the digitized images in the splicing regions to obtain the pathological slice image corresponding to the pathological slice includes:

[0023] Determining repeated regions in adjacent digitized images in the stitched region;

[0024] Extracting feature points of the repeated area and feature descriptors corresponding to the feature points;

[0025] According to the feature descriptors, feature points of adjacent digitized images are matched to determine matching points;

[0026] Based on the matching points, adjacent digitized images are spliced, and the spliced ​​image is the pathological slice image corresponding to the pathological slice.

[0027] In one embodiment, before the digitized images are spliced ​​according to their positions to obtain corresponding pathological slice images, the method further includes:

[0028] Brightness correction is performed on the digitized image to obtain a brightness-corrected digitized image.

[0029] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above-mentioned pathological section processing methods when executing the computer program.

[0030] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of any one of the pathological section processing methods described above.

[0031] The above-mentioned pathological slice processing method, computer equipment and storage medium perform image acquisition on the pathological slice according to the received acquisition parameters, obtain multiple corresponding digital images of the pathological slice, thereby automatically acquiring the digital images corresponding to the pathological slice. The digital images are named according to the order of acquisition, and the digital images are spliced ​​after the positions are determined according to the names of the digital images to obtain the corresponding pathological slice images. Then, the pathological slice images are classified based on the neural network to obtain the classification results, so as to assist the user in making judgments. The user of the above method can realize the image acquisition and analysis of the pathological slice by inputting the acquisition parameters, without the need for the user to manually operate the microscope to read the slices, thereby improving work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A diagram showing an application environment of a pathological section processing method in one embodiment;

[0033] Figure 2 A schematic diagram of a process for processing a pathological section in one embodiment;

[0034] Figure 3 It is a flowchart of the steps of splicing digitized images according to the positions of the digitized images to obtain corresponding pathological slice images in one embodiment;

[0035] Figure 4 is a schematic diagram of images to be stitched in one embodiment;

[0036] Figure 5 A schematic diagram of a splicing method in one embodiment;

[0037] Figure 6 A schematic flow chart of the steps of splicing digitized images in a splicing area to obtain a pathological slice image corresponding to a pathological slice in one embodiment;

[0038] Figure 7 A schematic diagram of the structure of a neural network in one embodiment;

[0039] Figure 8 is a structural block diagram of a pathological slice processing device in one embodiment;

[0040] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0042] The pathological section processing method provided in this application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the microscope 104 through the network. The terminal 102 controls the microscope 104 to collect images of pathological sections through acquisition parameters to obtain multiple corresponding digital images. The terminal 102 names the digital images according to the acquisition order; the terminal 102 determines the position of the digital image according to the name of the digital image; splices the digital image according to the position of the digital image to obtain the corresponding pathological section image; the terminal 102 uses a preset neural network to classify the pathological section image to obtain a classification result; the neural network is pre-trained based on the pathological section image of the annotated pathological tissue. Among them, the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablets and portable wearable devices, and the microscope 104 is an automatic microscope, such as a micro-motion platform microscope. It can be understood that the microscope 104 is a microscope capable of automatic scanning.

[0043] In one embodiment, Figure 2 As shown, a pathological section processing method is provided, which is applied to Figure 1 The terminal in is used as an example to illustrate, including the following steps:

[0044] Step 202: perform image acquisition on the pathological slice according to acquisition parameters to obtain a plurality of corresponding digital images.

[0045] Among them, pathological sections are a type of pathological specimens, which are made by taking a certain size of diseased tissue using pathological histological methods. Acquisition parameters are information used to indicate how to acquire images of pathological sections. Image acquisition refers to the method of obtaining digital images, for example, by scanning or photographing pathological sections to obtain digital images corresponding to pathological sections.

[0046] Specifically, when it is necessary to analyze the prepared pathological slices, the user configures the acquisition parameters of the image acquisition through the terminal connected to the microscope, and sends the acquisition instruction to the microscope through the terminal. After the terminal receives the acquisition instruction of the user, it responds to the acquisition instruction and controls the microscope to perform image acquisition on the pathological slice according to the acquisition parameters configured by the user, thereby obtaining a plurality of corresponding digital images. For example, the user first places the pathological slice on the stage in the microscope. It should be understood that the stage in the microscope is a stage that has been loaded with a micro-motion platform. The micro-motion platform refers to a device that can move in the X direction and the Y direction. By providing a motor in the X direction and the Y direction of the micro-motion platform, the micro-motion platform is controlled to move in the X-axis and the Y-axis based on the motor. When performing image acquisition, the terminal controls the micro-motion platform to move through the acquisition parameters so that the acquired area of ​​the pathological slice on the stage can be within the shooting range of the camera, and at the same time controls the camera installed in the microscope to shoot the acquired area of ​​the pathological slice to acquire the digital image corresponding to the acquired area.

[0047] In one embodiment, the acquisition parameters include at least one or more of an acquisition mode, an acquisition size, and a moving step length. Capturing images of pathological slices according to the acquisition parameters to obtain a plurality of corresponding digitized images specifically includes: traversing and acquiring pathological slices according to the acquisition mode, the acquisition size, and the moving step length to obtain corresponding digitized images.

[0048] Among them, the acquisition method refers to how to control the movement of the micro-motion platform, including but not limited to one or more of S-type and Z-type. It can be understood that the movement direction of the micro-motion platform is controlled by the acquisition method. The acquisition size refers to the size of the acquired area, which can be understood as the range of acquisition, that is, the size of the slice area within the shooting range each time the pathological slice is acquired. The moving step refers to the distance of movement, such as the distance of controlling the micro-motion platform to move once. The repeated areas between the upper and lower and left and right adjacent images can be determined by different moving steps. For example, the acquisition size includes length and width. If the moving step is less than the length and width, an overlapping acquisition area can be formed, and the two adjacent images naturally have repeated areas during acquisition. If the moving step is equal to the length and width, the overlapping area cannot be formed, which is not conducive to splicing into a complete image, or the moving step is greater than the length and width, then a blind area for acquisition will be formed, resulting in missed acquisition, resulting in incomplete image information.

[0049] Specifically, when the image of the pathological slice is collected according to the acquisition parameters, the direction and distance of the movement of the micro-motion platform are controlled by the acquisition mode and the moving step length in the acquisition parameters, and the acquisition is performed according to the acquisition size. It can be understood that the pathological slice is collected once each time the micro-motion platform moves, until every area of ​​the pathological slice is collected, that is, the traversal acquisition is completed, and multiple corresponding digital images are obtained.

[0050] Step S204: naming the digitized images according to the acquisition sequence.

[0051] The acquisition order refers to the order in which the digital images are acquired, including but not limited to from top to bottom, from left to right, from bottom to top, and from right to left. The acquisition order and the acquisition method jointly determine the movement of the micro-motion platform during image acquisition. For example, if the acquisition method is S-type and the acquisition order is from right to left and from top to bottom, the movement of the micro-motion platform is a forward S. Conversely, if the acquisition order is from bottom to top and from left to right, the micro-motion platform is an inverted S. In other words, by acquiring pathological sections row by row, the corresponding digital images are obtained. Therefore, the naming of digital images according to the acquisition order can be understood as naming according to the number of rows and columns in which the digital image is located. For example, the name of a digital image can be "x rows and x columns".

[0052] Step S206: determining the position of the digitized image according to the name of the digitized image.

[0053] Step S208, splicing the digitized images according to their positions to obtain corresponding pathological slice images.

[0054] Wherein, stitching refers to stitching two or more images into one image. In this embodiment, all the collected digitized images are stitched into one complete image. Since the digitized image represents an image of a certain area in the pathological section, when all the digitized images are stitched into one complete image, the complete image is the digitized image corresponding to the pathological section, that is, the pathological section image.

[0055] Specifically, after the position of the digitized image is determined according to the name of the digitized image, the digitized image is arranged according to the position of each digitized image, and the adjacent digitized images are spliced ​​according to the order of arrangement. When all the digitized images are spliced ​​with the adjacent digitized images, the corresponding pathological slice image is obtained. Since the digitized images are named according to the acquisition order after acquisition, the adjacent relationship between the digitized images can be determined and the correct splicing order can be obtained through the name arrangement, so as to accurately splice into a complete image.

[0056] Step S210, classifying the pathological slice images using a preset neural network to obtain classification results; the neural network is pre-trained based on the pathological slice images of annotated pathological tissues.

[0057] The full name of neural network is artificial neural network, which is an algorithmic mathematical model that imitates the behavioral characteristics of animal neural networks and performs distributed parallel information processing. In this embodiment, the neural network is pre-trained based on pathological slice images of annotated pathological tissues.

[0058] Specifically, when a pathological slice image is obtained, a pre-trained neural network is called. The pathological slice image is input into the neural network, and the pathological slice image is subjected to convolution and pooling operations by the neural network to determine the category of the pathological tissue in the pathological slice image, and obtain a classification result, thereby assisting the user to further analyze the pathological tissue in the pathological slice. Among them, the neural network can be any one or more of the networks such as ResNet (Residual Neural Network), VGG (Visual Geometry Group Network), GoogleNet (Google InceptionNet), DenseNet (Densely Connected Convolutional Networks), etc.

[0059] The above-mentioned pathological slice processing method performs image acquisition on the pathological slice according to the received acquisition parameters, obtains multiple corresponding digital images of the pathological slice, and thus realizes automatic acquisition of the digital images corresponding to the pathological slice. The digital images are named according to the order of acquisition, and the digital images are spliced ​​after the positions are determined according to the names of the digital images to obtain the corresponding pathological slice images. Then, the pathological slice images are classified based on the neural network to obtain the classification results, so as to assist the user in making judgments. The above-mentioned method allows the user to realize the image acquisition and analysis of the pathological slice by inputting the acquisition parameters, without the need for the user to manually operate the microscope for reading the slices, thereby improving work efficiency.

[0060] In one embodiment, Figure 3 As shown, the digitized images are spliced ​​according to their positions to obtain corresponding pathological slice images, including the following steps:

[0061] Step S302, arranging the digitized images according to their positions to obtain images to be spliced.

[0062] like Figure 4As shown, a schematic diagram of images to be spliced ​​is provided. Specifically, according to the name of the digitized image, the position of the digitized image is determined, and then the digitized image is arranged according to the position of the digitized image to form an image including multiple rows and columns of digitized images, which is the image to be spliced. For example, if the name of the digitized image is "one row and one column", the position of the digitized image is at the first row and first column of the image to be spliced.

[0063] Step S304 , taking the middle row image and the middle column image of the image to be spliced ​​as the splicing boundary, splicing the digitized images in the image to be spliced ​​according to the splicing boundary to obtain the pathological slice image corresponding to the pathological slice.

[0064] Among them, the middle row is the row between the first row and the last row, excluding the first row and the last row. The middle row image refers to the digitized image in the row. The middle column is the column between the first column and the last column, excluding the first column and the last column. The middle column image refers to the digitized image in the column.

[0065] Specifically, after determining the position of the digitized image according to the name of the digitized image, after arranging the digitized image according to the position of the digitized image, select any row except the first row and the last row as the middle row, and select any column except the first column and the last column as the middle column. The digitized image in the middle row and the digitized image in the middle column are used as the splicing boundary of image splicing, and the digitized images in the image to be spliced ​​are spliced ​​according to the splicing boundary. It can be understood that the splicing is started with the splicing boundary as the starting splicing point. When all the digitized images in the image to be spliced ​​are spliced, the obtained image is the pathological section image corresponding to the pathological section.

[0066] In this embodiment, after the positions are determined by the names of the digitized images, the digitized images are arranged to obtain the images to be spliced, and then spliced, so that the digitized images are in the correct connection order, thereby preventing splicing errors and improving the accuracy of splicing.

[0067] In one embodiment, the middle row image and the middle column image of the image to be stitched are used as stitching boundaries, and the digitized images in the image to be stitched are stitched according to the stitching boundaries to obtain the pathological slice image corresponding to the pathological slice. Specifically, the process includes: stitching the digitized images in the middle row image and the middle column image to form a cross-shaped image; dividing the image to be stitched according to the boundaries of the cross-shaped image to obtain four stitching areas; and stitching the digitized images in each stitching area to obtain the pathological slice image corresponding to the pathological slice.

[0068] The cross-shaped image refers to an image that is shaped like a cross. Since the rows and columns are in a vertical cross relationship, in this embodiment, the image obtained by splicing the images of any row and any column is a cross-shaped image.

[0069] Specifically, Figure 5 As shown, a schematic diagram of a splicing method is provided. Figure 5 , after determining the middle row ① and the middle column ②, firstly, the digitized images in the middle row ① and the middle column ② are spliced ​​to obtain a cross-shaped image composed of ① and ②. The digitized images in the middle row ① and the middle column ② are spliced ​​by determining the repeated area between the adjacent digitized images in the middle row ① and the middle column ②, and determining the matching points that can be spliced ​​according to the feature points in the repeated area and the feature descriptors corresponding to the feature points. The digitized images are spliced ​​together according to the determined matching points. Then, with the middle row ① and the middle column ② as the dividing line, the entire image to be spliced ​​is divided into four splicing areas, namely Figure 5 After obtaining the four stitching areas divided by the cross-shaped image, the images in each stitching area are stitched independently. The stitching can be performed based on the cross-shaped boundary included in each area as the starting boundary, or any group of adjacent digitized images in the stitching area can be stitched. When the digitized images in the stitching area are stitched, a complete pathological slice image is obtained.

[0070] In this embodiment, the stitching work is completed by dividing the image into four areas and stitching them independently, which can avoid the phenomenon of inaccurate image stitching caused by frequent blank areas or blurred areas at the boundaries of the image, thereby improving the accuracy of stitching.

[0071] In one embodiment, Figure 6 As shown, stitching the digitized images in the stitching area to obtain the pathological slice image corresponding to the pathological slice includes the following steps:

[0072] Step S602: determining repeated regions in adjacent digitized images in the stitching region.

[0073] The repeated area refers to the same area between adjacent images, and the size of the repeated area is determined by the moving step of the acquisition parameter.

[0074] Specifically, when performing image stitching, the overlapping area can be obtained from adjacent digitized images by using the moving step length of the acquisition parameters. For example, if two digitized images are adjacent to each other, the overlapping area is obtained from the lower boundary of one of the digitized images and from the upper boundary of the other digitized image according to the moving step length. Alternatively, the overlapping area is obtained by performing intersection processing on two adjacent digitized images.

[0075] Step S604: extracting feature points of the repeated area and feature descriptors corresponding to the feature points.

[0076] There is a one-to-one correspondence between feature points and feature descriptors, and the number of feature points and feature descriptors is the same. Feature points can also be called key points, and feature descriptors are used to describe some parameters of feature points.

[0077] Specifically, after obtaining the repeated region of the adjacent digitized images, the feature points and feature descriptors in the repeated region of the two adjacent digitized images are extracted using a preset feature extraction algorithm. It should be understood that since the repeated region exists in both adjacent digitized images, the feature points and feature descriptors in the repeated region of each of the two adjacent digitized images are extracted. The feature extraction algorithm includes but is not limited to the SURF algorithm, SIFT feature extraction algorithm, Harris corner feature extraction algorithm, ORB feature extraction algorithm, FAST algorithm, etc.

[0078] Step S606: matching feature points of adjacent digitized images according to the feature descriptors to determine matching points.

[0079] Among them, since the feature descriptor is a parameter used to describe the feature point, that is, through the feature descriptors of the feature points in the repeated areas of the two adjacent images, the same feature points in the repeated areas can be matched and determined, and the same feature points are matching points. It can be understood that the matching point includes two feature points, that is, a pair of feature points, and the two feature points come from two adjacent images respectively.

[0080] Specifically, after obtaining the feature points and corresponding feature descriptors of the repeated regions of the two adjacent digital images, the distance between the feature descriptors in the repeated regions of the two adjacent digital images can be calculated using a Hamming distance algorithm, etc., so as to obtain the distance values ​​between multiple feature descriptor pairs. The calculated multiple distance values ​​are compared with a preset distance threshold. When the calculated distance value is greater than the preset distance threshold, the feature descriptor pair and the corresponding feature point corresponding to the distance are eliminated. When the calculated distance value is less than or equal to the preset distance threshold, the feature descriptor pair and the corresponding feature point pair corresponding to the distance are retained, and finally multiple retained feature point pairs and feature descriptor pairs are obtained, and the retained feature point pairs are the matching points obtained by matching.

[0081] In addition, if more accurate matching points are needed, the matching points obtained by matching can be matched again using a preset matching algorithm, wherein the preset matching algorithm includes but is not limited to a RANSAC (random sample consensus) algorithm, a KNN-matching (K-nearest neighbor matching) algorithm, a BFMatcher (brute force matching) algorithm, and the like.

[0082] Step S608: splicing adjacent digitized images based on matching points, and the spliced ​​image is a pathological slice image corresponding to the pathological slice.

[0083] Specifically, when extracting feature points and feature descriptors using a preset feature extraction algorithm, the coordinates of each feature point can also be extracted, and the coordinates can be understood as coordinates on the image coordinate system. That is, the obtained matching points also have corresponding coordinates. After calculating the relative displacement of the matching points through the coordinates, the relative displacement corresponding to the repeated area of ​​the two adjacent digitized images is calculated by the relative displacement of the matching points and the weight corresponding to the matching points. The relative displacement corresponding to the repeated area is used as the target relative displacement between the two adjacent digitized images, and the two digitized images are spliced ​​using the target relative displacement.

[0084] The relative displacement of the matching points can be obtained by subtracting the coordinates of the two feature points in the matching points and taking the absolute value, and then the relative displacement between the two feature points can be obtained, that is, the relative displacement of the matching points. Similarly, the relative displacement of other matching points can also be calculated using this method. The weight can be calculated based on the distance between the feature descriptors in the above repeated area, and the calculation formula is as follows:

[0085]

[0086] Among them, d i represents the distance between the feature descriptors corresponding to the i-th matching point, w i represents the weight corresponding to the i-th matching point pair, and n is the number of matching points. After that, the weight corresponding to each matching point and the relative displacement between the two feature points in each matching point can be weighted and summed to obtain the relative displacement corresponding to the repeated area, which is used as the target relative displacement between two adjacent digitized images.

[0087] The target relative displacement includes the target vertical relative displacement and the target horizontal relative displacement. When splicing two adjacent digitized images, one of the two adjacent digitized images can be determined as a reference image and the other digitized image can be determined as a moving image. Afterwards, the moving image can be translated in the vertical direction by the target vertical relative displacement and in the horizontal direction by the target horizontal relative displacement based on the reference image, and the two adjacent digitized images can be spliced. It should be understood that when translating the moving image, it can be translated in the vertical direction first and then in the horizontal direction, or it can be translated in the horizontal direction first and then in the vertical direction.

[0088] In this embodiment, secondary matching is performed through feature points and feature descriptors, so that the matching degree of the obtained matching points is higher, so that when the relative displacement of the matching points is used to calculate the relative displacement of the repeated area, the obtained displacement is more accurate, and the relative displacement between the two adjacent digitized images is more accurate, so that the images can be stitched more accurately, thereby improving the quality of image stitching.

[0089] In one embodiment, the image is usually affected by factors such as ambient light and microscope beams during image acquisition, resulting in uneven illumination. Once uneven illumination occurs, the local brightness of the image will be inconsistent. Although it may not be easy to find the difference when looking at a single digitized image, when spliced ​​into a complete image, the complete image will form a rasterization phenomenon, thereby affecting the quality of the entire image. Therefore, after obtaining the digitized image, the digitized image is subjected to brightness correction processing, and then spliced ​​according to the digitized image after brightness correction, thereby preventing the rasterization phenomenon.

[0090] The brightness correction of the digitized image is performed to obtain the brightness-corrected digitized image, which specifically includes: respectively obtaining the brightness channel image in each digitized image; performing mean calculation on the image matrix corresponding to each brightness channel image to obtain the mean image; normalizing the mean image to generate a mask image; dividing the image matrix corresponding to the brightness channel image of each digitized image with the image matrix corresponding to the mask image, and the image corresponding to the obtained image matrix is ​​the brightness-corrected digitized image.

[0091] Specifically, the brightness channel image refers to the image of the brightness channel. Taking the color space as an example, it can be understood as the V channel image in the HSV color space and the Y channel image in the YIQ color space. Taking the image matrix form as an example, it can be understood as extracting the matrix dimension corresponding to the brightness in the image matrix. After the brightness channel image is separated, the image matrices corresponding to all brightness channel images are accumulated and the average value is taken. The resulting image matrix is ​​the mean image. Then, the mean image is normalized to obtain a mask image. An image in which the pixel value of each pixel in the image is from 0 to 1 is the mask image. Therefore, normalization can be understood as changing each pixel in the mean image from 0-255 to within the range of 0-1. Among them, separation of the brightness channel image and normalization processing can be completed by calling relevant image processing tools, such as MATLAB, OpenCV, etc. Then, the brightness channel image in the digitized image is divided by the obtained mask image, and the image obtained by dividing is the brightness corrected image. It can be understood that the image matrix corresponding to the brightness channel image in the digitized image is divided by the image matrix corresponding to the mask image, and the image corresponding to the obtained image matrix is ​​the image after brightness correction.

[0092] In this embodiment, brightness correction is performed on digitized images, and the digitized images after brightness correction are spliced ​​to prevent the images from being rasterized, thereby improving the quality of the images.

[0093] In one embodiment, Figure 7 As shown, a schematic diagram of the structure of a neural network is provided, based on Figure 7 The neural network shown in the figure classifies the pathological slice image and obtains the classification result. Figure 7 , the neural network structure includes four densely connected modules (Dense Block), four convolutional layers, four pooling layers, and one activation layer. Specifically, the obtained pathological slice image is input into the neural network, and feature extraction is performed through the convolutional layer, densely connected module and pooling layer of the neural network to obtain the feature map output by the third densely connected module and the feature map output by the last densely connected module. The feature map output by the last densely connected module is upsampled to obtain a new feature map, and the new feature map is feature fused with the feature map output by the third densely connected module to obtain a fused feature. The fused feature will be further pooled and passed through the activation layer to output the classification result. Among them, feature fusion includes feature addition and feature merging.

[0094] It should be understood that although Figure 2 , 3 The steps in the flowchart of 6 are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2 , 3 At least part of the steps in 6 may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0095] In one embodiment, Figure 8 As shown, a pathological slice processing device is provided, including: an acquisition module 802, a generation module 804, a determination module 806, a splicing module 808 and a classification module 810, wherein:

[0096] The acquisition module 802 is used to acquire images of the pathological sections according to acquisition parameters to obtain a plurality of corresponding digital images.

[0097] The generating module 804 is used to name the digitized images according to the acquisition sequence.

[0098] A determination module 806, configured to determine a position of the digitized image according to the name of the digitized image;

[0099] The stitching module 808 is used to stitch the digitized images according to their positions to obtain corresponding pathological slice images.

[0100] The classification module 810 is used to classify the pathological slice images using a preset neural network to obtain classification results; the neural network is pre-trained based on the pathological slice images of annotated pathological tissues.

[0101] In one embodiment, the acquisition module 802 is further used to traverse and acquire the pathological slices according to at least one of the acquisition mode, acquisition size, and moving step length to obtain a corresponding digital image.

[0102] In one embodiment, the stitching module 808 is further used to arrange the digitized images according to their positions to obtain images to be stitched; use the middle row image and the middle column image of the images to be stitched as stitching boundaries, and stitch the digitized images in the images to be stitched according to the stitching boundaries to obtain a pathological slice image corresponding to the pathological slice.

[0103] In one embodiment, the stitching module 808 is also used to stitch the digitized images in the middle row image and the middle column image to form a cross-shaped image; divide the image to be stitched according to the boundaries of the cross-shaped image to obtain four stitching areas; and stitch the digitized images in each stitching area respectively to obtain a pathological slice image corresponding to the pathological slice.

[0104] In one embodiment, the stitching module 808 is also used to determine the repeated areas in adjacent digitized images in the stitching area; extract the feature points of the repeated areas and the feature descriptors corresponding to the feature points; match the feature points of adjacent digitized images according to the feature descriptors to determine the matching points; and stitch the adjacent digitized images based on the matching points, and the stitched image is the pathological slice image corresponding to the pathological slice.

[0105] In one embodiment, the pathological slice processing device further includes a correction module for performing brightness correction on the digitized image to obtain a brightness-corrected digitized image.

[0106] In one embodiment, the correction module is also used to respectively obtain the brightness channel images in each digitized image; perform mean calculation on the image matrix corresponding to each brightness channel image to obtain a mean image; normalize the mean image to generate a mask image; divide the image matrix corresponding to the brightness channel image of each digitized image by the image matrix corresponding to the mask image, and the image corresponding to the obtained image matrix is ​​the digitized image after brightness correction.

[0107] The specific definition of the pathological slice processing device can be found in the definition of the pathological slice processing method above, which will not be repeated here. Each module in the above-mentioned pathological slice processing device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0108] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig. 9 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a pathological section processing method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0109] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0110] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0111] Acquire images of pathological sections according to acquisition parameters to obtain a plurality of corresponding digital images;

[0112] naming the digitized images according to the order of acquisition;

[0113] Determine the location of the digitized image based on the name of the digitized image;

[0114] splicing the digitized images according to their positions to obtain corresponding pathological slice images;

[0115] The pathological slice images are classified using a preset neural network to obtain classification results; the neural network is pre-trained based on the pathological slice images of annotated pathological tissues.

[0116] In one embodiment, when the processor executes the computer program, the following steps are further implemented: traversing and acquiring the pathological slices according to at least one of the acquisition mode, acquisition size, and moving step length to obtain corresponding digitized images.

[0117] In one embodiment, when the processor executes the computer program, the following steps are also implemented: arranging the digitized images according to the positions of the digitized images to obtain images to be stitched; using the middle row image and the middle column image of the images to be stitched as stitching boundaries, and stitching the digitized images in the images to be stitched according to the stitching boundaries to obtain a pathological slice image corresponding to the pathological slice.

[0118] In one embodiment, when the processor executes the computer program, the following steps are also implemented: splicing the digitized images in the middle row image and the middle column image to form a cross-shaped image; dividing the image to be spliced ​​according to the boundaries of the cross-shaped image to obtain four splicing areas; and splicing the digitized images in each splicing area to obtain a pathological slice image corresponding to the pathological slice.

[0119] In one embodiment, when the processor executes the computer program, the following steps are also implemented: determining repeated areas in adjacent digitized images in the stitching area; extracting feature points of the repeated areas and feature descriptors corresponding to the feature points; matching feature points of adjacent digitized images based on the feature descriptors to determine matching points; stitching adjacent digitized images based on the matching points, and the stitched image is a pathological slice image corresponding to the pathological slice.

[0120] In one embodiment, when the processor executes the computer program, the following steps are further implemented: brightness correction is performed on the digitized image to obtain a brightness-corrected digitized image.

[0121] In one embodiment, when the processor executes the computer program, the following steps are also implemented: respectively obtaining the brightness channel images in each digitized image; performing mean calculation on the image matrix corresponding to each brightness channel image to obtain a mean image; normalizing the mean image to generate a mask image; dividing the image matrix corresponding to the brightness channel image of each digitized image by the image matrix corresponding to the mask image, and the image corresponding to the obtained image matrix is ​​the brightness-corrected digitized image.

[0122] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0123] Acquire images of pathological sections according to acquisition parameters to obtain a plurality of corresponding digital images;

[0124] naming the digitized images according to the order of acquisition;

[0125] Determine the location of the digitized image based on the name of the digitized image;

[0126] splicing the digitized images according to their positions to obtain corresponding pathological slice images;

[0127] The pathological slice images are classified using a preset neural network to obtain classification results; the neural network is pre-trained based on the pathological slice images of annotated pathological tissues.

[0128] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: traversing and acquiring the pathological slices according to at least one of the acquisition mode, acquisition size, and moving step length to obtain corresponding digitized images.

[0129] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: arranging the digitized images according to the positions of the digitized images to obtain images to be stitched; using the middle row image and the middle column image of the images to be stitched as stitching boundaries, and stitching the digitized images in the images to be stitched according to the stitching boundaries to obtain a pathological slice image corresponding to the pathological slice.

[0130] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: splicing the digitized images in the middle row image and the middle column image to form a cross-shaped image; dividing the image to be spliced ​​according to the boundaries of the cross-shaped image to obtain four splicing areas; and splicing the digitized images in each splicing area to obtain a pathological slice image corresponding to the pathological slice.

[0131] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: determining repeated areas in adjacent digitized images in the stitching area; extracting feature points of the repeated areas and feature descriptors corresponding to the feature points; matching feature points of adjacent digitized images based on the feature descriptors to determine matching points; stitching adjacent digitized images based on the matching points, and the stitched image is a pathological slice image corresponding to the pathological slice.

[0132] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: brightness correction is performed on the digitized image to obtain a brightness-corrected digitized image.

[0133] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: respectively obtaining the brightness channel images in each digitized image; performing mean calculation on the image matrix corresponding to each brightness channel image to obtain a mean image; normalizing the mean image to generate a mask image; dividing the image matrix corresponding to the brightness channel image of each digitized image by the image matrix corresponding to the mask image, and the image corresponding to the obtained image matrix is ​​the brightness-corrected digitized image.

[0134] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0135] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0136] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for processing pathological sections, characterized in that: include: Perform image acquisition on the pathological sections according to acquisition parameters to obtain a plurality of corresponding digital images; Determine the position of the digitized image according to the name of the digitized image; the name of the digitized image is determined according to the acquisition order; Arranging the digitized images according to their positions to obtain images to be spliced; Splicing the digitized images in the middle row image and the middle column image of the image to be spliced ​​to form a cross-shaped image; the middle row image is the digitized image in the row between the first row and the last row, and the middle column image is the digitized image in the column between the first column and the last column; Dividing the image to be stitched according to the boundaries of the cross-shaped image to obtain four stitching areas; Determining repeated regions in adjacent digitized images in the stitched region; Acquire feature points of the repeated area and feature descriptors corresponding to the feature points; Matching feature points of adjacent digitized images according to feature descriptors corresponding to the feature points to determine matching points; Based on the matching points, adjacent digitized images are spliced ​​to obtain a pathological slice image corresponding to the pathological slice; Using a preset neural network to classify the pathological slice images to obtain a classification result; the neural network is pre-trained based on the pathological slice images of the annotated pathological tissues; Wherein, the acquiring the feature points of the repeated area and the feature descriptors corresponding to the feature points comprises: A preset feature extraction algorithm is used to extract feature points of repeated areas in adjacent digitized images in the spliced ​​area and feature descriptors corresponding to the feature points.

2. The method according to claim 1, characterized in that The step of matching the feature points of adjacent digitized images according to the feature descriptors corresponding to the feature points to determine matching points includes: Calculating the distance between the feature descriptors in the repeated regions of two adjacent digitized images; Comparing the calculated multiple distances with a preset distance threshold; When the calculated distance is greater than the distance threshold, the feature descriptor pair and the corresponding feature point pair corresponding to the distance are eliminated; When the calculated distance is less than or equal to the distance threshold, retaining the feature descriptor pair and the corresponding feature point pair corresponding to the distance; The matching points are obtained according to the retained feature point pairs.

3. The method according to claim 1 or 2, characterized in that: The step of splicing adjacent digitized images based on the matching points to obtain a pathological slice image corresponding to the pathological slice includes: Obtaining the coordinates of each of the feature points; Calculating the relative displacement of each matching point using each coordinate; The relative displacement corresponding to the repeated area of ​​two adjacent digitized images is calculated by using the relative displacement of the matching points and the weights corresponding to the matching points; According to the relative displacement corresponding to the repeated area, two adjacent digitized images are spliced, and the spliced ​​image is the pathological slice image corresponding to the pathological slice.

4. The method according to claim 1, characterized in that: The acquisition parameters include at least one of an acquisition mode, an acquisition size, and a moving step length; The step of acquiring images of pathological sections according to acquisition parameters to obtain corresponding digital images includes: The pathological slices are traversed and collected according to at least one of the collection mode, collection size and moving step length to obtain corresponding digital images.

5. The method according to claim 4, characterized in that The collection method includes any one or more of an S-type method and a Z-type method.

6. The method according to claim 1, characterized in that After obtaining the digitized images and before splicing the digitized images, the method further includes: Respectively acquiring brightness channel images in each of the digitized images; Performing mean calculation on the image matrices corresponding to the brightness channel images to obtain a mean image; Normalizing the mean image to generate a mask image; The image matrix corresponding to the brightness channel image of each of the digitized images is divided by the image matrix corresponding to the mask image, and the image corresponding to the obtained image matrix is ​​the brightness-corrected digitized image.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images