Medical image multi-dimensional mixed end-to-end training system, method, equipment and medium

Abstract

The invention discloses a medical image multi-dimensional mixed end-to-end training system, method and device and a medium. According to the technical scheme provided by the invention, the system comprises a data acquisition module, an extraction module, a fusion module and a training module, the data acquisition module is used for acquiring a pathological image pyramid and a first label; the extraction module is used for extracting different layers of pathological images and second labels from the pathological image pyramid and cutting the pathological images and the second labels to obtain target image blocks; the fusion module is used for storing the target image block and the second label into a buffer pool; after a preset number of target image blocks are stored, fusing the target image blocks to generate a training data set and outputting the training data set; the training module is used for acquiring a training data set to perform model training; according to the method, the images and the labels can be extracted from different layers of the pathological image pyramid, the training set is generated for model training, information of each layer in the pathological image pyramid is synthesized, waste of storage resources is reduced, and the processing efficiency is greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent medical care, and in particular to a medical image multi-dimensional hybrid end-to-end training system, method, computing device and computer storage medium. Background Art

[0002] At present, since the pathological images of patients contain rich information about tissue structure and cell morphology, pathological images have become an important basis for doctors to make pathological diagnoses during the diagnosis and treatment process. However, due to the huge amount of information and complex content in pathological images, only long-term trained and experienced pathologists can accurately draw diagnostic conclusions based on the observation of pathological images. This has led to an extreme shortage of pathologists in my country. The use of deep learning models to process pathological images can assist pathologists in completing pathological diagnoses. Therefore, this technology has been increasingly used in the process of pathological diagnosis.

[0003] At present, in the process of establishing a pathological image diagnosis model, a set of pathological images is obtained as a training set. However, pathological images are composed of the same image with different resolutions in a pyramid-like configuration. This configuration makes it impossible to directly input pathological images into the pathological image diagnosis model for training. Instead, it is necessary to extract the pathological images of the corresponding resolution in a certain layer of the pyramid and store the intermediate data for training. If only the image corresponding to a certain layer is extracted, the information contained in other layers of the pathological image will be lost, greatly reducing the generalization ability of the model; if the images in all layers are to be processed, a large number of intermediate files will be generated, which will reduce the processing speed and efficiency and cause a large amount of storage resources to be wasted. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a medical image multi-dimensional hybrid end-to-end training system and corresponding computing devices and computer storage media.

[0005] According to one aspect of the present invention, a medical image multi-dimensional hybrid end-to-end training system is provided, comprising: a data acquisition module, an extraction module, a fusion module and a training module; wherein:

[0006] The data acquisition module is used to acquire a pathological image pyramid and a first label corresponding to the pathological image pyramid; wherein the first label includes a second label corresponding to each layer of the pathological image pyramid;

[0007] The extraction module is used to extract pathological images of different layers and corresponding second labels from the pathological image pyramid; based on the extracted multiple pathological images and second labels, cut and obtain the target image block;

[0008] The fusion module is used to store the obtained target image blocks and the corresponding second labels into a buffer pool; when a preset number of target image blocks are stored in the buffer pool, the target image blocks in the buffer pool are fused to generate a training data set and the training data set is output;

[0009] The training module is used to obtain a training data set and input it into a pathological image processing model for training.

[0010] The above scheme further includes a preprocessing module for preprocessing the pathological image pyramid before the extraction module extracts the pathological images of different layers and the corresponding second labels from the pathological image pyramid;

[0011] The preprocessing module comprises:

[0012] An acquisition submodule is used to acquire any layer of an image in the pathological image pyramid image as a first image to be processed, and respectively acquire the values ​​of the R channel, the G channel, and the B channel of all pixel points in the first image to be processed in the RGB color space;

[0013] Generate a first sub-image according to the values ​​of all pixels in the R channel;

[0014] Generate a second sub-image according to the values ​​of all pixels in the G channel;

[0015] Generate a third sub-image according to the values ​​of all pixels in the B channel;

[0016] Inputting the first image to be processed, the first sub-image, the second sub-image and the third sub-image into the feature extraction model respectively to obtain a total feature vector, a first feature vector, a second feature vector and a third feature vector;

[0017] Calculate the similarities between the first eigenvector, the second eigenvector, the third eigenvector and the total eigenvector respectively to obtain a first similarity, a second similarity and a third similarity;

[0018] Comparing the first similarity, the second similarity and the third similarity, and determining that the sub-image corresponding to the maximum similarity is the second image to be processed;

[0019] Computing submodule, used for:

[0020] Randomly select a pixel point in the second image to be processed, take the pixel point as the center and a preset distance as the radius to obtain the first area; calculate the weighted average of the corresponding channel values ​​of each pixel point in the first area and replace the channel value of the pixel point; based on the above method, traverse each pixel point in the second image to be processed to obtain a blurred image;

[0021] Calculate the difference between the corresponding pixel points in the second image to be processed and the blurred image to obtain a first difference; compare the absolute value of the first difference with a preset threshold, if the absolute value of the first difference is greater than or equal to the preset threshold, set the pixel value of the pixel point to the first pixel value, if the absolute value of the first difference is less than the preset threshold, set the pixel value of the pixel point to the second pixel value; obtain a difference image based on the pixel point with the first pixel value and the pixel point with the second pixel value; the first pixel value is greater than the second pixel value, and the second pixel value is greater than 0;

[0022] determining a distribution map of differences in the difference image;

[0023] According to the distribution map, a preset distribution map-sharpening factor is queried to determine a target sharpening factor;

[0024] The first image to be processed is sharpened according to the target sharpening coefficient to obtain an enhanced image of the first image to be processed.

[0025] In the above scheme, the pathological image pyramid acquired by the data acquisition module is divided into multiple layers, wherein:

[0026] Each layer of the pathological image pyramid is a pathological image at a different resolution; and the resolution of the pathological image in each layer gradually increases from top to bottom.

[0027] In the above solution, the extraction module is further used to:

[0028] For each layer of the pathological image pyramid, set the corresponding extraction possibility;

[0029] According to the extraction possibility of each layer, pathological images of different layers and corresponding second labels are extracted from the pathological image pyramid;

[0030] For the extracted pathological image and the corresponding second label, a random point in the pathological image is selected as the cropping center, and the target image block is obtained by cropping according to a preset size.

[0031] In the above solution, the extraction module extracts the pathological image and the second label, and crops the target image block in a multi-threaded parallel manner.

[0032] In the above scheme, the training data set includes at least one target image block and a corresponding second label, so that the pathological image processing model can simultaneously obtain image information and label information;

[0033] The fusion module is further used for:

[0034] After outputting the training dataset, clear the buffer pool.

[0035] In the above solution, the fusion module further includes a data processing submodule, and the data processing submodule is used to:

[0036] Obtain the data format of the training data set required by the pathological image processing model;

[0037] Based on the target image block, a corresponding number of first threads are created, and corresponding first thread identifiers are saved in a thread list, and the thread list is used as a thread pool; for each first thread, a processing result list is created;

[0038] Each target image block is assigned to an idle thread in the thread pool, and a serial number is assigned to the corresponding idle thread according to the order in which the target image blocks are obtained; each target image block is processed according to the data format through each idle thread, and the processing results are processed according to the serial number to complete the creation of the training data set.

[0039] In the above solution, the data acquisition module further includes a tag management submodule; wherein the tag management submodule is used to:

[0040] Acquire a dimension indicator relationship table of the pathological image, wherein the dimension indicator relationship is configured in the dimension indicator relationship table;

[0041] Matching the labels in the preset label dictionary with the dimension indicator relationships in the dimension indicator relationship table;

[0042] An index table is established based on the matching label and dimension indicator relationship. The index table is used to find the corresponding label based on the matching dimension indicator relationship.

[0043] Keywords are extracted from the indicator names in the dimension indicator relationship table to form a keyword thesaurus; the keyword thesaurus is used to search for corresponding tags based on keywords.

[0044] According to another aspect of the present invention, a multi-dimensional hybrid end-to-end training method for medical images is provided, comprising:

[0045] Acquire a pathological image pyramid and a first label corresponding to the pathological image pyramid; wherein the first label includes a second label corresponding to each layer of the pathological image pyramid;

[0046] Extracting pathological images of different layers and corresponding second labels from the pathological image pyramid; based on the extracted pathological images and second labels, cutting out target image blocks;

[0047] The obtained target image blocks and the corresponding second labels are stored in a buffer pool; when a preset number of target image blocks are stored in the buffer pool, the target image blocks in the buffer pool are merged to generate a training data set and the training data set is output;

[0048] Obtain training data sets and input them into the pathology image processing model for training.

[0049] According to another aspect of the present invention, there is provided a computing device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0050] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations implemented by the above-mentioned medical image multi-dimensional hybrid end-to-end training system.

[0051] According to yet another aspect of the present invention, a computer storage medium is provided, wherein the storage medium stores at least one executable instruction, wherein the executable instruction enables a processor to execute operations implemented by the above-mentioned multi-dimensional hybrid end-to-end training system for medical images.

[0052] According to the technical solution provided by the present invention, a medical image multi-dimensional hybrid end-to-end training system includes: a data acquisition module, an extraction module, a fusion module and a training module; wherein the data acquisition module is used to acquire a pathological image pyramid and a first label corresponding to the pathological image pyramid; wherein the first label includes a second label corresponding to each layer of the pathological image pyramid; the extraction module is used to extract pathological images of different layers and corresponding second labels from the pathological image pyramid; based on the extracted multiple pathological images and the second labels, a target image block is obtained by cutting; the fusion module is used to store the obtained target image block and the corresponding second label into a buffer pool; when a preset number of target image blocks are stored in the buffer pool, the target image blocks in the buffer pool are fused to generate a training data set and the training data set is output; the training module is used to acquire the training data set and input it into a pathological image processing model for training. By extracting pathological images and labels of corresponding resolutions from different layers of the pathological image pyramid according to different extraction possibilities, the images and labels are stored in the buffer pool after cropping, and multiple cropped target image blocks are fused to generate a training data set. By acquiring images and labels in different layers, the generated training data set can fully obtain the information contained in each layer of the pathological image, avoiding information loss and greatly improving the generalization ability of the trained pathological image processing model; at the same time, by directly generating a training data set for output after fusing the acquired image blocks, the generation of a large number of intermediate files is avoided, greatly reducing the waste of storage resources. In addition, by using multi-threaded processing during extraction and creation of training data sets, the processing speed and efficiency are improved. By corresponding the dimensional indicators of pathological images to labels, an index table and a tube detection vocabulary are established, which more concisely reflects the meaning of the labels, making it easier for users to understand and search based on labels.

[0053] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0055] Figure 1 A structural block diagram of a medical image multi-dimensional hybrid end-to-end training system according to an embodiment of the present invention is shown;

[0056] Figure 2 A schematic diagram of a pathological image pyramid according to an embodiment of the present invention is shown;

[0057] Figure 3 A structural block diagram of a medical image multi-dimensional hybrid end-to-end training system according to another embodiment of the present invention is shown;

[0058] Figure 4 A schematic diagram of a process of a multi-dimensional hybrid end-to-end training method for medical images according to an embodiment of the present invention is shown;

[0059] Figure 5 A schematic diagram of the structure of a computing device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0060] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0061] Figure 1 The structure block diagram of a medical image multi-dimensional hybrid end-to-end training system according to an embodiment of the present invention is shown, the system comprises: a data acquisition module 101, an extraction module 102, a fusion module 103 and a training module 104; wherein,

[0062] The data acquisition module 101 is used to acquire a pathological image pyramid and a first label corresponding to the pathological image pyramid; wherein the first label includes a second label corresponding to each layer of the pathological image pyramid;

[0063] The extraction module is used to extract pathological images of different layers and corresponding second labels from the pathological image pyramid; based on the extracted multiple pathological images and second labels, cut and obtain the target image block;

[0064] The fusion module is used to store the obtained target image blocks and the corresponding second labels into a buffer pool; when a preset number of target image blocks are stored in the buffer pool, the target image blocks in the buffer pool are fused to generate a training data set and the training data set is output;

[0065] The training module is used to obtain a training data set and input it into a pathological image processing model for training.

[0066] Preferably, it further comprises a preprocessing module for preprocessing the pathological image pyramid before the extraction module extracts the pathological images of different layers and the corresponding second labels from the pathological image pyramid;

[0067] The preprocessing module comprises:

[0068] An acquisition submodule is used to acquire any layer of an image in the pathological image pyramid image as a first image to be processed, and respectively acquire the values ​​of the R channel, the G channel, and the B channel of all pixel points in the first image to be processed in the RGB color space;

[0069] Generate a first sub-image according to the values ​​of all pixels in the R channel;

[0070] Generate a second sub-image according to the values ​​of all pixels in the G channel;

[0071] Generate a third sub-image according to the values ​​of all pixels in the B channel;

[0072] Inputting the first image to be processed, the first sub-image, the second sub-image and the third sub-image into the feature extraction model respectively to obtain a total feature vector, a first feature vector, a second feature vector and a third feature vector;

[0073] Calculate the similarities between the first eigenvector, the second eigenvector, the third eigenvector and the total eigenvector respectively to obtain a first similarity, a second similarity and a third similarity;

[0074] Comparing the first similarity, the second similarity and the third similarity, and determining that the sub-image corresponding to the maximum similarity is the second image to be processed;

[0075] Computing submodule, used for:

[0076] Randomly select a pixel point in the second image to be processed, take the pixel point as the center and a preset distance as the radius to obtain the first area; calculate the weighted average of the corresponding channel values ​​of each pixel point in the first area and replace the channel value of the pixel point; based on the above method, traverse each pixel point in the second image to be processed to obtain a blurred image;

[0077] Calculate the difference between the corresponding pixel points in the second image to be processed and the blurred image to obtain a first difference; compare the absolute value of the first difference with a preset threshold, if the absolute value of the first difference is greater than or equal to the preset threshold, set the pixel value of the pixel point to the first pixel value, if the absolute value of the first difference is less than the preset threshold, set the pixel value of the pixel point to the second pixel value; obtain a difference image based on the pixel point with the first pixel value and the pixel point with the second pixel value; the first pixel value is greater than the second pixel value, and the second pixel value is greater than 0;

[0078] determining a distribution map of differences in the difference image;

[0079] According to the distribution map, a preset distribution map-sharpening factor is queried to determine a target sharpening factor;

[0080] The first image to be processed is sharpened according to the target sharpening coefficient to obtain an enhanced image of the first image to be processed.

[0081] In this embodiment, the values ​​of the R channel, G channel, and B channel of the pixels in the image in the RGB color space are different, and the amount of information contained is also different, so the values ​​of the R channel, G channel, and B channel of all pixels in the first image to be processed in the RGB color space are obtained respectively.

[0082] In this embodiment, the first image to be processed, the first sub-image, the second sub-image and the third sub-image are respectively input into the feature extraction model. The specific implementation of the feature extraction model includes but is not limited to a convolutional neural network, which is not specifically limited here.

[0083] In this embodiment, the similarity is calculated in the following way:

[0084]

[0085] Among them, x i is the i-th component of the first eigenvector, y i is the i-th component of the total eigenvector, n is the number of components of the first eigenvector, and is also the number of components of the total eigenvector, and both numbers are equal; S is the similarity value between the first sub-image and the first image to be processed.

[0086] In this embodiment, the sub-image corresponding to the maximum similarity is determined to be the second image to be processed, which indicates that the sub-image corresponding to the maximum similarity contains the largest amount of information of the first image to be processed, and the sub-image corresponding to the maximum similarity is selected to contain the most features in the first image to be processed, so as to reduce the amount of data calculation and improve the calculation rate.

[0087] In this embodiment, the specific implementation of blurring the image is Gaussian blurring.

[0088] In this embodiment, the specific implementation method of the difference distribution map is to use the pixel values ​​in the difference image as a data set, count the frequency of these pixel values, that is, the number of times each pixel value appears, arrange these pixel values ​​according to their sizes, and calculate the differences between them, use these differences as the vertical coordinates and the corresponding pixel values ​​as the horizontal coordinates, and draw a bar chart or a line chart, etc., which is a difference distribution map.

[0089] The beneficial effects of the above technical solution are as follows: the pathological image in the pathological image pyramid is relatively large, and directly preprocessing the pathological image will increase the amount of computer calculation, which is not conducive to batch processing of pathological images; this solution adopts the distribution of R channel, G channel, and B channel of all pixel points in the first image to be processed in the RGB color space, and obtains the sub-image of the corresponding channel based on the distribution; the sub-image with the highest similarity to the first image to be processed is screened out to obtain the second image to be processed, and the second image to be processed is subjected to Gaussian blur processing; the difference between the corresponding pixel points in the second image to be processed and the blurred image is calculated, and compared with the preset threshold value to obtain the difference; value image; determine the difference distribution map from the difference image; query the preset distribution map-sharpening coefficient based on the distribution map to determine the target sharpening coefficient; sharpen the first image to be processed according to the target sharpening coefficient to obtain an enhanced image of the first image to be processed; adopt this scheme to sharpen the first image to be processed, which greatly reduces the amount of computer calculation and improves the image processing speed of the first image to be processed. The enhanced image of the first image to be processed is obtained by sharpening, and the image quality is improved. Based on the preprocessed pathological image pyramid, it is convenient for the extraction module to more accurately extract pathological images of different layers and determine the corresponding second label.

[0090] Specifically, each layer of the pathological image pyramid is a pathological image at a different resolution; and the resolution of the pathological image in each layer gradually increases from top to bottom.

[0091] like Figure 2 As shown, Figure 2 FIG. 4 shows a schematic diagram of a pathological image pyramid according to an embodiment of the present invention; wherein:

[0092] The resolution of the pathological image in the top layer is the smallest, and the resolution of the pathological image in each layer gradually increases from top to bottom, and the resolution of the pathological image in the bottom layer is the largest.

[0093] The extraction module 102 is used to extract pathological images of different layers and corresponding second labels from the pathological image pyramid; based on the extracted multiple pathological images and second labels, cut and obtain a target image block.

[0094] Specifically, for each layer of the pathological image pyramid, the corresponding extraction possibility is set;

[0095] According to the extraction possibility of each layer, pathological images of different layers and corresponding second labels are extracted from the pathological image pyramid;

[0096] For the extracted pathological image and the corresponding second label, a random point in the pathological image is selected as the cropping center, and the target image block is obtained by cropping according to a preset size.

[0097] Preferably, the extraction possibility can be set according to the image resolution priority in the training objective to set the extraction possibilities corresponding to different layers in the pathological image pyramid; for example, if the training is mainly for high-resolution images, the extraction possibility of the high-resolution corresponding layer in the pathological image pyramid is higher, and relatively, the extraction possibility of the low-resolution corresponding layer is lower; if the training is mainly for low-resolution images, the extraction possibility of the low-resolution corresponding layer in the pathological image pyramid is higher, and relatively, the extraction possibility of the high-resolution corresponding layer is lower.

[0098] Preferably, the preset size is a preset image area size in pixels, such as 320×320.

[0099] Specifically, the extraction module 102 extracts the pathological image and the second label, and cuts the target image block in a multi-threaded parallel manner.

[0100] The fusion module 103 is used to store the obtained target image blocks and the corresponding second labels into a buffer pool; when a preset number of target image blocks are stored in the buffer pool, the target image blocks in the buffer pool are fused to generate a training data set and the training data set is output.

[0101] Specifically, the training data set includes at least one target image block and a corresponding second label, so that the pathological image processing model can simultaneously obtain image information and label information;

[0102] The fusion module 103 is further used to: clear the buffer pool after outputting the training data set.

[0103] The training module 104 is used to obtain a training data set and input it into a pathological image processing model for training.

[0104] The medical image multi-dimensional hybrid end-to-end training system provided according to the present embodiment includes: a data acquisition module, an extraction module, a fusion module and a training module; wherein the data acquisition module is used to acquire a pathological image pyramid and a first label corresponding to the pathological image pyramid; wherein the first label includes a second label corresponding to each layer of the pathological image pyramid; the extraction module is used to extract pathological images of different layers and corresponding second labels from the pathological image pyramid; based on the extracted multiple pathological images and second labels, a target image block is obtained by cutting; the fusion module is used to store the obtained target image block and the corresponding second label into a buffer pool; when a preset number of target image blocks are stored in the buffer pool, the target image blocks in the buffer pool are fused to generate a training data set and the training data set is output; the training module is used to acquire the training data set and input it into a pathological image processing model for training. By extracting pathological images and labels of corresponding resolutions from different layers of the pathological image pyramid according to different extraction possibilities, storing them in a buffer pool after cropping, and fusing multiple cropped target image blocks to generate a training data set, and by obtaining images and labels in different layers, the generated training data set can fully obtain the information contained in each layer of the pathological image, avoiding information loss and greatly improving the generalization ability of the trained pathological image processing model; at the same time, by fusing the acquired image blocks and directly generating a training data set for output, the generation of a large number of intermediate files is avoided, greatly reducing the waste of storage resources. In addition, by using multi-threaded processing during extraction and image cropping, the processing speed and efficiency are improved.

[0105] Figure 3 FIG. 4 shows a structural block diagram of a medical image multi-dimensional hybrid end-to-end training system according to another embodiment of the present invention. Figure 3 As shown, it includes: a data acquisition module 301, an extraction module 302, a fusion module 303 and a training module 304; wherein,

[0106] The data acquisition module 301 further includes a tag management submodule 305; wherein the tag management submodule 305 is used to:

[0107] Acquire a dimension indicator relationship table of the pathological image, wherein the dimension indicator relationship is configured in the dimension indicator relationship table;

[0108] Matching the labels in the preset label dictionary with the dimension indicator relationships in the dimension indicator relationship table;

[0109] An index table is established based on the matching label and dimension indicator relationship. The index table is used to find the corresponding label based on the matching dimension indicator relationship.

[0110] Keywords are extracted from the indicator names in the dimension indicator relationship table to form a keyword thesaurus; the keyword thesaurus is used to search for corresponding tags based on keywords.

[0111] The fusion module 303 further includes a data processing submodule 306, and the data processing submodule 306 is used to:

[0112] Obtain the data format of the training data set required by the pathological image processing model;

[0113] Based on the target image block, a corresponding number of first threads are created, and corresponding first thread identifiers are saved in a thread list, and the thread list is used as a thread pool; for each first thread, a processing result list is created;

[0114] Each target image block is assigned to an idle thread in the thread pool, and a serial number is assigned to the corresponding idle thread according to the order in which the target image blocks are obtained; each target image block is processed according to the data format through each idle thread, and the processing results are processed according to the serial number to complete the creation of the training data set.

[0115] In this embodiment, in the medical image multi-dimensional hybrid end-to-end training system, the functions that can be implemented by the data acquisition module 301, the extraction module 302, the fusion module 303 and the training module 304 are as described above and will not be repeated here.

[0116] According to the above system, the processing speed and efficiency can be improved by using multi-threaded processing when extracting and cutting pathological images and creating training data sets. By matching the dimensional indicators of pathological images with labels, an index table and a tube detection vocabulary are established to more concisely reflect the meaning of the labels, which is convenient for users to understand and search based on labels.

[0117] Figure 4 FIG. 4 is a flow chart of a multi-dimensional hybrid end-to-end training method for medical images according to an embodiment of the present invention; FIG. Figure 4 As shown, the method comprises the following steps:

[0118] Step S401 : obtaining a pathological image pyramid and a first label corresponding to the pathological image pyramid; wherein the first label includes a second label corresponding to each layer of the pathological image pyramid.

[0119] Specifically, the pathological image pyramid is divided into multiple layers, wherein each layer of the pathological image pyramid is a pathological image at a different resolution; and the resolution of the pathological image in each layer gradually increases from top to bottom.

[0120] Preferably, a dimension indicator relationship table of the pathological image is obtained, wherein the dimension indicator relationship is configured in the dimension indicator relationship table;

[0121] Matching the labels in the preset label dictionary with the dimension indicator relationships in the dimension indicator relationship table;

[0122] An index table is established based on the matching label and dimension indicator relationship. The index table is used to find the corresponding label based on the matching dimension indicator relationship.

[0123] Keywords are extracted from the indicator names in the dimension indicator relationship table to form a keyword thesaurus; the keyword thesaurus is used to search for corresponding tags based on keywords.

[0124] Step S402, extracting pathological images of different layers and corresponding second labels from the pathological image pyramid; and cutting out target image blocks based on the extracted pathological images and second labels.

[0125] Specifically, for each layer of the pathological image pyramid, the corresponding extraction possibility is set;

[0126] According to the extraction possibility of each layer, pathological images of different layers and corresponding second labels are extracted from the pathological image pyramid;

[0127] For the extracted pathological image and the corresponding second label, a random point in the pathological image is selected as the cropping center, and the target image block is obtained by cropping according to a preset size.

[0128] Specifically, the pathological image and the second label are extracted, and the target image block is cropped in a multi-threaded parallel manner.

[0129] Step S403, storing the obtained target image blocks and the corresponding second labels in a buffer pool; when a preset number of target image blocks are stored in the buffer pool, the target image blocks in the buffer pool are merged to generate a training data set and the training data set is output.

[0130] Specifically, the training data set includes at least one target image block and a corresponding second label, so that the pathological image processing model can simultaneously obtain image information and label information.

[0131] Specifically, after outputting the training data set, the buffer pool is cleared.

[0132] Preferably, obtaining the data format of the training data set required by the pathological image processing model;

[0133] Based on the target image block, a corresponding number of first threads are created, and corresponding first thread identifiers are saved in a thread list, and the thread list is used as a thread pool; for each first thread, a processing result list is created;

[0134] Each target image block is assigned to an idle thread in the thread pool, and a serial number is assigned to the corresponding idle thread according to the order in which the target image blocks are obtained; each target image block is processed according to the data format through each idle thread, and the processing results are processed according to the serial number to complete the creation of the training data set.

[0135] Step S404, obtaining a training data set and inputting it into a pathological image processing model for training.

[0136] According to the above-mentioned multi-dimensional hybrid end-to-end training method for medical images, a pathological image pyramid and a first label corresponding to the pathological image pyramid can be obtained; wherein the first label includes a second label corresponding to each layer of the pathological image pyramid; pathological images of different layers and the corresponding second labels are extracted from the pathological image pyramid; based on the extracted pathological images and the second labels, target image blocks are cut and obtained; the obtained target image blocks and the corresponding second labels are stored in a buffer pool; when a preset number of target image blocks are stored in the buffer pool, the target image blocks in the buffer pool are fused to generate a training data set and the training data set is output; the training data set is obtained and input into a pathological image processing model for training. By extracting pathological images and labels of corresponding resolutions from different layers of the pathological image pyramid according to different extraction possibilities, the images and labels are stored in the buffer pool after cropping, and multiple cropped target image blocks are fused to generate a training data set. By acquiring images and labels in different layers, the generated training data set can fully obtain the information contained in each layer of the pathological image, avoiding information loss and greatly improving the generalization ability of the trained pathological image processing model; at the same time, by directly generating a training data set for output after fusing the acquired image blocks, the generation of a large number of intermediate files is avoided, greatly reducing the waste of storage resources. In addition, by using multi-threaded processing during extraction and creation of training data sets, the processing speed and efficiency are improved. By corresponding the dimensional indicators of pathological images to labels, an index table and a tube detection vocabulary are established, which more concisely reflects the meaning of the labels, making it easier for users to understand and search based on labels.

[0137] The present invention also provides a non-volatile computer storage medium, which stores at least one executable instruction, and the executable instruction can execute the operations implemented by the medical image multi-dimensional hybrid end-to-end training system in any of the above method embodiments.

[0138] Figure 5 A schematic diagram of the structure of a computing device according to an embodiment of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the computing device.

[0139] like Figure 5As shown, the computing device may include: a processor (processor) 502 , a communications interface (Communications Interface) 504 , a memory (memory) 506 , and a communication bus 508 .

[0140] in:

[0141] The processor 502 , the communication interface 504 , and the memory 506 communicate with each other via a communication bus 508 .

[0142] The communication interface 504 is used to communicate with other devices such as clients or other servers.

[0143] The processor 502 is used to execute the program 510, and specifically can execute the relevant steps implemented in the above-mentioned embodiment of the medical image multi-dimensional hybrid end-to-end training system.

[0144] Specifically, the program 510 may include program codes, and the program codes include computer operation instructions.

[0145] The processor 502 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the computing device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0146] The memory 506 is used to store the program 510. The memory 506 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0147] The program 510 can be specifically used to enable the processor 502 to execute the medical image multi-dimensional hybrid end-to-end training system in any of the above-mentioned embodiments. The specific implementation of each step in the program 510 can refer to the corresponding descriptions in the corresponding steps and units implemented in the above-mentioned medical image multi-dimensional hybrid end-to-end training system embodiment, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the aforementioned method embodiment, which will not be repeated here.

[0148] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A multi-dimensional hybrid end-to-end training system for medical images, comprising: Data acquisition module, extraction module, fusion module and training module; among them, The data acquisition module is used to acquire a pathological image pyramid and a first label corresponding to the pathological image pyramid; wherein the first label includes a second label corresponding to each layer of the pathological image pyramid; The extraction module is used to extract pathological images of different layers and corresponding second labels from the pathological image pyramid; based on the extracted multiple pathological images and second labels, cut and obtain the target image block; The fusion module is used to store the obtained target image blocks and the corresponding second labels into a buffer pool; when a preset number of target image blocks are stored in the buffer pool, the target image blocks in the buffer pool are fused to generate a training data set and the training data set is output; The training module is used to obtain a training data set and input it into a pathological image processing model for training.

2. The system according to claim 1, characterized in that It also includes a preprocessing module, which is used to preprocess the pathological image pyramid before the extraction module extracts the pathological images of different layers and the corresponding second labels from the pathological image pyramid; The preprocessing module comprises: An acquisition submodule is used to acquire any layer of an image in the pathological image pyramid image as a first image to be processed, and respectively acquire the values ​​of the R channel, the G channel, and the B channel of all pixel points in the first image to be processed in the RGB color space; Generate a first sub-image according to the values ​​of all pixels in the R channel; Generate a second sub-image according to the values ​​of all pixels in the G channel; Generate a third sub-image according to the values ​​of all pixels in the B channel; Inputting the first image to be processed, the first sub-image, the second sub-image and the third sub-image into the feature extraction model respectively to obtain a total feature vector, a first feature vector, a second feature vector and a third feature vector; Calculate the similarities between the first eigenvector, the second eigenvector, the third eigenvector and the total eigenvector respectively to obtain a first similarity, a second similarity and a third similarity; Comparing the first similarity, the second similarity and the third similarity, and determining that the sub-image corresponding to the maximum similarity is the second image to be processed; Computing submodule, used for: Randomly select a pixel point in the second image to be processed, take the pixel point as the center and a preset distance as the radius to obtain the first area; calculate the weighted average of the corresponding channel values ​​of each pixel point in the first area and replace the channel value of the pixel point; based on the above method, traverse each pixel point in the second image to be processed to obtain a blurred image; Calculate the difference between the corresponding pixel points in the second image to be processed and the blurred image to obtain a first difference; compare the absolute value of the first difference with a preset threshold, if the absolute value of the first difference is greater than or equal to the preset threshold, set the pixel value of the pixel point to the first pixel value, if the absolute value of the first difference is less than the preset threshold, set the pixel value of the pixel point to the second pixel value; obtain a difference image based on the pixel point with the first pixel value and the pixel point with the second pixel value; the first pixel value is greater than the second pixel value, and the second pixel value is greater than 0; determining a distribution map of differences in the difference image; According to the distribution map, a preset distribution map-sharpening factor is queried to determine a target sharpening factor; The first image to be processed is sharpened according to the target sharpening coefficient to obtain an enhanced image of the first image to be processed.

3. The system according to claim 1, characterized in that The extraction module is further used for: For each layer of the pathological image pyramid, set the corresponding extraction possibility; According to the extraction possibility of each layer, pathological images of different layers and corresponding second labels are extracted from the pathological image pyramid; For the extracted pathological image and the corresponding second label, a random point in the pathological image is selected as the cropping center, and the target image block is obtained by cropping according to a preset size.

4. The system according to claim 1 or 3, characterized in that: The extraction module extracts the pathological image and the second label and cuts the target image block in a multi-threaded parallel manner.

5. The system according to claim 1, characterized in that The training data set includes at least one target image block and a corresponding second label, so that the pathological image processing model can simultaneously obtain image information and label information; The fusion module is further used for: After outputting the training dataset, clear the buffer pool.

6. The system according to claim 1, characterized in that The fusion module further includes a data processing submodule, and the data processing submodule is used to: Obtain the data format of the training data set required by the pathological image processing model; Based on the target image block, a corresponding number of first threads are created, and corresponding first thread identifiers are saved in a thread list, and the thread list is used as a thread pool; for each first thread, a processing result list is created; Each target image block is assigned to an idle thread in the thread pool, and a serial number is assigned to the corresponding idle thread according to the order in which the target image blocks are obtained; each target image block is processed according to the data format through each idle thread, and the processing results are processed according to the serial number to complete the creation of the training data set.

7. The system according to claim 1, characterized in that The data acquisition module further includes a tag management submodule; wherein the tag management submodule is used to: Acquire a dimension indicator relationship table of the pathological image, wherein the dimension indicator relationship is configured in the dimension indicator relationship table; Matching the labels in the preset label dictionary with the dimension indicator relationships in the dimension indicator relationship table; An index table is established based on the matching label and dimension indicator relationship. The index table is used to find the corresponding label based on the matching dimension indicator relationship. Keywords are extracted from the indicator names in the dimension indicator relationship table to form a keyword thesaurus; the keyword thesaurus is used to search for corresponding tags based on keywords.

8. A multi-dimensional hybrid end-to-end training method for medical images, comprising: Acquire a pathological image pyramid and a first label corresponding to the pathological image pyramid; wherein the first label includes a second label corresponding to each layer of the pathological image pyramid; Extracting pathological images of different layers and corresponding second labels from the pathological image pyramid; based on the extracted pathological images and second labels, cutting out target image blocks; The obtained target image blocks and the corresponding second labels are stored in a buffer pool; when a preset number of target image blocks are stored in the buffer pool, the target image blocks in the buffer pool are merged to generate a training data set and the training data set is output; Obtain training data sets and input them into the pathology image processing model for training.

9. A computing device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the function performed by the medical image multi-dimensional hybrid end-to-end training system as described in any one of claims 1-7.

10. A computer storage medium, wherein at least one executable instruction is stored in the storage medium, and the executable instruction enables a processor to execute the functions performed by the medical image multi-dimensional hybrid end-to-end training system as described in any one of claims 1-7.

Images