Method and circuit for rapidly screening pathological section images based on FPGA (Field Programmable Gate Array)

Abstract

The invention discloses an FPGA-based pathological section image rapid screening method and circuit, and the method comprises the steps: obtaining to-be-screened pathological section image data from an external device through an FPGA, carrying out the unpacking of the image data through the FPGA, carrying out the processing of the data, obtaining the definition of an image and the difference degree between the images, transmitting the definition and the difference degree to the external device, and carrying out the rapid screening of the pathological section image through the external device. And the external equipment screens the images by taking the two indexes as judgment standards. According to the method, rapid screening of the pathological section images can be achieved, on the premise that operation of doctors is not affected, fuzzy and redundant pathological section images entering the database are reduced, and the data quality of the database is improved to a great extent.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing technology, and relates to a method and circuit for rapid screening of pathological slide images based on FPGA. Background Technology

[0002] Establishing a medical pathology slide image database is an important foundation for the development of medical informatization and precision medicine. The accumulation of large-scale pathology slide images can establish a "gold standard" for disease diagnosis, reduce the subjective differences in human diagnosis, and promote the further application of artificial intelligence in medicine. In addition, it can also serve as educational and training resources, evidence in medical disputes, and so on.

[0003] Establishing a medical pathology slide image database requires photographing pathology slides under a microscope. However, the obtained images cannot be directly entered into the database due to two major problems: blurriness and redundancy. Otherwise, the data quality of the database would be severely reduced. Therefore, it is necessary to screen the photographed images.

[0004] Currently, there are three solutions to the two major problems of image blurring and redundancy: The first is to capture WSI (whole slice image), which involves scanning the slide using specialized equipment. This method does not produce blurry or redundant images, but the drawback is that the size of a single scanned image can be as large as several GB or even TB, placing stringent demands on the storage capacity of the database. In addition, additional scanning equipment needs to be purchased. The second method is for medical workers to manually select suitable images for the database. This method has a low barrier to entry and is currently the most commonly used method. However, it is a tedious and laborious task for medical workers, greatly increasing their workload. The third method is for medical workers to use the CPU to automatically filter images. Compared to the second method, this method is more convenient. However, due to the poor real-time performance of CPU calculations, it requires cooperation between medical workers and the CPU to complete the filtering task, which still increases the workload of medical workers to some extent.

[0005] Based on this, the present invention develops a method and circuit for rapid screening of pathological slide images based on a field programmable gate array (FPGA), which can quickly screen out clear and non-redundant pathological slide images without requiring additional cooperation from medical workers. This improves the medical pathological slide image database without adding extra workload to medical workers. Summary of the Invention

[0006] The purpose of this invention is to address the shortcomings of existing technologies by providing a rapid screening method and circuit for pathological slide images based on FPGA. To address the problems of blurry and redundant images captured during the establishment of a medical pathological slide database, this invention proposes a hardware architecture for parallel calculation of image sharpness and inter-image differences, along with a rapid screening method that requires no manual intervention.

[0007] The technical solution adopted in this invention is as follows:

[0008] A rapid screening method for pathological slide images based on FPGA, comprising:

[0009] The FPGA acquires pathological slide image data to be screened from an external device. After unpacking the image data, the FPGA processes the data to obtain the image clarity and the difference between images, and transmits it to the external device. The external device then uses these two indicators as judgment criteria to screen the images.

[0010] In the above technical solution, further, after the FPGA unpacks the image data, it generates a pixel data stream of one pixel per system clock cycle; and calculates the grayscale value based on the pixels in the pixel data stream to generate a grayscale value data stream.

[0011] Furthermore, grayscale values ​​are extracted from the grayscale data stream at fixed intervals and entered into a buffer. Then, the difference between the grayscale values ​​in the buffer is calculated. If the difference is greater than a preset threshold, it is considered that a texture has been detected. The number of times a texture is detected during the processing of an image is counted, and the statistical result is used as the image sharpness.

[0012] Furthermore, a fixed-length grayscale value sequence is formed by sampling from the grayscale data stream at specific intervals. The grayscale value sequence is binarized to form an image feature sequence and cached. During the generation of a new image feature sequence, the old image feature sequence is retrieved from the cache, compared with the new image feature sequence, and the Hamming distance between the new and old image feature sequences is calculated. The image feature sequence in the cache is then updated. The calculated Hamming distance is used as the difference between the images.

[0013] Furthermore, the binarization specifically involves averaging the gray values ​​of the gray value sequence, setting values ​​greater than the average value to 1, and values ​​equal to or lower than the average value to 0.

[0014] A rapid screening circuit for pathological slide images based on FPGA, implemented on FPGA, includes: a high-speed data transmission module, a data unpacking module, an image grayscale value calculation module, an image sharpness evaluation module, and an image difference evaluation module;

[0015] The high-speed data transmission module connects to an external device via a PCIE interface to acquire pathological slide image data packets from the external device and transmit them to the data unpacking module. After the image preprocessing calculation is completed, the results are presented to the external device via the PCIE interface.

[0016] The data unpacking module unpacks the data packet according to the information contained in the header file of the data packet, thereby generating a pixel data stream of one pixel data per system clock cycle;

[0017] The image grayscale value calculation module calculates the grayscale value corresponding to each pixel data according to the pixel data stream generated by the data unpacking module, thereby generating a grayscale value data stream with one grayscale value per system clock cycle;

[0018] The image sharpness evaluation module calculates the difference between the gray values ​​of two pixels at fixed intervals based on the gray value data stream generated by the image gray value calculation module. If the difference is greater than a threshold, a texture is considered to have appeared. The number of textures in the image is counted to obtain the image sharpness, which is then transmitted to the high-speed data transmission module. The image difference evaluation module converts an image into a series of feature sequences based on the gray value data stream generated by the image gray value calculation module and caches them. After obtaining the feature sequence of a new image, the Hamming distance between the two feature sequences is calculated. The Hamming distance is used as the difference between the two images and transmitted to the high-speed data transmission module.

[0019] Furthermore, the high-speed data transmission module includes a data downlink FIFO and a data uplink register. The data downlink FIFO is used to organize the data received by the PCIe interface and output it to the data unpacking module. The data uplink register is used to collect the data processing results of the entire FPGA platform and upload them to external devices via the PCIe protocol.

[0020] Furthermore, the data unpacking module includes a header file parsing module, a bit-width conversion module, and a data error detection module. The header file parsing module receives the data stream output by the high-speed data transmission module. The first cycle of data is the file header, which contains file category and file length information. The file category needs to be matched with a preset file category in the header file. If the match is successful, subsequent data enters the bit-width conversion module; if the match is unsuccessful, the erroneous pathological slide image file is discarded. The bit-width conversion module is used to convert the obtained data into a pixel data stream of one pixel per system clock cycle, which is then output to the subsequent processing module.

[0021] Beneficial effects

[0022] The method and circuit of this invention can achieve rapid screening of pathological slide images, reducing the number of blurry and redundant pathological slide images entering the database without affecting the doctor's operation, thus greatly improving the data quality of the database. This invention implements the main functions through an FPGA platform and interacts with external devices through the PCIE protocol, which has excellent device compatibility and can be widely used on hospital computer equipment. Attached Figure Description

[0023] Figure 1 This is a schematic diagram showing the overall circuit structure of the present invention and its connection to external devices via the PCIE protocol.

[0024] Figure 2 This is a schematic diagram of the data unpacking module of the present invention;

[0025] Figure 3 This is a schematic diagram of the image sharpness evaluation module of the present invention;

[0026] Figure 4 This is a schematic diagram of the image difference module of the present invention. Detailed Implementation

[0027] The present invention will be further described below with reference to the embodiments and accompanying drawings. The present invention includes, but is not limited to, the following embodiments.

[0028] The proposed rapid screening method for pathological slide images utilizes an FPGA platform for its main functions and interacts with external devices via the PCIe protocol. The judgment criteria consist of image sharpness and the degree of difference between images. The image sharpness evaluation algorithm is as follows:

[0029] 1) Obtain image data from external devices, unpack the data, and generate a pixel data stream of one pixel per system clock cycle;

[0030] 2) Calculate the grayscale value based on the pixels in the pixel data stream and generate a grayscale value data stream;

[0031] 3) Extract grayscale values ​​from the grayscale data stream at specific intervals and buffer them. Then calculate the difference between the grayscale values ​​in the buffer. If the difference is greater than a preset threshold, a texture is considered detected. Taking an image with a resolution of 1216*1824 as an example, the specific interval is usually determined based on the average number of pixels occupied by a single cell in the image, and is usually set to 50 pixels. The preset threshold is usually set based on the difference between the grayscale value of the cell tissue and the grayscale value of the background, and is usually set to 30.

[0032] 4) Count the number of times textures are detected during the processing of an image, and present the statistical results as the image sharpness to the external device.

[0033] The process of the image difference evaluation algorithm is as follows:

[0034] 1) Obtain image data from external devices, unpack the data, and generate a pixel data stream of one pixel per system clock cycle;

[0035] 2) Calculate the grayscale value based on the pixels in the pixel data stream and generate a grayscale value data stream;

[0036] 3) Sample grayscale values ​​from the grayscale data stream at specific intervals to form a fixed-length grayscale value sequence. Binarize this sequence using a specific method to form an image feature sequence and cache it. Taking an image with dimensions 1216*1824 as an example, the specific interval is typically sampling grayscale values ​​every 32 rows, for a total of 38 rows. Each row requires a 32-pixel interval for grayscale value sampling, resulting in 56 grayscale values ​​per row, thus forming a grayscale value sequence of length 2128. The longer the grayscale value sequence, the more sensitive the image difference evaluation algorithm is to the differences between images; typically, a length of 2128 is sufficiently sensitive. The specific method typically involves averaging the grayscale values ​​in the sequence, setting values ​​greater than the average to 1, and values ​​equal to or below the average to 0.

[0037] 4) During the process of generating a new image feature sequence, the old image feature sequence is retrieved from the cache, compared with the new image feature sequence, the Hamming distance between the new and old feature sequences is calculated, and the image feature sequence in the cache is updated.

[0038] 5) Present the statistical Hamming distance as the difference between images to the external device.

[0039] An external device comprehensively considers two indicators: difference and sharpness, to filter images. This external device is typically a computer host with a PCIe card slot. The filtering method is as follows: first, image difference is considered; if the difference exceeds 100, a new image is considered captured, and a high-quality image needs to be selected from the subsequent images. Next, the sharpness of subsequent images is observed; if the sharpness exceeds 60,000, it is considered a high-quality image and output. Image output stops after outputting one high-quality image. If the difference exceeds 100 again, the above process is repeated.

[0040] The rapid screening circuit for pathological slide images of the present invention comprises five modules: a high-speed data transmission module, a data unpacking module, an image grayscale value calculation module, an image sharpness evaluation module, and an image difference evaluation module. The modules are described below:

[0041] 1) The high-speed data transmission module acquires pathological slide image data packets from external devices through the PCIE interface, and after the image preprocessing calculation is completed, it presents the results to the external devices through the PCIE interface;

[0042] 2) The data unpacking module processes the image data packets obtained by the high-speed data transmission module and unpacks the data packets according to the file type, file length and other information contained in the header of the data packets, thereby generating a pixel data stream of one pixel data per system clock cycle for use by other analysis modules;

[0043] 3) The image grayscale value calculation module calculates the grayscale value corresponding to each pixel data according to the pixel data stream generated by the data unpacking module, thereby generating a grayscale value data stream with one grayscale value per system clock cycle for use by other analysis modules;

[0044] 4) The image sharpness evaluation module calculates the difference between the gray values ​​of two pixels at fixed intervals based on the gray value data stream generated by the image gray value calculation module. If the difference is greater than the threshold, it is considered that a texture has appeared. The number of textures in the whole image is counted to obtain the sharpness evaluation score of the pathological slide image, and then the score is presented to the external device through the high-speed data transmission module.

[0045] 5) The image difference evaluation module converts an image into a series of feature sequences and caches them based on the gray value data stream generated by the image gray value calculation module. After obtaining the feature sequence of a new image, the Hamming distance between the two feature sequences is calculated. The Hamming distance is used as the difference between the two images and is presented to the external device through the high-speed data transmission module. Then, the feature sequences in the cache are updated.

[0046] According to a specific embodiment of the present invention, as shown in the appendix Figure 1 As shown, the high-speed data transmission module, data unpacking module, image grayscale value calculation module, image sharpness evaluation module, and image difference evaluation module are all implemented on the FPGA platform. The high-speed data transmission module includes two sub-modules: a data downlink FIFO and a data uplink register. The data downlink FIFO organizes the data in the PCIE into a data stream and outputs it to the data unpacking module. The data uplink register is used to collect the data processing results of the entire FPGA platform and upload them to external devices via the PCIE protocol.

[0047] Generally, the PCIe link clock speed far exceeds the FPGA's operating clock speed. Therefore, the data receiving bit width on the FPGA usually needs to be larger to match the FPGA's data receiving rate with the PCIe data transmission rate. In this embodiment, the FPGA's data receiving bit width is 64 bits. It should be noted that storing one pixel requires 24 bits, and 64 is not divisible by this value. Therefore, a specially designed data unpacking module is needed to extract the pixel data.

[0048] like Figure 2As shown, the data unpacking module comprises three sub-modules: a header file parsing module, a bit-width conversion module, and a data error detection module. The header file parsing module receives a 64-bit wide data stream. The first cycle of data is the file header, which contains file type and length information. The file type needs to be matched against a preset file type in the header file. If the match is successful, subsequent data enters the bit-width conversion module; otherwise, the erroneous pathological slide image file is discarded. In this embodiment, the data stream output by the high-speed data transmission module has a bit width of 64 bits, and the image pixel format is 24-bit RGB. After obtaining the data, the bit-width conversion module needs to split the 64-bit data into 24-bit pixel data. This embodiment uses an asynchronous FIFO to design a 64-bit to 24-bit conversion module, which outputs the data to the subsequent processing module. Since the bandwidth for writing data to the FIFO is greater than the FIFO reading speed, a reverse voltage signal is designed in this module to temporarily stop the high-speed data transmission module from transmitting data to the asynchronous FIFO of the data unpacking module. The data error detection module receives the file length information from the header parsing module and counts the pixels output by the bit-width conversion module. If the count result does not match the file length, the erroneous pathological slide image file is discarded.

[0049] As attached Figure 1 As shown, the circuit system includes an image grayscale value calculation module. This module is relatively simple and therefore not illustrated separately. The function of this module is to calculate the grayscale value of each pixel based on the pixel data stream generated by the data unpacking module. The calculation formula is:

[0050] Gray=(Red*306+Green*601+Blue*116) / 1024

[0051] In the specific calculation process, the division by 1024 step is achieved through truncation. Ultimately, this module generates a grayscale data stream with one grayscale value per cycle, which is then used by other modules.

[0052] As attached Figure 3 As shown, the image sharpness evaluation module comprises three sub-modules: a grayscale value caching module, a texture detection module, and a texture counting module. The grayscale value caching module receives data from the grayscale data stream and caches it in a FIFO. When the amount of data in the FIFO reaches 20, it begins reading data, ensuring that the data read from the FIFO and the latest data to be written to the FIFO are 20 grayscale values ​​apart. The texture detection module receives the data read from the FIFO and the latest data to be written to the FIFO, calculating the difference between the two data every 10 cycles. If the difference is greater than 10, a texture is detected, and a relevant signal is given. The texture counting module counts the textures based on the signal from the texture detection module and reports the result to the high-speed data transmission module after image processing is complete.

[0053] As attached Figure 4 As shown, the image difference evaluation module comprises five sub-modules: a grayscale value caching module, an average value calculation module, an image feature sequence generation module, a feature sequence comparison module, and a feature sequence cache FIFO. The grayscale value caching module receives the grayscale value data stream, samples it at specific intervals, collects 2128 grayscale values ​​from all image data to form a grayscale value sequence, and caches it in the FIFO. The average value calculation module averages all grayscale values. The image feature sequence generation module retrieves the grayscale value sequence from the FIFO and binarizes it according to the average value, setting values ​​greater than the average to 1 and values ​​equal to or lower than the average to 0, thus generating the image feature sequence. The feature sequence cache FIFO is responsible for caching the feature sequence of the previous image. The feature sequence comparison module receives the feature sequence of the current image and retrieves the feature sequence of the previous image from the feature sequence buffer FIFO. The values ​​of the two are compared one by one. If they do not match, the counter is incremented by one. The final result of the counter is the Hamming distance between the two sequences, which can be used to represent the difference between the two images. This difference is reported to the high-speed data transmission module.

[0054] The high-speed data transmission module transmits two indicators, clarity and difference, to an external device, which is a regular computer host. The filtering method is as follows: first, image difference is considered; if the difference exceeds 100, a new image is considered to have been captured, and a high-quality image needs to be selected from the subsequent images; then, the clarity of the subsequent images is observed; if the clarity exceeds 60,000, it is considered a high-quality image and is output; after outputting a high-quality image, image output stops unless the difference exceeds 100 again, in which case the above process is repeated, thereby achieving image filtering.

[0055] The embodiments described above are merely some preferred embodiments of the present invention, and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained by equivalent substitution or equivalent transformation fall within the protection scope of the present invention.

Claims

1. A method for rapid screening of pathological slide images based on FPGA, characterized in that, include: The FPGA acquires pathological slide image data to be screened from an external device. After unpacking the image data, the FPGA processes the data to obtain the image clarity and the difference between images, and transmits it to the external device. The external device then uses these two indicators as judgment criteria to screen the images.

2. The method for rapid screening of pathological slide images based on FPGA according to claim 1, characterized in that, After the FPGA unpacks the image data, it generates a pixel data stream of one pixel per system clock cycle; The grayscale value is calculated based on the pixels in the pixel data stream, and a grayscale value data stream is generated.

3. The method for rapid screening of pathological slide images based on FPGA according to claim 2, characterized in that, Gray values ​​are extracted from the gray value data stream at fixed intervals and put into a buffer. Then the difference between the gray values ​​in the buffer is calculated. If the difference is greater than a preset threshold, a texture is considered to have been detected. The number of times a texture is detected during the processing of an image is counted, and the statistical result is used as the image sharpness.

4. The method for rapid screening of pathological slide images based on FPGA according to claim 2, characterized in that, A fixed-length grayscale value sequence is formed by sampling grayscale data streams at specific intervals. The grayscale value sequence is binarized to form an image feature sequence, which is then cached. During the generation of a new image feature sequence, the old image feature sequence is retrieved from the cache and compared with the new image feature sequence. The Hamming distance between the new and old image feature sequences is calculated, and the image feature sequence in the cache is updated. The calculated Hamming distance is used as the difference between the images.

5. The method for rapid screening of pathological slide images based on FPGA according to claim 3, characterized in that, The binarization specifically involves averaging the gray values ​​of the gray value sequence, setting values ​​greater than the average to 1, and values ​​equal to or lower than the average to 0.

6. A rapid screening circuit for pathological slide images based on FPGA, characterized in that, Implemented based on FPGA, including: high-speed data transmission module, data unpacking module, image grayscale value calculation module, image sharpness evaluation module, and image difference evaluation module; The high-speed data transmission module connects to an external device via a PCIE interface to acquire pathological slide image data packets from the external device and transmit them to the data unpacking module. After the image preprocessing calculation is completed, the results are presented to the external device via the PCIE interface. The data unpacking module unpacks the data packet according to the information contained in the header file of the data packet, thereby generating a pixel data stream of one pixel data per system clock cycle; The image grayscale value calculation module calculates the grayscale value corresponding to each pixel data according to the pixel data stream generated by the data unpacking module, thereby generating a grayscale value data stream with one grayscale value per system clock cycle. The image sharpness evaluation module calculates the difference between the grayscale values ​​of two pixels at fixed intervals according to the grayscale value data stream generated by the image grayscale value calculation module. If the difference is greater than a threshold, it is considered that a texture has appeared. The number of textures in the image is counted to obtain the image sharpness, which is then transmitted to the high-speed data transmission module. The image difference evaluation module converts an image into a series of feature sequences and caches them according to the grayscale value data stream generated by the image grayscale value calculation module. After obtaining the feature sequence of a new image, the Hamming distance between the two feature sequences is calculated. The Hamming distance is used as the difference between the two images and transmitted to the high-speed data transmission module.

7. The FPGA-based rapid screening circuit for pathological slide images according to claim 6, characterized in that, The high-speed data transmission module includes a data downlink FIFO and a data uplink register. The data downlink FIFO is used to organize the data received by the PCIe interface and output it to the data unpacking module. The data uplink register is used to collect the data processing results of the entire FPGA platform and upload them to external devices via the PCIe protocol.

8. The FPGA-based rapid screening circuit for pathological slide images according to claim 6, characterized in that, The data unpacking module includes a header file parsing module, a bit width conversion module, and a data error detection module. The header file parsing module receives the data stream output by the high-speed data transmission module. The first cycle of data is the file header, which contains file category and file length information. The file category needs to be matched with the preset file category in the header file. If the match is successful, the subsequent data enters the bit width conversion module. If the match is unsuccessful, the erroneous pathological slide image file is discarded. The bit width conversion module is used to convert the obtained data into a pixel data stream with one pixel data per system clock cycle, and output it to the subsequent processing module.

Images