A pathological model edge computing box device based on edge computing

Abstract

The present invention provides a pathology model edge computing box device based on edge computing, comprising: at least one image acquisition terminal and an inference terminal; wherein the image acquisition terminal acquires images of pathological tissue samples; the inference terminal performs recognition and inference on the images received from the image acquisition terminal and returns the results to the image acquisition terminal. The pathology model edge computing box device based on edge computing of the present invention performs local data processing on the edge device, preventing sensitive data from being transmitted to the cloud, helping to protect the privacy of users and patients. Furthermore, since local processing eliminates the need for data transmission, additional costs are avoided.

Description

Technical Field

[0001] The present invention relates to the field of edge computing technology, and in particular to a pathology model edge computing box device based on edge computing. Background Art

[0002] In recent years, the rapid development and widespread application of deep learning technology, particularly the rise of convolutional neural networks (CNNs), has achieved tremendous success in various fields, including medical image analysis. Trained on large amounts of pathological image data, deep learning models can automatically learn and extract features from images, accurately locating and segmenting lesions, and providing doctors with reliable diagnostic tools.

[0003] Existing cloud computing models are used to assist doctors. However, while traditional cloud computing can perform pathology image analysis and calculations on cloud servers, it also faces challenges such as privacy protection, high latency, and high data transmission costs, limiting its application in real-time diagnosis and mobile healthcare. This is because cloud computing requires transmitting large amounts of pathology image data from edge devices to cloud servers for processing, resulting in long data transmission delays and high server room maintenance costs. Furthermore, because pathology images may contain sensitive personal health information, cloud computing must ensure privacy during data transmission and processing, adding additional privacy protection measures and costs. Summary of the Invention

[0004] One of the purposes of the present invention is to provide a pathology model edge computing box device based on edge computing. By performing local data processing on the edge device, sensitive data can be avoided from being transmitted to the cloud, which helps to protect the privacy of users and patients. Moreover, since it is processed locally, no data transmission is required, thus avoiding additional costs.

[0005] An embodiment of the present invention provides a pathology model edge computing box device for edge computing, including: at least one image acquisition end and an inference end;

[0006] Among them, the image acquisition end acquires images of pathological tissue samples; the inference end recognizes and infers the images received from the image acquisition end and returns the results to the image acquisition end.

[0007] Preferably, the image acquisition end includes: an electron microscope and a PC terminal.

[0008] Preferably, the PC performs the following operations:

[0009] Acquiring an image of the pathological tissue sample through an electron microscope to obtain an initial image;

[0010] Preprocess the initial image;

[0011] The pre-processed initial image is sent to the inference end via the TCP / IP network protocol.

[0012] Preferably, the preprocessing includes: image brightness adjustment and / or selection of a region of interest.

[0013] Preferably, the PC terminal also performs the following operations:

[0014] Match the pre-processed initial image with the temporary image in the temporary library;

[0015] When there is a temporary image that matches the initial image, a local recognition and reasoning result of the initial image is generated and output according to the recognition and reasoning result associated with the temporary image.

[0016] Preferably, a deep learning model is configured at the inference end, wherein the deep learning model includes: a model for cell detection, a model for cell counting, a model for pathology classification, and a model for lesion detection, or one or more combinations thereof.

[0017] Preferably, the inference end further performs the following operations:

[0018] Retrieve historical images of the object corresponding to the initial image and corresponding recognition inference results;

[0019] A growth graph is constructed based on the retrieved historical images, the recognition and reasoning results corresponding to the historical images, the initial image, and the recognition and reasoning results corresponding to the initial image, and is output to the image acquisition end.

[0020] Preferably, the inference end further performs the following operations:

[0021] The accuracy is calculated based on the output of each model's neurons and the model's output during the generation of the recognition inference result corresponding to the initial image;

[0022] When the accuracy is less than or equal to the preset accuracy threshold, the initial image is sent to the server;

[0023] Receiving the recognition and reasoning result of the server side for the initial image and verifying the recognition and reasoning result of the reasoning side based on the received recognition and reasoning result;

[0024] If the verification fails, access the server to update the configured deep learning model.

[0025] Preferably, the inference end further performs the following operations:

[0026] When the recognition reasoning result is a result in the pre-configured verification library, the initial image is sent to the server for recognition reasoning.

[0027] Preferably, the inference end further performs the following operations:

[0028] When the identification reasoning result is a result in a pre-configured verification library, a pre-configured communication list of reasoning terminals of other adjacent devices is retrieved;

[0029] Send the initial image to a preset number of inference terminals arranged in descending order of difference, and receive the returned recognition inference results;

[0030] Based on the returned recognition and reasoning results, a joint analysis display interface is generated and output.

[0031] The present invention has the following advantages:

[0032] 1. By performing local data processing on edge devices, sensitive data can be prevented from being transmitted to the cloud, helping to protect the privacy of users and patients.

[0033] 2. Edge computing reduces data transmission time and latency compared to cloud computing. Edge devices effectively utilize local computing resources to perform data processing and analysis near the device, reducing data volume and transmission time, and improving real-time performance. This not only reduces data transmission and processing costs, but also saves bandwidth costs.

[0034] 3. Edge computing demonstrates a clear advantage in inference time. Compared to cloud computing models that transmit data to cloud servers for processing, inference time on edge devices is much shorter. This reduces latency in result feedback, helping to achieve real-time diagnosis and mobile healthcare requirements.

[0035] 4. Edge computing reduces dependence on network connectivity. Even when the network is unstable or disconnected, edge devices can still process data and make decisions, improving system reliability. This can reduce server room maintenance costs for medical institutions.

[0036] 5. The portability of edge devices also makes them suitable for teaching, for example, by medical researchers. Traditional medical education typically relies on large servers and computer equipment, which limits the mobility and flexibility of the teaching process.

[0037] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0038] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0040] Figure 1 Schematic diagram of a pathology model edge computing box device based on edge computing in an embodiment of the present invention;

[0041] Figure 2 This is a topological diagram of a pathological model edge computing box device based on edge computing in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0043] The embodiment of the present invention provides a pathological model edge computing box device based on edge computing, such as Figures 1 to 2 As shown, it includes: at least one image acquisition end 1 and an inference end 2;

[0044] The image acquisition terminal 1 acquires images of pathological tissue samples; the inference terminal 2 performs recognition and inference on the images received from the image acquisition terminal and returns the results to the image acquisition terminal.

[0045] The image acquisition terminal 1 includes an electron microscope and a PC terminal.

[0046] The PC performs the following operations:

[0047] Acquiring an image of the pathological tissue sample through an electron microscope to obtain an initial image;

[0048] Preprocess the initial image;

[0049] The pre-processed initial image is sent to the inference end via the TCP / IP network protocol.

[0050] The preprocessing includes: image brightness adjustment and / or selection of a region of interest.

[0051] A deep learning model is configured at the inference end 2, wherein the deep learning model includes: a model for cell detection, a model for cell counting, a model for pathology classification, and a model for lesion detection, or a combination thereof.

[0052] The working principle and beneficial effects of the above technical solution are:

[0053] The edge computing-based pathology model edge computing box device provided in this embodiment includes: at least one image acquisition terminal and an inference terminal. The case image acquisition terminal (i.e., the image acquisition terminal) is used to acquire pathology images. By connecting an electron microscope to a computer, it can acquire images of pathological tissue samples and transmit the acquired images to the computer for image preprocessing. After obtaining the pathology image inference results from the inference terminal, post-processing operations such as visualization can be performed for analysis and research by medical researchers. The edge computing inference terminal (i.e., the inference terminal) is used for inference of pathology images and has the computing power and storage resources required for inference. As a computing platform for pathology image recognition and inference, trained deep learning models, such as those for cell detection, cell counting, pathology classification, and lesion detection, are deployed on the device. It is responsible for performing recognition and inference tasks on received pathology images and returning the results to the computer at the image acquisition terminal. The overall process: First, an electron microscope is used to acquire high-resolution images of pathological specimens. The acquired images are transmitted to the computer via an electron microscope system connected to the computer. Image preprocessing operations, such as image brightness adjustment and region of interest selection, are then performed in the computer application. Next, the computer establishes a TCP / IP network protocol connection with the edge device on the inference side for data communication and transmission. After the computer receives the pathology image data from the electron microscope, it transmits the image data to the inference side via the TCP / IP protocol, where it performs inference operations using the deployed model. After inference is complete, the recognition and inference results of the pathology image are returned to the computer, enabling two-way communication and transmission. Finally, the computer performs post-processing steps such as parsing and visualizing the inference results for reference by medical researchers. The inference side and the acquisition side have a one-to-many relationship, meaning that an edge device on the inference side can simultaneously provide inference for multiple acquisition sides. Through the above steps, we have implemented a pathology image recognition and inference solution based on edge computing, distributing the pathology image acquisition and recognition tasks across different devices, achieving collaborative processing of computing tasks and efficient resource utilization.

[0054] In one embodiment, the PC terminal further performs the following operations:

[0055] Match the pre-processed initial image with the temporary image in the temporary library;

[0056] When there is a temporary image that matches the initial image, a local recognition and reasoning result of the initial image is generated and output according to the recognition and reasoning result associated with the temporary image.

[0057] The execution steps of the PC end provided in this embodiment are to more quickly perform recognition and reasoning results on the initial image. By building a local storage temporary library on the PC end, a small number of (0-20) temporary images and the recognition and reasoning results corresponding to the temporary images are stored in the temporary library; by directly matching the initial image with the temporary images stored in the temporary library, a rapid prediction of the initial image is achieved, and the recognition and reasoning results of the reasoning end can be verified to a certain extent. The temporary images in the temporary library are

[0058] To implement growth analysis and display of pathological trends, in one embodiment, the inference end further performs the following operations:

[0059] Retrieve historical images of the object corresponding to the initial image and corresponding recognition inference results;

[0060] A growth graph is constructed based on the retrieved historical images, the recognition and reasoning results corresponding to the historical images, the initial image, and the recognition and reasoning results corresponding to the initial image, and is output to the image acquisition end.

[0061] To ensure the validity of the model on the inference side, in one embodiment, the inference side also performs the following operations:

[0062] The accuracy is calculated based on the output of the neurons of each model and the output of the model during the generation of the recognition inference result corresponding to the initial image. The accuracy is the ratio of the number of neurons whose output of the neurons of each model is the same as the output of the model to the total number of neurons.

[0063] When the accuracy is less than or equal to the preset accuracy threshold, the initial image is sent to the server;

[0064] Receive the recognition and reasoning results of the server for the initial image and verify the recognition and reasoning results of the reasoning end based on the received recognition and reasoning results; the verification mainly determines whether the reasoning results sent by the server are the same as the reasoning results of the reasoning end; if they are the same, the verification passes; otherwise, it fails;

[0065] If the verification fails, access the server to update the configured deep learning model.

[0066] This embodiment filters the initial image by triggering, and verifies the feedback result obtained based on the filtered initial image sent to the server. When the recognition and reasoning results are inconsistent, the verification fails, and it is necessary to access the server to update the configured deep learning model.

[0067] In one embodiment, the inference side also performs the following operations:

[0068] When the recognition reasoning result is a result in the pre-configured verification library, the initial image is sent to the server for recognition reasoning.

[0069] In this embodiment, a verification library is constructed at the inference end. The doctor configures the results in the verification library in advance. When the result of the inference at the inference end is the result in the library, it is uploaded to the server for verification. Since the inference end is an edge computing device after all, due to its performance (limited computing power), the doctor can configure the verification library to provide server-side computing power support for the initial image of the recognition reasoning result that needs to be re-confirmed, thereby ensuring the accuracy and effectiveness of the recognition reasoning result.

[0070] In one embodiment, the inference side also performs the following operations:

[0071] When the identification reasoning result is a result in a pre-configured verification library, a pre-configured communication list of reasoning terminals of other adjacent devices is retrieved;

[0072] Send the initial image to a preset number of inference terminals arranged in descending order of difference, and receive the returned recognition inference results;

[0073] Based on the returned recognition and reasoning results, a joint analysis display interface is generated and output.

[0074] This embodiment configures a communication list of other inference terminals on the inference terminal. When verification is required, the initial image can be sent through communication with other inference terminals and feedback can be received, thereby verifying the recognition inference results of the initial image. When selecting the inference terminals, the difference can be arranged in descending order, and the inference terminals with the top N differences can be selected as communication targets; the difference is quantified by comprehensively analyzing the differences in the inference terminal models, specifically including model parameters (model type, number of neurons, etc.), loss function, and the differences in the data used for training (the proportion of normal tissue images and pathological tissue images, and the proportion of tissue images of each type of pathological type). The steps for determining the degree of difference are as follows: calculating the similarity of the labeled data sets between the two models; determining the degree of difference based on the similarity; the degree of difference is the value obtained by subtracting the similarity from 1; wherein, the labeled data set is constructed and associated with the model through the following steps, and the specific steps are as follows: taking the model parameters, loss function, and data used for training as the data to be analyzed; performing feature extraction on the analyzed data to construct a labeled data set; the parameters in the labeled data set include: parameters representing each model parameter, parameters representing the type of loss function, parameters representing the proportion of normal tissue images and pathological tissue images in the training data, and parameters representing the proportion of tissue images of each type of pathological type.

[0075] In one embodiment, the joint analysis display interface also displays the confidence parameters of each inference end for the inference results returned by it;

[0076] The confidence parameter is determined by the following steps:

[0077] Obtain the number of processed records on the inference side, the accuracy of the processed records, the proportion of processed records that are identical to the returned inference results (the ratio of the number of identical records to the total number of records), and the time of the last update as analysis data;

[0078] Extract features from the analysis data and construct an analysis parameter set based on them. The analysis parameter set includes: a parameter representing the number of processed records, a parameter representing the accuracy of the processed records, a parameter representing the percentage of processed records that are identical to the returned inference result, and a parameter representing the time difference between the last update time and the current time.

[0079] According to the analysis parameter set, the corresponding confidence parameters are retrieved from the pre-configured confidence parameter analysis library;

[0080] Among them, the confidence parameter analysis set is analyzed and configured in advance by professionals, and the analysis parameter set and the confidence parameter are associated one-to-one in the confidence parameter analysis library; in the process of constructing the confidence parameter analysis set, the larger the parameter of the number of processing records, the larger the confidence parameter; the higher the parameter representing the accuracy of the processing records, the larger the confidence parameter; the larger the parameter representing the proportion of processing records that are the same as the returned inference results, the larger the confidence parameter; the smaller the parameter representing the time difference between the last update time and the current time, the larger the confidence parameter; the display of confidence parameters facilitates the screening of results by the corresponding staff at the inference end.

[0081] In one embodiment, when receiving an initial image from another inference end, determining the processing timing of the initial image based on the current working state;

[0082] When the inference end is idle, the initial image is processed immediately;

[0083] When the inference end is in operation, the idle time intervals between the previous N processing times and the time spent on the previous N processing times are obtained. When the average of the idle time intervals between the current N processing times is greater than or equal to half the average of the time spent on the previous N processing times, the initial image is processed when the time spent on the next processing time is less than or equal to the average of the time spent on the previous N processing times. Where N is an integer greater than or equal to 2.

[0084] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A pathology model edge computing box device based on edge computing, characterized in that: include: At least one image acquisition terminal and inference terminal; Among them, the image acquisition end acquires images of pathological tissue samples; the inference end performs recognition and inference on the images received from the image acquisition end and returns the results to the image acquisition end; The inference side also performs the following operations: When the recognition reasoning result is a result in a pre-configured verification library, the initial image is sent to a preset number of reasoning terminals in a pre-configured communication list of reasoning terminals, which are arranged in descending order of difference; wherein the difference is quantified by comprehensively analyzing the differences of the models of the reasoning terminals; Based on the returned recognition and reasoning results, a joint analysis display interface is generated and output; The joint analysis display interface also displays the confidence parameters of each inference end for the inference results it returns; The confidence parameter is determined by the following steps: Obtain the number of processed records on the inference side, the accuracy of the processed records, the percentage of processed records that match the returned inference results, and the time of the last update as analysis data; Extract features from the analysis data and construct an analysis parameter set based on them. The analysis parameter set includes: a parameter representing the number of processed records, a parameter representing the accuracy of the processed records, a parameter representing the percentage of processed records that are identical to the returned inference result, and a parameter representing the time difference between the last update time and the current time. According to the analysis parameter set, the corresponding confidence parameters are retrieved from the pre-configured confidence parameter analysis library; In the process of constructing the confidence parameter analysis set, the larger the parameter representing the number of processed records, the larger the confidence parameter; the higher the parameter representing the accuracy of the processed records, the larger the confidence parameter; the larger the parameter representing the proportion of processed records that are the same as the returned inference results, the larger the confidence parameter; the smaller the parameter representing the time difference between the last update time and the current time, the larger the confidence parameter.

2. The pathological model edge computing box device based on edge computing according to claim 1 is characterized in that: The image acquisition end includes: electron microscope and PC terminal.

3. The pathological model edge computing box device based on edge computing according to claim 2 is characterized in that: Perform the following operations on the PC: Acquiring an image of the pathological tissue sample through an electron microscope to obtain an initial image; Preprocess the initial image; The pre-processed initial image is sent to the inference end via the TCP / IP network protocol.

4. The pathological model edge computing box device based on edge computing according to claim 3 is characterized in that: The preprocessing includes: image brightness adjustment and / or selection of a region of interest.

5. The pathological model edge computing box device based on edge computing according to claim 1 is characterized in that: The PC also performs the following operations: Match the pre-processed initial image with the temporary image in the temporary library; When there is a temporary image that matches the initial image, a local recognition and reasoning result of the initial image is generated and output according to the recognition and reasoning result associated with the temporary image.

6. The pathological model edge computing box device based on edge computing according to claim 1 is characterized in that: A deep learning model is configured at the inference end, wherein the deep learning model includes: a model for cell detection, a model for cell counting, a model for pathology classification, and a model for lesion detection, or a combination of one or more of these.

7. The pathological model edge computing box device based on edge computing according to claim 1 is characterized in that: The inference side also performs the following operations: Retrieve historical images of the object corresponding to the initial image and corresponding recognition inference results; A growth graph is constructed based on the retrieved historical images, the recognition and reasoning results corresponding to the historical images, the initial image, and the recognition and reasoning results corresponding to the initial image, and is output to the image acquisition end.

8. The pathological model edge computing box device based on edge computing according to claim 1 is characterized in that: The inference side also performs the following operations: The accuracy is calculated based on the output of each model's neurons and the model's output during the generation of the recognition inference result corresponding to the initial image; When the accuracy rate is less than or equal to a preset accuracy threshold, sending the initial image to the server; Receiving the recognition and reasoning result of the server side for the initial image and verifying the recognition and reasoning result of the reasoning side based on the received recognition and reasoning result; If the verification fails, access the server to update the configured deep learning model.

9. The pathological model edge computing box device based on edge computing according to claim 1 is characterized in that: The inference side also performs the following operations: When the recognition reasoning result is a result in the pre-configured verification library, the initial image is sent to the server for recognition reasoning.

Images