Pulmonary CT image AI auxiliary diagnosis system and method based on embedded edge computing

Abstract

This invention belongs to the fields of medical imaging equipment and computer-aided diagnosis, specifically relating to an AI-assisted diagnostic system and method for lung CT images based on embedded edge computing. The system includes a development board, a data input interface, an intelligent edge computing module, a visualization output interface, and a web display module. Using a high-performance embedded development board as its core, this invention achieves real-time and accurate analysis of lung CT images in a network-free environment through hardware and software co-optimization and a lightweight dual-model architecture. This significantly reduces diagnostic latency and device power consumption. Simultaneously, the intuitive interactive visualization interface effectively assists doctors in improving diagnostic efficiency and accuracy. The average time for a single CT image analysis is less than 70 seconds, and the peak memory usage does not exceed 1.2GB.

Description

Technical Field

[0001] This invention belongs to the fields of medical imaging equipment and computer-aided diagnosis, specifically relating to an AI-assisted diagnostic system and method for lung CT images based on embedded edge computing. Background Technology

[0002] Edge computing is a computing paradigm that performs data processing, storage, and network services closer to the data source.

[0003] In the field of lung imaging-assisted diagnosis, most current mainstream computer-aided diagnostic systems employ a centralized deep learning architecture. These systems typically rely on uploading high-resolution CT image data of patients to the cloud or a central server for processing and analysis. This approach presents several significant problems: First, high-resolution medical images (such as thin-slice CT) generate massive amounts of data, and the process of transmitting them to the cloud is time-consuming, leading to diagnostic delays and failing to meet real-time clinical requirements. Second, the centralized processing model is highly dependent on network bandwidth and central computing resources, limiting system availability in scenarios with limited medical resources or poor network conditions. Finally, remotely transmitting patient image data containing sensitive information poses potential risks to data security and patient privacy.

[0004] In recent years, the rise of edge computing technology has provided a new approach to solving the aforementioned bottlenecks. Unlike the traditional model that relies on remote centers, the core of edge computing lies in deploying data processing and analysis tasks on network edge devices close to the data generation source (such as the radiology department of a hospital). This technical solution has the following advantages: First, by processing image data locally, it can greatly reduce data transmission volume, significantly reduce diagnostic latency, achieve near real-time auxiliary analysis, and improve diagnostic efficiency; second, edge devices operate independently, reducing dependence on central servers and stable network connections, and enhancing the system's robustness and accessibility in complex medical environments; third, the patient's original image data is analyzed locally without uploading to external systems, fundamentally strengthening the protection of data privacy and security, and better meeting the compliance requirements of medical data management. Therefore, exploring the application of edge computing architecture to lung cancer image-assisted diagnosis is of great significance for building an efficient, reliable, and secure clinical decision support system.

[0005] Patent CN111062043A proposes a medical image recognition system based on edge computing. Its core solution is: CT image - edge device preprocessing - cloud center analysis - return of diagnostic report. However, the results need to be transmitted back from the cloud, with an average delay of ≥3.5 hours. The interactivity is insufficient, only outputting text reports and lacking three-dimensional visualization capabilities. The single model structure has a detection rate of only 91.3% for early lung cancer.

[0006] Therefore, in summary, the evolution of lung imaging-assisted diagnostic technology is facing the challenge of a core paradigm shift. Currently, edge computing solutions, represented by patent CN111062043A, while attempting to overcome the drawbacks of traditional pure cloud architectures through the "edge-cloud collaboration" approach, still haven't fully realized the potential of edge computing. Their "preprocessing at the edge, core analysis in the cloud" model is essentially still limited by network transmission and cloud processing queues, resulting in unresolved diagnostic delays (average ≥3.5 hours), making them unsuitable for time-sensitive clinical scenarios such as emergency care and intraoperative navigation. Furthermore, existing solutions also have significant shortcomings in diagnostic interactivity and accuracy: on the one hand, their output format is singular, lacking intuitive and three-dimensional visualization assistance, limiting doctors' in-depth understanding of the spatial relationships of lesions and surgical planning; on the other hand, their single model structure has limited detection capabilities for early-stage small lung nodules (especially nodules with a diameter ≤3mm) (detection rate below 95%), posing a significant risk of missed diagnoses.

[0007] Therefore, existing technologies still have significant gaps in three key dimensions: real-time performance, interactivity, and high sensitivity in detecting minute lesions. To build a truly efficient, reliable, and accurate auxiliary system for the early diagnosis of lung diseases, a novel technological approach is urgently needed to achieve rapid and accurate image analysis on terminal devices, and to provide intuitive decision support based on this analysis. Summary of the Invention

[0008] This invention provides an AI-assisted diagnostic system and method for lung CT images based on embedded edge computing, aiming to solve the following technical problems: (1) Breaking through the bottleneck of cloud diagnosis delay: In response to the delay of several hours caused by the reliance on cloud computing in existing medical AI systems, this invention realizes real-time analysis of CT images on the embedded end, meeting the needs of rapid diagnosis in scenarios such as emergency in a network-free environment. (2) Improving the detection rate of early small lesions: Overcoming the industry problem of insufficient sensitivity of existing technologies in detecting small lung nodules, significantly reducing the risk of missed diagnosis of early lung cancer.

[0009] The technical solution of the present invention is as follows: The AI-assisted diagnostic system for lung CT images based on embedded edge computing includes: a development board, a data input interface, an intelligent edge computing module, a visualization output interface, and a web display module.

[0010] The development board has a built-in NPU and CPU unit to accelerate the inference calculation of the segmentation model and classification model in the intelligent edge computing module. The data input interface is used to receive lung CT image data from a CT scanner or external storage device and transmit it to the intelligent edge computing module; The intelligent edge computing module, deployed on the development board, is used to directly perform lung CT image analysis locally on the device. The intelligent edge computing module includes pre-converted and optimized segmentation and classification models. The segmentation model is a lung tissue and nodule region segmentation network built on a convolutional neural network. It uses an encoder-decoder structure to extract features and fuse spatial information from the input lung CT images. Its input is multi-channel two-dimensional CT image data containing target slices and their context slice information, and its output is a pixel-level probability segmentation map of the lung region and suspicious nodule region, thereby achieving accurate segmentation of lung tissue and preliminary localization of suspicious nodules. The classification model is a nodule benign or malignant discrimination network constructed based on a three-dimensional convolutional neural network. Its input is a fixed-size three-dimensional CT image block cropped around the central region of the suspicious nodule located by the segmentation model. The spatial texture and morphological features of the nodule are extracted through multi-layer three-dimensional convolution and downsampling operations, and the benign or malignant judgment result and malignant probability value of the nodule are output through a fully connected classification layer. The segmentation model and the classification model constitute a dual-model architecture for end-side collaborative reasoning. The segmentation model is used to narrow down the candidate region range and reduce the amount of classification computation, while the classification model is used to perform fine discrimination of candidate nodules, thereby improving the system's diagnostic accuracy and reasoning efficiency. Both the segmentation model and the classification model are executed on the development board's NPU unit after the trained deep neural network model is converted from ONNX format to RKNN format, and then subjected to lightweight compression and operator optimization to meet the real-time inference requirements of embedded devices.

[0011] The aforementioned visualization output interface is used to output the analysis results of the intelligent edge computing module to a display device; The aforementioned WEB display module runs on the development board and is used to generate and provide an interactive web page interface to dynamically present the diagnostic results.

[0012] Preferably, the steps of the intelligent edge computing module performing lung CT image analysis include: The segmentation model is used to process the input lung CT images to accurately segment lung tissue and locate suspicious nodule areas. The classification model is used to calculate the segmented suspicious nodules, determine their benign or malignant nature, and output the probability of malignancy.

[0013] Preferably, the content presented by the interactive webpage interface provided by the WEB display module includes: A 3D lung model generated based on the segmentation results, supporting mouse rotation and zoom; The locations of suspicious nodules are marked in the form of a heatmap on the three-dimensional lung model; Detailed coordinates, benign / malignant assessment results, and malignant probability value for each suspicious nodule; The system generates a structured diagnostic report that includes nodule location and nature assessment.

[0014] Preferably, the segmentation model is in the lung.rknn format and the classification model is in the lung_cls.rknn format. The two are obtained by converting the self-built neural network model from the ONNX format to the RKNN format and working together on the development board.

[0015] The AI-assisted diagnostic method for lung cancer CT images based on embedded edge computing, using the aforementioned system, involves the following steps: Receive lung CT image data through the data input interface; On the development board, its NPU unit is used to call the segmentation model in the intelligent edge computing module to process CT images, complete lung tissue segmentation and nodule localization; The classification model in the intelligent edge computing module is invoked to classify the located nodules as benign or malignant. The web presentation module generates an interactive webpage containing a 3D lung model, nodule heatmap, detailed diagnostic information, and a structured report. The interactive webpage is output to a display device via a visual output interface for doctors to use as a diagnostic reference.

[0016] The beneficial effects of this invention are: This invention uses a high-performance embedded development board as its core and achieves real-time, accurate analysis of lung CT images in a network-free environment through hardware and software co-optimization and a lightweight dual-model architecture. This significantly reduces diagnostic latency and device power consumption. Simultaneously, an intuitive interactive visualization interface effectively assists doctors in improving diagnostic efficiency and accuracy. The system's average time for a single CT image analysis is less than 70 seconds, with peak memory usage not exceeding 1.2GB. Attached Figure Description

[0017] Figure 1 This is the overall architecture of the system of the present invention; Figure 2 This is the overall architecture of the software system of the present invention; Figure 3 This is the overall architecture of the intelligent edge computing module of the present invention; Figure 4 This is a flowchart of the intelligent edge computing module of the present invention. Detailed Implementation

[0018] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings and technical solutions.

[0019] Figure 1 This is a schematic diagram of the AI-assisted diagnostic system for lung CT images based on embedded edge computing described in this invention. In this embodiment, the Feiling Embedded ELF2 development board is used as the hardware core to construct a complete closed loop from image acquisition, edge intelligent analysis to result visualization, wherein: A data input interface for receiving lung CT image data from a CT scanner or external storage device, wherein the data format includes, but is not limited to, mhd / raw format; The Feiling ELF2 development board is equipped with the RK3588 chip, whose built-in NPU and CPU (Neural Network Processing Unit) are used to accelerate deep learning model inference. The intelligent edge computing module, deployed on the ELF2 development board, includes a pre-converted and optimized segmentation model (lung.rknn) and classification model (lung_cls.rknn) for performing lung tissue segmentation and nodule benign / malignant classification directly on the device. The WEB display module, which also runs on the ELF2 development board, is used to generate and provide a dynamic, interactive HTML web page interface. The visualization output interface is used to output the web page interface generated by the WEB display module to the medical display screen for doctors to use for diagnosis.

[0020] Preferably, all the above calculations and analyses are performed locally on the ELF2 development board without connecting to the cloud, ensuring data privacy and real-time performance.

[0021] Preferably, the above-mentioned segmentation model and classification model are obtained by converting the self-constructed neural network model from ONNX format to RKNN format, so as to make full use of the NPU computing power of the RK3588 chip.

[0022] The steps for the intelligent edge computing module to perform CT image analysis include: The input CT image sequence is processed by the segmentation model (lung.rknn) to accurately segment the lung tissue region and locate suspicious nodules within it; The classification model (lung_cls.rknn) calculates the probability of malignancy for each segmented suspicious nodule and outputs its probability of malignancy, and completes the benign or malignant judgment based on a preset threshold.

[0023] The web display module is built on the Flask framework for backend services and uses the Three.js library for frontend 3D rendering. The interactive web interface it provides presents the following content: A 3D lung model generated based on the segmentation results, supporting mouse rotation and zoom; The locations of suspicious nodules are highlighted in the form of a heatmap on the 3D model; The list displays the detailed coordinates, benign / malignant assessment results, and specific malignant probability values ​​for each suspicious nodule; The system automatically generates a structured diagnostic report containing all nodule location and nature assessment conclusions.

[0024] The hardware component of the system also includes a custom-designed shell using 3D printing technology. This shell is designed to fit the shape of the ELF2 development board and provides necessary interface locations and space for the cooling fan to ensure stable operation of the device.

[0025] The aforementioned ELF2 development board can be understood as an embedded edge computing node that integrates computing, storage, and communication capabilities. It directly acquires CT image data through its data interface (such as USB), utilizes onboard computing resources (CPU, NPU) to complete complex AI inference tasks that originally required cloud servers, and directly outputs intelligent diagnostic results through interfaces such as HDMI.

[0026] Preferably, the system takes an average of about 68 seconds to process a whole lung CT image at a time, with peak memory usage controlled within 1.2GB, and power consumption reduced by more than 60% compared to traditional cloud solutions.

[0027] Figure 2-4 These are the core module block diagrams and flowcharts of the system described in this invention, wherein: like Figure 2 As shown, the system adopts a three-tier architecture design: Application Interaction Layer: Built on the Flask and Three.js framework, it is responsible for implementing the 3D visualization interface of CT data, which can interactively display the segmented nodules and present the classification results. The response latency of this layer is less than 200ms.

[0028] The core algorithm layer comprises two main components: a segmentation model and a classification model. The segmentation model uses a variant of U-Net, taking a 7×512×512 CT image tensor as input and outputting an 8-channel mask for the lung and nodule regions. After quantization, the inference speed on embedded devices can reach 20 milliseconds per frame. The classification model employs a 3D ResNet architecture, taking a 32×48×48 three-dimensional image patch cropped centered on the nodule as input and outputting the malignancy probability. Its classification performance achieves an AUC of 0.911 on the test set.

[0029] Hardware driver layer: Responsible for underlying data preprocessing and hardware acceleration. This layer processes the raw medical image data (mhd / raw format) into standard HU tensors and uses tools such as RKNN-Toolkit2 for model quantization and NPU instruction set optimization, achieving a balance between performance and power consumption through dynamic frequency adjustment (0.8~1.2GHz).

[0030] Figure 3 The core workflow within the intelligent edge computing module was further demonstrated, consisting of three parallel collaborative units: Nodule Analyzer: Receives coordinate information, extracts the region of interest from CT images after coordinate transformation, calculates the probability of nodule malignancy, and generates a structured diagnostic report.

[0031] Resource optimizer: Continuously monitors system resources (such as CPU load), dynamically makes decisions and adjusts the calculation precision based on resource scarcity (such as switching to low-precision calculation or maintaining full precision) to ensure stable system operation under different loads.

[0032] Model Inference Engine: Executes the complete inference pipeline, including data preprocessing, calling the NPU via RKNN tools to accelerate inference (or falling back to GPU / CPU when resources are limited), and post-processing to output the final mask data.

[0033] Figure 4 This demonstrates the software code organization structure that supports the aforementioned algorithms and processes. The core modules include: Model definition module: model.py is used to define the structure and tools of neural network models.

[0034] Data processing module: dsets.py is used to define data loading and processing classes.

[0035] Training and caching modules: training.py is responsible for the model training and validation process; prepcache.py is used for the preprocessing and cache loading of CT scan data.

[0036] Specific task modules: p2ch13.py encapsulates lung CT segmentation functionality; p2ch14.py encapsulates lung nodule classification functionality.

[0037] Visualization tool module: vis.py serves as a public visualization tool, providing visualization support for CT scans and nodules for segmentation and classification tasks.

[0038] The system's workflow includes: Step S1: Receive raw CT image data from the CT device or USB flash drive through the data input interface on the board. Step S2: The intelligent edge computing module loads and runs the segmentation model (lung.rknn), preprocesses the input CT images and segments the lung tissue, and outputs a mask containing the location of nodules. Step S3: The intelligent edge computing module loads and runs the classification model (lung_cls.rknn), and performs feature extraction and benign / malignant classification on each detected nodule based on the segmentation results of step S2. In step S4, the WEB display module receives the analysis results (including nodule coordinates, malignancy probability, etc.) output by the intelligent edge computing module, and calls the Three.js engine to generate a three-dimensional lung model and heat map. In step S5, the WEB display module starts the local HTTP service, uses the display screen connected via the HDMI interface as the output device, and renders the final interactive diagnostic report webpage.

[0039] Steps S2 and S3 include: Before model inference, the input CT data is preprocessed by normalization, resampling, and other operations to ensure that it meets the model input requirements.

[0040] The NPU of the RK3588 chip is used to perform asynchronous accelerated inference on the segmentation model and the classification model, so as to achieve efficient parallel processing of segmentation and classification tasks.

[0041] Preferably, after step S5, the method further includes: Doctors can use a wireless mouse and keyboard connected to the ELF2 development board to interact with the output web interface, including: rotating and zooming the 3D lung model to observe lesions from multiple angles; clicking on a specific nodule to view its detailed parameters; and adjusting the window width / window level of the CT image to better observe tissue density.

[0042] In step S1, the hardware driver layer is responsible for managing the underlying hardware resources, including parsing CT image data files, allocating and managing memory, and scheduling NPU computing tasks.

[0043] Performing model inference on an embedded development board involves lightweighting and optimizing large deep learning models, thus requiring the use of dedicated toolchains provided by chip manufacturers (such as RKNN-Toolkit2). After model conversion and quantization, the model size and computational load are significantly reduced, enabling operation on resource-constrained embedded devices. For example, a self-built neural network model trained in the PyTorch framework is first exported to ONNX format, then converted and optimized to RKNN format using RKNN-Toolkit2, and finally deployed to the ELF2 development board. Preprocessing refers to the standardization operations performed on the data before it is input into the model. Preprocessing includes, but is not limited to, pixel value normalization, image size unification, and data format conversion. Preprocessing is specified by the edge computing engine and runs in real time on the ELF2 development board.

[0044] The segmentation and classification models are professional medical image analysis models built and trained by this invention. Therefore, the raw CT data does not need to leave the device, and all sensitive information is processed locally. After the ELF2 development board runs the model, it sends the structured diagnostic results (including nodule coordinates, malignancy probability, etc.) to the WEB display module for rendering. Doctors can obtain all diagnostic information through a web interface. Throughout the process, the raw CT data, model parameters, and intermediate calculation results are all stored and processed locally on the device, ensuring the security and privacy of medical data.

[0045] In a preferred embodiment, the lung cancer CT image diagnosis method based on embedded edge computing includes: Insert the USB flash drive containing lung CT image data into the USB port of the ELF2 development board; The system automatically identifies the data and activates the intelligent edge computing module; The segmentation model (lung.rknn) is loaded into the NPU and CPU to perform inference on CT images and complete the segmentation of lungs and nodules; The classification model (lung_cls.rknn) is loaded into the NPU and CPU to perform inference on the segmented nodules and complete the classification of benign and malignant nodules; The web-based display module integrates all analysis results and uses Three.js to generate interactive 3D models and reports. The system outputs the final diagnostic webpage to a medical monitor via an HDMI interface; Doctors use a wireless mouse and keyboard to navigate web pages, access detailed diagnostic information, and formulate diagnostic opinions based on this information.

Claims

1. A lung CT image AI-assisted diagnostic system based on embedded edge computing, characterized in that, The system includes: a development board, a data input interface, an intelligent edge computing module, a visualization output interface, and a web display module; The development board has a built-in NPU and CPU unit to accelerate the inference calculation of the segmentation model and classification model in the intelligent edge computing module. The data input interface is used to receive lung CT image data from a CT scanner or external storage device and transmit it to the intelligent edge computing module; The intelligent edge computing module, deployed on the development board, is used to directly perform lung CT image analysis locally on the device. The intelligent edge computing module includes pre-converted and optimized segmentation and classification models. The aforementioned visualization output interface is used to output the analysis results of the intelligent edge computing module to a display device; The aforementioned WEB display module runs on the development board and is used to generate and provide an interactive web page interface to dynamically present the diagnostic results.

2. The AI-assisted diagnostic system for lung CT images based on embedded edge computing according to claim 1, characterized in that, The segmentation model and classification model are as follows: The segmentation model is a lung tissue and nodule region segmentation network built on a convolutional neural network. It uses an encoder-decoder structure to extract features and fuse spatial information from the input lung CT images. Its input is multi-channel two-dimensional CT image data containing target slices and their context slice information, and the output is a pixel-level probability segmentation map of the lung region and suspicious nodule region, thereby achieving accurate segmentation of lung tissue and preliminary localization of suspicious nodules. The classification model is a nodule benign or malignant discrimination network constructed based on a three-dimensional convolutional neural network. Its input is a fixed-size three-dimensional CT image block cropped around the central region of the suspicious nodule located by the segmentation model. The spatial texture and morphological features of the nodule are extracted through multi-layer three-dimensional convolution and downsampling operations, and the benign or malignant judgment result and malignant probability value of the nodule are output through a fully connected classification layer. The segmentation model and the classification model constitute a dual-model architecture for end-side collaborative reasoning. The segmentation model is used to narrow down the candidate region range and reduce the amount of classification computation, while the classification model is used to perform fine discrimination of candidate nodules, thereby improving the system's diagnostic accuracy and reasoning efficiency. Both the segmentation model and the classification model are executed on the development board's NPU unit after the trained neural network model is converted from ONNX format to RKNN format, and then subjected to lightweight compression and operator optimization to meet the real-time inference requirements of embedded devices.

3. The AI-assisted diagnostic system for lung CT images based on embedded edge computing according to claim 1, characterized in that, The steps of the intelligent edge computing module performing lung CT image analysis include: The segmentation model is used to process the input lung CT images to accurately segment lung tissue and locate suspicious nodule areas. The classification model is used to calculate the segmented suspicious nodules, determine their benign or malignant nature, and output the probability of malignancy.

4. The AI-assisted diagnostic system for lung CT images based on embedded edge computing according to claim 1, characterized in that, The interactive webpage interface provided by the aforementioned WEB display module presents the following content: A 3D lung model generated based on the segmentation results, supporting mouse rotation and zoom; The locations of suspicious nodules are marked in the form of a heatmap on the three-dimensional lung model; Detailed coordinates, benign / malignant assessment results, and malignant probability value for each suspicious nodule; The system generates a structured diagnostic report that includes nodule location and nature assessment.

5. The AI-assisted diagnostic system for lung CT images based on embedded edge computing according to claim 1, characterized in that, The segmentation model is in the lung.rknn format, and the classification model is in the lung_cls.rknn format. The two are obtained by converting the self-built neural network model from the ONNX format to the RKNN format and working together on the development board.

6. A lung cancer CT image AI-assisted diagnosis method based on embedded edge computing, using the system described in any one of claims 1-5, characterized in that, The steps are as follows: Receive lung CT image data through the data input interface; On the development board, its NPU unit is used to call the segmentation model in the intelligent edge computing module to process CT images, complete lung tissue segmentation and nodule localization; The classification model in the intelligent edge computing module is invoked to classify the located nodules as benign or malignant. The web presentation module generates an interactive webpage containing a 3D lung model, nodule heatmap, detailed diagnostic information, and a structured report. The interactive webpage is output to a display device via a visual output interface for doctors to use as a diagnostic reference.

Images