Intelligent tumor recognition system

Abstract

The invention provides an intelligent tumor identification system, which comprises an image classification module for identifying and classifying ultrasonic images, and if no tumor tissue is found, prompting that no tumor tissue exists; if the tumor tissue is found, the ultrasonic image is output; and the image segmentation module is used for segmenting the ultrasonic image output by the image classification module and outputting a tumor contour and a tumor area. According to the invention, accurate identification and boundary positioning of head and neck tumors can be realized, and intraoperative auxiliary identification is completed.

Description

Technical Field

[0001] The present invention relates to a tumor intelligent recognition system. Background Art

[0002] Currently, research on intraoperative intelligent recognition of head and neck tumors mainly relies on traditional imaging techniques and auxiliary diagnosis systems based on optics, CT, or MRI. In clinical practice, doctors often judge the thoroughness of tumor resection through visual experience. However, this method is highly subjective, prone to missed resection or misresection, and it is difficult to ensure real-time performance and accuracy. On the other hand, some research attempts to use equipment based on optical imaging for tissue recognition, but these devices generally require large-sized instruments, high energy consumption, and relatively complex optical systems, and are not suitable for real-time and portable intraoperative operation.

[0003] In recent years, due to advantages such as non-invasiveness, real-time performance, low cost, and good portability, ultrasonic imaging has gradually been introduced into intraoperative navigation and auxiliary decision-making. Some research works use ultrasonic probes combined with image processing algorithms to roughly segment and identify the boundaries of tissue structures. The main structures usually include:

[0004] Signal acquisition module: An ultrasonic transducer is used to collect the reflected signals of the target area, and after front-end amplification, filtering, and analog-to-digital conversion, digital images are generated;

[0005] Image processing module: Based on traditional image processing algorithms, such as edge detection, region segmentation, morphological processing, etc., the ultrasonic images are preprocessed and preliminarily segmented;

[0006] Discrimination and feedback module: Combining preset rules or simple machine learning methods, the segmentation results are judged and fed back to the surgeon through a display.

[0007] At the same time, some research also uses deep learning models (such as CNN or U-Net) to identify and segment tumors in ultrasonic images. These methods have shown certain effects under laboratory conditions, but generally have the following problems and disadvantages in practical applications:

[0008] Image noise and low contrast problems: Ultrasonic images themselves have obvious speckle noise and low contrast problems. Traditional algorithms and some deep learning models are prone to misjudgment or missed judgment in complex intraoperative environments, and it is difficult to accurately distinguish tumor tissue from normal tissue.

[0009] Insufficient real-time performance: Many existing systems are difficult to balance processing speed and algorithm complexity, cannot meet the requirements of intraoperative real-time feedback, resulting in large delays, thus affecting surgical decisions.

[0010] Problems of device volume and adaptability: Most existing ultrasound systems are large-scale instruments, lacking portability and flexibility. Moreover, the co-design of their hardware and embedded algorithms is not tight enough, making it difficult to operate stably in complex intraoperative environments.

[0011] Limited intelligence level: Currently, most of the adopted recognition methods are based on rules or shallow machine learning, lacking the extraction of deep semantic information of tissue structures, unable to achieve high-precision localization and quantitative evaluation of tumor boundaries, and difficult to meet the clinical requirements for intelligent auxiliary decision-making.

[0012] The root causes of these problems are as follows: Ultrasonic imaging technology is limited by physical imaging principles and noise characteristics, and traditional image processing algorithms are difficult to adapt to the heterogeneity of complex biological tissues; while most existing deep learning methods rely on a large amount of labeled data, and the models are generally large and have high computational complexity, making it difficult to achieve efficient operation on resource-constrained intraoperative embedded platforms.

[0013] Therefore, how to provide an intelligent and portable tumor intelligent recognition system that can accurately identify and localize the boundaries of head and neck tumors and complete intraoperative ultrasound-assisted recognition has become a problem that the industry needs to solve. Summary of the Invention

[0014] Aiming at the shortcomings of the existing technology, the purpose of the present invention is to provide a tumor intelligent recognition system, which breaks through the bottlenecks such as image noise processing, insufficient real-time performance, and poor device adaptability in the existing technology, realizes accurate recognition and boundary localization of head and neck tumors, thereby providing reliable and scientific technical support for intraoperative decision-making, and ultimately improving the surgical success rate and the quality of life of patients.

[0015] To achieve the above purpose, the present invention provides a tumor intelligent recognition system, which includes:

[0016] An image classification module, which recognizes and classifies ultrasonic images. If no tumor tissue is found, it prompts that there is no tumor tissue; if tumor tissue is found, it outputs the ultrasonic image.

[0017] An image segmentation module, which segments the ultrasonic image output by the image classification module and outputs the tumor contour and tumor area.

[0018] In the present invention, the image classification module adopts MobileNetV2 as the classification model. In the present invention, the image segmentation module adopts Mobile-UNet as the segmentation model, combines a lightweight MobileNetV2 encoder with a classic U-Net decoder, and fuses multi-scale information through skip connections, ensuring both segmentation accuracy and meeting real-time requirements.

[0019] In the present invention, the classification steps of the image classification module include:

[0020] A1: Data preprocessing and augmentation;

[0021] Data preprocessing: Normalize the intraoperative ultrasound images (e.g., scale the pixel values to [0, 1]) and resize them to a fixed size (such as 224×224);

[0022] Data augmentation: Use methods such as rotation, translation, horizontal flipping, brightness / contrast adjustment to increase sample diversity, alleviate the problem of insufficient data, and improve the generalization ability of the model.

[0023] A2: Fine-tuning of the model structure;

[0024] Initial layer freezing: The MobileNetV2 model is loaded with pre-training on ImageNet, and the front feature extraction layers are frozen (e.g., freeze the first 70% of the parameters), and only the last few layers are kept trainable;

[0025] Classification head design: After the original global average pooling layer, connect a new fully connected layer with 2 output nodes for binary classification; the activation function is Softmax.

[0026] Through fine-tuning, freezing some pre-trained weights can prevent overfitting in the case of small samples, and at the same time, the newly designed classification head can be optimized for the features of ultrasound images.

[0027] A3: Determine the fine-tuning strategy and hyperparameters;

[0028] Initial training: Only train the newly added classification head, use a relatively high learning rate (set to 1e-3), and train for 200 epochs to make the classification head initially adapt to the data;

[0029] Joint training: Unfreeze some of the frozen layers, use a lower learning rate (set to 1e-4), and perform full-network joint training to finely adjust the pre-trained weights;

[0030] Optimizer and loss function: Use the Adam optimizer with a weight decay of 1e-4; the loss function uses cross-entropy loss, and at the same time, metrics such as accuracy and recall can be monitored.

[0031] The relatively high learning rate in the initial stage helps the new layer to converge quickly, and the subsequent joint training can further improve the classification accuracy while maintaining the original feature extraction ability.

[0032] In the present invention, the segmentation steps of the image segmentation module include:

[0033] B1: Data preprocessing and augmentation;

[0034] Data preprocessing: Unify the size of the ultrasound images (256×256) and perform normalization processing;

[0035] Data augmentation: Methods such as rotation, scaling, and flipping are used to amplify training samples, increasing the robustness to different lesion morphologies.

[0036] B2: Fine-tuning of the model structure;

[0037] Encoder: The pre-trained MobileNetV2 model is used as the encoder, and the last fully connected layer is removed;

[0038] Decoder: A series of upsampling modules are designed. Each module consists of a transposed convolution or bilinear upsampling, concatenating the features of the corresponding encoder layer (skip connection), and a convolution module (3×3 convolution + BN + ReLU);

[0039] Boundary enhancement: A lightweight attention module or edge extraction module is added to the decoder to help the model focus on the boundary information between the tumor and normal tissues.

[0040] The pre-trained weights of the encoder can provide stable semantic features, while skip connections and attention mechanisms contribute to detail recovery and edge sharpening.

[0041] B3: Determine the fine-tuning strategy and hyperparameters;

[0042] Decoder pre-training: Freeze the encoder and only train the decoder part; Use a learning rate of 1e-3 and train for 200 epochs to make the newly added decoder weights converge initially;

[0043] Full-network joint fine-tuning: Unfreeze some encoder layers and jointly train the entire network; Use a lower learning rate (1e-4) and train for 300 epochs;

[0044] Optimizer and loss function: Select the Adam optimizer with a weight decay set to 1e-4; The loss function uses the weighted sum of Dice loss and Binary Cross-Entropy loss, and the formula is: L = αDiceLoss + βBCE, where α = 0.5 and β = 0.5 can be set; At the same time, monitor the segmentation metrics (Dice coefficient and intersection over union IoU).

[0045] Through staged training, the pre-trained knowledge can be fully utilized and adapted to the new domain; The combined use of Dice loss and cross-entropy loss can take into account both the regional overlap rate and pixel-level classification accuracy, thereby improving the segmentation effect.

[0046] In the present invention, the tumor intelligent recognition system further includes an image acquisition device, and the image acquisition device includes a micro ultrasonic probe and a signal processing module.

[0047] The micro ultrasonic probe is a high-frequency, linear array micro ultrasonic probe. Its miniaturized and high-resolution design makes it suitable for real-time scanning of subcutaneous tissues during surgery, ensuring the acquisition of high-quality ultrasonic images within a limited surgical space. The micro ultrasonic probe adopts a dedicated packaging design to meet the requirements of medical disinfection, ensuring its stability and safety in a complex surgical environment.

[0048] Signal processing module: Through a low-noise amplifier, precision filtering, and high-speed analog-to-digital conversion, it effectively captures and real-time digitizes ultrasonic signals, completes the data preprocessing of ultrasonic images, provides an accurate data basis for subsequent image processing, and ensures the high-quality acquisition and noise reduction effect of intraoperative ultrasonic images.

[0049] In this solution, the parameters of the micro ultrasonic probe are as follows:

[0050] 1. Operating frequency: 15 - 25 MHz tunable wideband;

[0051] 2. Number of array elements: 256-channel micro matrix array;

[0052] 3. Scanning mode: Three-dimensional volume scanning (30 frames per second);

[0053] 4. Penetration depth: 0.5 - 4 cm (dynamically adjustable);

[0054] 5. Resolution: Axial 50 μm / Lateral 120 μm;

[0055] 6. Probe size: Φ3.5 mm × 25 mm (can rotate 360°);

[0056] 7. Material packaging: Medical-grade titanium alloy shell / IPX7 waterproof.

[0057] In the present invention, the tumor intelligent recognition system further includes a control device, which can display the recognition results output by the system in the form of images and numerical values in real time, assist surgeons in intraoperative judgment, and provide a user interaction interface for parameter adjustment and operation recording.

[0058] In this solution, the control device uses a high-resolution touch display screen, supports high-definition image display and dynamic annotation; adopts an intuitively designed graphical user interface (GUI) to display the original ultrasonic image, classification result, segmentation boundary, tumor area, and other diagnostic information; supports touch, voice commands, or gesture operations, facilitating doctors to quickly retrieve and confirm information during surgery.

[0059] In the present invention, the tumor intelligent recognition system further includes a storage and communication device, which can realize local storage and backup of intraoperative data; at the same time, it supports wireless data transmission to realize remote consultation, expert consultation, and data update.

[0060] In the present invention, the storage and communication device includes a storage unit, a communication interface, and a remote platform.

[0061] In this solution, the storage unit incorporates a high-speed storage device (such as an SSD or eMMC) for storing ultrasonic image data and processing results; the communication interface supports Wi-Fi, Bluetooth, and wired Ethernet to ensure high-speed data transmission inside and outside the operating room; the remote platform is equipped with a dedicated cloud platform interface to achieve data synchronization, remote monitoring, and model update feedback, providing technical support for multi-center collaboration.

[0062] In the present invention, the tumor intelligent recognition system is set on an embedded intelligent processing platform; the embedded intelligent processing platform integrates lightweight deep learning models (fine-tuned MobileNetV2 for classification and Mobile-UNet for segmentation), adopts an edge computing mode, is responsible for real-time preprocessing of the collected digital image data, calls the pre-trained artificial intelligence models for classification and segmentation recognition, judges in real time whether there is a tumor under the skin, and accurately extracts the tumor boundary and area information; realizes intelligent recognition and precise tumor boundary positioning to ensure rapid response during the operation.

[0063] In this solution, the specific configuration of the embedded intelligent processing platform includes:

[0064] 1. Hardware platform: Adopt a low-power and high-performance embedded computing platform, such as the NVIDIA Jetson Nano / AGX Xavier series, or an edge AI chip based on the ARM Cortex-A series, supporting GPU / TPU acceleration;

[0065] 2. Software environment: Based on the Linux / RTOS operating system, integrate lightweight neural network inference engines such as TensorFlow Lite or ONNX Runtime to achieve efficient real-time AI model inference.

[0066] The present invention solves the technical problems of poor accuracy and insufficient real-time performance in intraoperative tumor recognition, being real-time, precise, intelligent, and portable.

[0067] The present invention utilizes the transposition of a micro ultrasonic probe to operate on the subcutaneous tissue during the operation to obtain the ultrasonic image data of this part, and uses the trained artificial intelligence model for intelligent recognition, so as to reflect whether there is tumor tissue under the skin of this part. If there is, the boundary is recognized and visually displayed.

[0068] The present invention adopts an integrated algorithm, combines a deep model with a level set edge refinement method, realizes end-to-end system integration, closely integrates functions such as ultrasonic image acquisition, data processing, AI recognition, result visualization display, and data transmission, ensures their collaborative work and guarantees real-time data transmission, and constructs a tumor intelligent recognition system with good portability, high real-time performance, and simple operation suitable for the intraoperative environment.

[0069] Compared with the prior art, the present invention has the following beneficial effects:

[0070] Real-time and accurate identification of tumor tissues: By using a micro ultrasonic probe to collect real-time ultrasonic images of subcutaneous tissues during surgery and combining a deep learning model that has been lightweight optimized and fine-tuned, intelligent classification and segmentation of the images are carried out to achieve accurate identification and boundary localization of tumor tissues, thus avoiding subjective errors brought by traditional visual judgment.

[0071] Boundary recognition and visual feedback: After identifying the tumor area, the system can automatically extract the tumor boundary, calculate key information such as its area, and feedback it to the surgeon through real-time visualization means to assist in judging the resection range, reduce the risk of missed resection, and reduce damage to normal tissues.

[0072] Portable and real-time application: The present invention designs a set of micro ultrasonic devices and supporting embedded intelligent processing systems, fully considering the portability and low power consumption requirements for intraoperative use, enabling the device to not only meet high-precision image acquisition but also achieve high-speed intelligent processing on an embedded platform, ensuring the stability and real-time performance of the entire system during actual surgery. Specific implementation manner

[0073] Example 1

[0074] This embodiment provides a tumor intelligent recognition system, which includes an image acquisition device, an image classification module, an image segmentation module, a control device, and a storage and communication device.

[0075] The image acquisition device includes a micro ultrasonic probe and a signal processing module; the micro ultrasonic probe is a high-frequency, linear array micro ultrasonic probe, and its parameters are as follows: 1. Operating frequency: 15 - 25 MHz tunable broadband; 2. Number of array elements: 256-channel micro matrix array; 3. Scanning mode: three-dimensional volume scanning (30 frames per second); 4. Penetration depth: 0.5 - 4 cm (dynamically adjustable); 5. Resolution: 50 μm axially / 120 μm laterally; 6. Probe size: Φ3.5 mm × 25 mm (can rotate 360°); 7. Material encapsulation: medical-grade titanium alloy shell / IPX7 waterproof.

[0076] The image classification module adopts MobileNetV2 as the classification model, and the classification steps of the image classification module include:

[0077] A1: Data preprocessing and augmentation;

[0078] Data preprocessing: Normalize the intraoperative ultrasound images (scale pixel values to [0, 1]) and resize them to a fixed size (224×224);

[0079] Data augmentation: Use methods such as rotation, translation, horizontal flipping, and brightness / contrast adjustment to increase sample diversity.

[0080] A2: Fine-tuning of the model structure;

[0081] Initial layer freezing: The MobileNetV2 model is loaded with pre-training on ImageNet, and the front feature extraction layers are frozen (the first 70% of the parameters are frozen), and only the last few layers are kept trainable;

[0082] Classification head design: After the original global average pooling layer, a new fully connected layer is connected, with 2 nodes output for binary classification; the activation function uses Softmax.

[0083] A3: Determine the fine-tuning strategy and hyperparameters;

[0084] Initial training: Only train the newly added classification head, use a relatively high learning rate (set to 1e-3), and train for 200 epochs to make the classification head initially adapt to the data;

[0085] Joint training: Unfreeze some of the frozen layers, use a relatively low learning rate (set to 1e-4), and perform full-network joint training to finely adjust the pre-trained weights;

[0086] Optimizer and loss function: Use the Adam optimizer, and set the weight decay to 1e-4; the loss function uses cross-entropy loss, and at the same time, metrics such as accuracy and recall can be monitored.

[0087] The image segmentation module uses Mobile-UNet as the segmentation model, combining a lightweight MobileNetV2 encoder and a classic U-Net decoder; the segmentation steps of the image segmentation module include:

[0088] B1: Data preprocessing and augmentation;

[0089] Data preprocessing: Unify the size (256×256) of the ultrasound images and perform normalization processing;

[0090] Data augmentation: Use methods such as rotation, scaling, and flipping to augment the training samples.

[0091] B2: Fine-tuning of the model structure;

[0092] Encoder: The pre-trained MobileNetV2 model is used as the encoder, and the last fully connected layer is removed;

[0093] Decoder: A series of upsampling modules are designed. Each module consists of a transposed convolution or bilinear upsampling, concatenating the corresponding encoder layer features (skip connection), and a convolution module (3×3 convolution + BN + ReLU);

[0094] Boundary enhancement: A lightweight attention module or edge extraction module is added to the decoder to help the model focus on the boundary information between the tumor and normal tissues.

[0095] B3: Determine the fine-tuning strategy and hyperparameters;

[0096] Decoder pre-training: Freeze the encoder and only train the decoder part; Use a learning rate of 1e-3 and train for 200 epochs to make the newly added decoder weights converge initially;

[0097] Full network joint fine-tuning: Unfreeze some encoder layers and jointly train the entire network; Use a lower learning rate (1e-4) and train for 300 epochs;

[0098] Optimizer and loss function: Select the Adam optimizer and set the weight decay to 1e-4; The loss function uses the weighted sum of the Dice loss and Binary Cross-Entropy loss, and the formula is: L = αDiceLoss + βBCE, where α = 0.5 and β = 0.5 are set; At the same time, monitor the segmentation metrics (Dice coefficient and intersection over union IoU).

[0099] The control device uses a high-resolution touch display screen and an intuitively designed graphical user interface (GUI), supporting touch, voice commands, or gesture operations.

[0100] The storage and communication device includes a storage unit, a communication interface, and a remote platform; The storage unit has a built-in high-speed storage device (SSD), the communication interface supports Wi-Fi, Bluetooth, and wired Ethernet, and the remote platform is equipped with a dedicated cloud platform interface.

[0101] The tumor intelligent recognition system is set on an embedded intelligent processing platform. The specific configuration of the embedded intelligent processing platform includes: NVIDIA Jetson AGX Xavier series chips, Linux operating system, and an integrated TensorFlow Lite inference engine.

[0102] Example 2

[0103] The difference between this example and Example 1 is that the storage device is eMMC.

[0104] Example 3

[0105] The difference between this embodiment and Embodiment 1 is that: the chip adopts the ARM Cortex-A series.

[0106] Embodiment 4

[0107] The difference between this embodiment and Embodiment 1 is that: the inference engine adopts ONNX Runtime.

[0108] Embodiment 5

[0109] The difference between this embodiment and Embodiment 1 is that: the operating system is RTOS.

[0110] Although the present invention is disclosed above with preferred embodiments, it is not intended to limit the scope of the present invention. Any person skilled in the art can make some improvements without departing from the scope of the present invention. That is, any equivalent improvements made in accordance with the present invention should be covered by the scope of the present invention.

Claims

1. An intelligent tumor recognition system, wherein, The tumor intelligent recognition system includes: An image classification module, which recognizes and classifies ultrasonic images. If no tumor tissue is found, it prompts that there is no tumor tissue; if tumor tissue is found, it outputs the ultrasonic image. An image segmentation module, which segments the ultrasonic image output by the image classification module and outputs the tumor contour and tumor area.

2. The tumor intelligent recognition system according to claim 1, wherein, The image classification module uses MobileNetV2 as the classification model.

3. The tumor intelligent recognition system according to claim 2, wherein The image segmentation module uses Mobile-UNet as the segmentation model.

4. The tumor intelligent recognition system according to claim 3, wherein, The classification steps of the image classification module include: A1: Data preprocessing and enhancement; A2: Fine-tuning of the model structure; A3: Determination of the fine-tuning strategy and hyperparameters.

5. The tumor intelligent recognition system according to claim 4, wherein, The segmentation steps of the image segmentation module include: B1: Data preprocessing and enhancement; B2: Fine-tuning of the model structure; B3: Determination of the fine-tuning strategy and hyperparameters.

6. The tumor intelligent recognition system according to claim 5, wherein, The tumor intelligent recognition system further includes an image acquisition device, which includes a micro ultrasonic probe and a signal processing module.

7. The tumor intelligent recognition system according to claim 6, wherein, The tumor intelligent recognition system further includes a control device.

8. The tumor intelligent recognition system according to claim 7, wherein, The tumor intelligent recognition system further includes a storage and communication device.

9. The tumor intelligent recognition system according to claim 8, wherein, The storage and communication device includes a storage unit, a communication interface, and a remote platform.

10. The tumor intelligent recognition system according to claim 9, wherein, The tumor intelligent recognition system is set on an embedded intelligent processing platform.