A large model-based localization and description system for osteosarcoma
By applying large-model technology in osteosarcoma image processing, the localization segmentation and description of osteosarcoma are achieved, the problem of lack of detailed description in the prior art is solved, and the accuracy and efficiency of diagnosis are improved.
Patent Information
- Application Number
- CN202411615195.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-13
- Publication Date
- 2025-04-08
AI Technical Summary
The prior art focuses on detection and segmentation in osteosarcoma image processing, and the lack of detailed description of osteosarcoma, which makes preoperative diagnosis challenging.
A large-model-based positioning and description system is adopted, including a coding module that uses the mobileSAM model to extract image features, the decoding module performs supervised decoding training to achieve tumor segmentation, and the LSTM network generates descriptive statements to describe the tumor.
The localization and segmentation and detailed description of osteosarcoma are achieved, which reduces the workload of doctors and improves the accuracy of preoperative diagnosis.
Smart Images
Figure CN119784832A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a tumor positioning segmentation and description system thereof, belonging to the technical field of tumor image processing. Background Art
[0002] With the development of artificial intelligence big model technology, the segmentation of big models in medical images has made certain research progress in academia and has been applied to a certain extent in reality, such as segmentation SAM and detection. This patent mainly applies the image big model technology to the segmentation and description of osteosarcoma. Osteosarcoma is a relatively common primary bone tumor with a high degree of malignancy. It is a malignant tumor of mesenchymal tissue, including traditional osteosarcoma, small cell osteosarcoma, well-differentiated osteosarcoma, and metastatic osteosarcoma. In view of the rarity of malignant osteosarcoma, due to the overlap of the age and location of different lesions, and the hidden and diverse clinical symptoms and signs, which are rarely specific, the phenomenon of "same disease, different images, different diseases, same images" is common, which makes preoperative diagnosis challenging. Therefore, the use of big models to accurately locate and describe is a great help to clinicians.
[0003] At present, the processing of osteosarcoma images (CT, MRI, X-ray) on the market mainly stays at the detection and identification of osteosarcoma: osteosarcoma target detection YOLO (positioning and classification), and osteosarcoma segmentation: divided into traditional convolutional neural network segmentation UNet series, but all current research networks lack the description of osteosarcoma, such as irregular shape of a certain location, rough surface, approximate area, what kind of osteosarcoma it is and to what extent it has developed, etc. Summary of the invention
[0004] The purpose of the present invention is to apply the large model technology to the segmentation and description of osteosarcoma.
[0005] In order to achieve the above-mentioned object, the technical solution of the present invention is to disclose a system for positioning and describing osteosarcoma based on a large model, which is characterized by comprising:
[0006] The encoding module directly uses the encoding network of the mobileSAM model to obtain universal encoding features based on the input tumor image;
[0007] The decoding module obtains the segmented mask based on the universal coding features obtained by the encoding module, thereby realizing the segmentation and positioning of the tumor;
[0008] LSTM network, using the image caption description of the LSTM network, obtains descriptive sentences based on the general coding features obtained by the encoding module, thereby realizing the description of the tumor.
[0009] Preferably, the decoding module adopts an upsampling decoder.
[0010] Preferably, when training the decoding module, supervised decoding training is adopted.
[0011] The present invention can solve the semantic segmentation of the target level (position, size), and can describe the segmented tumor (symptoms). Therefore, the technical solution disclosed by the present invention can not only realize the positioning segmentation of osteosarcoma, but also describe the tumor to a certain extent, thereby further relieving the workload of doctors. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] Figure 1 is the model network structure of the present invention;
[0013] Figure 2 It is a flow chart of the segmentation and recognition method;
[0014] Figure 3 This is the training process of the present invention. DETAILED DESCRIPTION
[0015] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.
[0016] like Figure 1 As shown, an embodiment of the present invention discloses a system for positioning and describing osteosarcoma based on a large model, which processes osteosarcoma images step by step based on a SAM large model network (combining image segmentation and image description in one network), including:
[0017] The encoding module is directly implemented using the encoding stage of mobileSAM to obtain better universal encoding features based on the input image. In the embodiment of the present invention, the encoding module mainly uses the lightweight encoding network stage of the large model mobileSAM to extract features to adapt to the use on the CPU. MobileSam is a general-level segmentation network, and the features extracted in the encoding stage have certain universality, so the embodiment of the present invention directly uses the network to extract universal features of tumor images (the detailed network structure can be directly obtained online).
[0018] After obtaining good general encoding features through the encoding module, the decoding module is set up for supervised learning to learn segmentation labels. The decoding stage is different from the prompt of sam. The main problem here is the segmentation recognition at the target level, so supervised decoding training is required, and the upsampling network is used to convert the features to mask size training.
[0019] After the LSTM network obtains good general coding features through the encoding module, the imagecaption image description based on LSTM is used to generate a description of the tumor.
[0020] Such a complete object-level lightweight semantic segmentation and image description network based on MobileSam is completed.
Claims
1. A system for positioning and describing osteosarcoma based on a large model, characterized in that: include: The encoding module directly uses the encoding network of the mobileSAM model to obtain universal encoding features based on the input tumor image; The decoding module obtains the segmented mask based on the universal coding features obtained by the encoding module, thereby realizing the segmentation and positioning of the tumor; LSTM network, using the image caption description of the LSTM network, obtains descriptive sentences based on the general coding features obtained by the encoding module, thereby realizing the description of the tumor.
2. A system for positioning and describing osteosarcoma based on a large model as claimed in claim 1, characterized in that: The decoding module adopts an up-sampling decoder.
3. The system for locating and describing osteosarcoma based on a large model as claimed in claim 1, characterized in that: When training the decoding module, supervised decoding training is adopted.