Interactive intelligent plasticizing cross-sectional image feature extraction method and system

Abstract

The invention discloses an interactive intelligent plasticized cross-sectional image feature extraction method and system, and belongs to the technical field of biomedical engineering, and the method comprises the steps: constructing a plasticized slice semantic segmentation data set, training a deep learning convolutional neural network to obtain a bone image region segmentation network, and carrying out the segmentation and enhancement processing of a plasticized cross-sectional image. And performing multi-dimensional feature extraction through the feature extraction interface to obtain image features. The method can automatically and accurately extract key features from the plasticized cross-sectional image, has the characteristics of convenience in operation, accuracy and high efficiency, can be applied to the fields of medical image analysis and the like, and provides powerful support for related research and diagnosis.

Description

Technical Field

[0001] The present invention relates to the field of biomedical engineering technology, and more particularly to an interactive intelligent plasticization tomographic image feature extraction method and system. Background Art

[0002] In the human body structure, bones and fascia or fibrous soft tissues are interconnected to form an organic whole. Bones can be divided into cortical bones and cancellous bones, and the mechanical properties of bones are closely related to the porosity and internal structure of cancellous bones. A full understanding of the architectural characteristics and distribution of trabeculae in cancellous bones (including trabecular direction, pore gradient distribution, degree of anisotropy and quantity) is crucial to accurately knowing the mechanical properties of bones. At the same time, the spatial course, distribution and connection of fascia or fibrous structures with surrounding tissues are of great help in accurately grasping the anatomical mechanical properties of the target structure. Furthermore, bones and soft tissues constitute a histological transition zone, and a systematic understanding of the special mechanical properties of this area is also an essential factor for understanding the physiological activities of the human body.

[0003] In order to systematically obtain accurate mechanical information of bone and fascia or fibrous structures, it is necessary to establish an anatomically localized, two-way connection between bone tissue and soft tissue. This correlation is specifically reflected in three aspects: first, the precise distribution of bone tissue and soft tissue over a wide range of viewing angles; second, the true coupling characteristics of the gradient changes of trabeculae and the orientation of soft tissue fibers; and third, the integrated relationship between the bone structure and the soft tissue interface. This undoubtedly puts forward technical requirements for the observation of bone and soft tissue structures in three dimensions: spatial resolution (large-scale high-resolution imaging), density resolution (ligament fiber orientation), and soft tissue contrast (identification of the bone and soft tissue interface).

[0004] However, existing multimodal imaging technologies have significant flaws. Although CT can observe trabecular bone structure over a large area, the image clarity is insufficient; Micro-CT can present high-definition trabecular bone structure, but is limited in size and cannot extract trabecular bone information over a large area. In addition, both technologies have obvious limitations in their effectiveness for soft tissue imaging. Although MRI can clearly display the boundaries of soft tissue, its display of the fine structure of trabecular bone is still limited. Histological technology is too microscopic, making it difficult for researchers to observe local structures from a global perspective, resulting in certain limitations in the information obtained. The lack of these multimodal imaging technologies has prevented the mechanical information of bones and soft tissues from being systematically integrated into a knowledge system, seriously restricting people's understanding of human physiological activities.

[0005] At present, the plasticization tomography technology made by embedding polymer materials, such as P45 biological plastination technology, has been widely used to solve clinical diseases caused by unclear display of bone, fascia or fibrous structures because of its advantage of large-scale in situ tomography to observe fine structures. However, it cannot be ignored that this tomography technology still has two technical limitations. On the one hand, it is not yet possible to automatically distinguish the structural characteristics of bone or soft tissue. Due to the reliance on manual intervention, the selection accuracy is limited. In the process of selecting the target area, it is inevitable that soft tissue structures will be mistakenly included in the bone tissue, resulting in unnecessary errors. On the other hand, it is not yet possible to automatically provide quantitative information of bone, fascia or fibrous structures, which greatly limits the further application of this tomography technology.

[0006] Therefore, how to provide a plasticized tomographic image feature extraction method and system that can accurately extract and quantitatively analyze bone and soft tissue related structural features is an urgent problem that needs to be solved by those skilled in the art. Summary of the Invention

[0007] In view of this, the present invention provides an interactive intelligent plasticized tomographic image feature extraction method and system. By constructing a plasticized slice semantic segmentation dataset, training a deep learning convolutional neural network to perform bone image region segmentation, and enhancing the segmentation results, the image features of the plasticized tomographic image are extracted, realizing the full process processing from image acquisition to feature extraction, and improving the efficiency and accuracy of plasticized tomographic image analysis.

[0008] In order to achieve the above object, the present invention adopts the following technical solutions:

[0009] In one aspect, the present invention provides an interactive intelligent plasticization tomographic image feature extraction method, comprising:

[0010] Collect multiple plasticized tomographic images and construct a plasticized slice semantic segmentation dataset;

[0011] Constructing a deep learning convolutional neural network, and using the plasticized slice semantic segmentation dataset to train the deep learning convolutional neural network to obtain a trained bone image region segmentation network;

[0012] The trained bone image region segmentation network is used to perform region segmentation on the collected plasticized tomographic images to obtain the segmentation results;

[0013] performing enhancement processing on the segmentation result to obtain an enhanced image;

[0014] Feature extraction is performed on the enhanced image to obtain image features of the plasticized tomographic image.

[0015] Preferably, the deep learning convolutional neural network is trained using the plasticized slice semantic segmentation dataset to obtain a trained bone image region segmentation network, comprising:

[0016] Using the plasticized slice semantic segmentation dataset to train the deep learning convolutional neural network to obtain a trained model;

[0017] Evaluating the trained model output and the real plasticized slice image using a mean square error loss function;

[0018] During the training process, a learning rate decay strategy is adopted, and the trained model is used as the deep learning convolutional neural network model for the next round of training for multiple trainings until the evaluation results do not improve for multiple consecutive times, and the trained bone image region segmentation network is output.

[0019] Preferably, performing enhancement processing on the segmentation result to obtain an enhanced image includes:

[0020] Converting the segmentation result into a grayscale image;

[0021] Performing local histogram equalization on the grayscale image to obtain an enhanced image.

[0022] Preferably, after obtaining the trained bone image region segmentation network, the method further includes:

[0023] The trained bone image region segmentation network is used to perform region segmentation on the collected plasticized tomographic images to obtain the segmentation results;

[0024] performing enhancement processing on the segmentation result to obtain an enhanced image;

[0025] converting the enhanced image into a CT image;

[0026] Obtain CT images of healthy human bones;

[0027] Converting the CT image into a solid through a solid operation;

[0028] Performing meshing processing on the entity and assigning material properties;

[0029] performing static mechanics analysis and gait mechanics analysis on the entity respectively;

[0030] Comparing and analyzing the static mechanics analysis results and the gait mechanics analysis results respectively, and obtaining a similarity value between the enhanced image and a healthy human bone;

[0031] The feasibility of the trained bone image region segmentation network is determined based on the similarity.

[0032] Preferably, performing feature extraction on the enhanced image to obtain image features of the plasticized tomographic image includes:

[0033] Using the DoG operator to enhance the detail features of the enhanced image to obtain a detail enhanced image;

[0034] The detail enhanced image is converted into a binary image through Otsu threshold segmentation;

[0035] An interactive feature extraction interface is constructed to perform feature extraction on the binary image to obtain image features of the plasticized tomographic image.

[0036] Preferably, performing feature extraction on the binary image to obtain image features of the plasticized tomographic image includes:

[0037] Selecting a target area in the binary image and determining the coordinates and size of the target area;

[0038] Calculate the area of ​​the target area according to the coordinates and size;

[0039] determining the porosity of the trabecular bone according to the ratio of background pixels in the target area;

[0040] calculating the structural direction entropy of the target area, and obtaining anisotropy based on the structural direction entropy;

[0041] Using a contour retrieval method to obtain the outermost contour of the foreground area within the target area;

[0042] compressing the outermost contour using a contour approximation algorithm, and using the number of compressed contour lists as the number of trabeculae in the target area;

[0043] The Hessian matrix and the eigenvector of the target area are calculated, and the trabecular direction information is obtained based on the Hessian matrix and the eigenvector.

[0044] In another aspect, the present invention provides an interactive intelligent plasticization tomographic image feature extraction system, wherein the system includes the above-mentioned interactive intelligent plasticization tomographic image feature extraction method, comprising:

[0045] The dataset construction module is used to collect multiple plasticized tomographic images and construct a plasticized slice semantic segmentation dataset;

[0046] A model construction module is used to construct a deep learning convolutional neural network and train the deep learning convolutional neural network using the plasticized slice semantic segmentation dataset to obtain a trained bone image region segmentation network;

[0047] A segmentation module is used to perform regional segmentation on the collected plasticized tomographic image using a trained bone image region segmentation network to obtain a segmentation result;

[0048] The feature extraction module is used to perform enhancement processing on the segmentation result to obtain an enhanced image, and perform feature extraction on the enhanced image to obtain image features of the plasticized tomographic image.

[0049] Preferably, the system further comprises:

[0050] The verification module is used to verify the feasibility of the trained bone image region segmentation network.

[0051] Through the above technical solutions, it can be seen that compared with the prior art, the present invention discloses an interactive intelligent plasticized tomographic image feature extraction method and system. The present invention integrates data set construction, model training, image segmentation, enhancement processing and feature extraction, realizes automated processing, and improves efficiency and accuracy. In addition, by converting the enhanced image into a CT image and comparing it with the bone CT image of a healthy person, the feasibility of the model is verified, providing a basis for model optimization, ensuring that the system is reliable and effective, and meeting the needs of practical applications. Furthermore, the present invention accurately identifies the bone area and soft tissue area images in the plasticized tomographic image through a deep learning algorithm, improves the efficiency and accuracy of image recognition, and provides a convenient method for analyzing the potential mechanical properties of the target area. Furthermore, the present invention builds an interactive interface for parameter extraction, which can personalize the identification of relevant features of trabeculae and soft tissues in any area of ​​interest, and intelligently extract the structural information of bones and soft tissues. While ensuring that the information is more comprehensive, personalized calculations can be performed for different research topics to provide systematic information for understanding the physiological activities of the human body. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0053] Figure 1 A schematic diagram of the process provided by the present invention.

[0054] Figure 2 This is the recognition process of plasticized tomographic slice images.

[0055] Figure 3 is a thumbnail.

[0056] Figure 4 A diagram showing the general size and direction information of the target area.

[0057] Figure 5 This is a structural diagram provided by the present invention. DETAILED DESCRIPTION

[0058] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0059] The embodiment of the present invention discloses an interactive intelligent plasticization tomographic image feature extraction method, such as Figure 1 As shown, including:

[0060] Multiple plastinated tomographic images were collected to construct a plastinated slice semantic segmentation dataset. Specifically, a semantic segmentation toolkit was used to manually segment the bone regions of a large number of plastinated slice images in programmable software to build a plastinated slice semantic segmentation dataset. The dataset was then divided into a training set and a test set in a ratio of 9:1 to serve as training data for deep learning.

[0061] Construct a deep learning convolutional neural network and train it using a plastinated slice semantic segmentation dataset to obtain a trained bone image region segmentation network.

[0062] The trained bone image region segmentation network is used to perform region segmentation on the collected plasticized tomographic images to obtain the segmentation results;

[0063] Perform enhancement processing on the segmentation result to obtain an enhanced image;

[0064] Feature extraction is performed on the enhanced image to obtain the image features of the plasticized tomographic image.

[0065] Furthermore, the deep learning convolutional neural network was trained using the plastinated slice semantic segmentation dataset to obtain a trained bone image region segmentation network, including:

[0066] The deep learning convolutional neural network was trained using the plastinated slice semantic segmentation dataset to obtain a trained model;

[0067] The trained model output was evaluated against the real plasticized slice images using the mean square error loss function;

[0068] During the training process, a learning rate decay strategy is adopted, and the trained model is used as the deep learning convolutional neural network model for the next round of training for multiple trainings until the evaluation results have not improved for multiple consecutive times. The trained bone image region segmentation network is output. The output image of the trained bone image region segmentation network is as follows: Figure 2 shown.

[0069] Preferably, performing enhancement processing on the segmentation result to obtain an enhanced image includes:

[0070] Convert the segmentation result into a grayscale image;

[0071] Local histogram equalization is performed on the grayscale image to enhance the global features and bone contour features of the bone image and obtain an enhanced image.

[0072] In another embodiment, after obtaining the trained bone image region segmentation network, the method further includes:

[0073] The trained bone image region segmentation network is used to perform region segmentation on the collected plasticized tomographic images to obtain the segmentation results;

[0074] Perform enhancement processing on the segmentation result to obtain an enhanced image;

[0075] The enhanced image is converted into a CT image; specifically, the local histogram equalization result image is converted into a CT image by programming in programmable software.

[0076] Obtain a CT image of a healthy human bone, extract a cross section close to the converted single-layer CT image using medical imaging software, and erase the remaining cross sections.

[0077] The CT images were converted into entities through entity manipulation; in medical image processing software, both were converted into entities through entity manipulation. In mechanical analysis software, material properties were assigned to the entities of the two CT images. Static mechanical analysis and gait mechanical analysis were performed on both, respectively. The similarity of the mechanical analysis results demonstrated the generalization of the network.

[0078] The entities were meshed and material properties were assigned. The mesh size was adjusted to make the mesh distribution of the two entities more uniform. Then, in the mechanical analysis software, the material properties of the entities in the two CT images were assigned, and static mechanical analysis and gait mechanical analysis were performed on the entities respectively.

[0079] The static mechanics analysis results and the gait mechanics analysis results were compared and analyzed respectively to obtain the similarity value between the enhanced image and the healthy human bone;

[0080] The feasibility of the trained bone image region segmentation network is determined based on similarity.

[0081] Furthermore, feature extraction is performed on the enhanced image to obtain the image features of the plasticized tomographic image, including:

[0082] Use the DoG operator to enhance the detail features of the enhanced image to obtain a detail enhanced image;

[0083] The detail enhanced image is converted into a binary image by Otsu threshold segmentation, where "zero" represents pores and "one" represents trabeculae, and the pepper noise in the image is eliminated by closing operation, as shown in Figure 3 shown.

[0084] An interactive feature extraction interface is constructed to extract features from binary images and obtain the image features of plasticized tomographic images.

[0085] Furthermore, feature extraction is performed on the binary image to obtain the image features of the plasticized tomographic image, including:

[0086] Select a target region in the binary image, determine the coordinates and size of the target region, and calculate the area of ​​the target region based on the coordinates and size;

[0087] Specifically, an interactive feature extraction interface is built in the programmable software through program writing, a rectangular object is created through mouse control, the coordinates and size of the target area are determined, and then the area of ​​the target area is calculated according to the coordinate position of the target area, such as Figure 4 shown.

[0088] The porosity of the trabecular bone is determined based on the proportion of background pixels in the target area. Specifically, the porosity of the trabecular bone is obtained by calculating the ratio of zero pixels to the total number of pixels in the target area.

[0089] Calculate the structural directional entropy of the target area and obtain anisotropy based on the structural directional entropy. The specific steps are: calculate the horizontal and vertical gradient images of the target area image, then calculate the phase angle of the gradient, setting the gradient phase angle within 0-180 degrees. Then, divide the angle into eight intervals, count the number of pixels in each interval, and calculate the probability distribution of pixels within each interval. Using these parameters as input, calculate the structural directional entropy of the binary image according to the structural directional entropy formula, and divide the result by log28 to obtain the normalized structural directional entropy.

[0090] The contour retrieval method is used to obtain the outermost contour of the foreground area within the target area;

[0091] The outermost contour is compressed using a contour approximation algorithm, and the number of compressed contour lists is used as the number of trabeculae in the target area;

[0092] Calculate the Hessian matrix and eigenvector of the target area, and obtain the trabecular direction information based on the Hessian matrix and eigenvector. After further completing the direction extraction, the grayscale image with direction information is converted into a pseudo-color image through pseudo-color mapping according to the needs, so that it can more clearly describe the trabecular direction information in the original image. And use this pseudo-color image as a display image of the interactive interface to show the user the trabecular direction information of the selected area. Figure 4 shown.

[0093] Finally, based on the results of bone image segmentation, the soft tissue can be reversely selected, and the attachment area and travel distance can be calculated based on the spatial vertex coordinates of the image pixels.

[0094] On the other hand, the present invention provides an interactive intelligent plasticization tomographic image feature extraction system, which is used to implement the above-mentioned interactive intelligent plasticization tomographic image feature extraction method, such as Figure 5 As shown, including:

[0095] The dataset construction module is used to collect multiple plasticized tomographic images and construct a plasticized slice semantic segmentation dataset;

[0096] The model building module is used to build a deep learning convolutional neural network and train it using the plastinated slice semantic segmentation dataset to obtain a trained bone image region segmentation network;

[0097] A segmentation module is used to perform regional segmentation on the collected plasticized tomographic image using a trained bone image region segmentation network to obtain a segmentation result;

[0098] The feature extraction module is used to enhance the segmentation result to obtain an enhanced image, and to extract features from the enhanced image to obtain image features of the plasticized tomographic image.

[0099] Furthermore, the system also includes:

[0100] The verification module is used to verify the feasibility of the trained bone image region segmentation network.

[0101] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0102] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An interactive intelligent plasticization tomographic image feature extraction method, characterized in that: include: Collect multiple plasticized tomographic images and construct a plasticized slice semantic segmentation dataset; Constructing a deep learning convolutional neural network, and using the plasticized slice semantic segmentation dataset to train the deep learning convolutional neural network to obtain a trained bone image region segmentation network; The trained bone image region segmentation network is used to perform region segmentation on the collected plasticized tomographic images to obtain the segmentation results; performing enhancement processing on the segmentation result to obtain an enhanced image; Feature extraction is performed on the enhanced image to obtain image features of the plasticized tomographic image.

2. The interactive intelligent plasticization tomographic image feature extraction method according to claim 1, characterized in that: The deep learning convolutional neural network is trained using the plastinated slice semantic segmentation dataset to obtain a trained bone image region segmentation network, including: Using the plasticized slice semantic segmentation dataset to train the deep learning convolutional neural network to obtain a trained model; Evaluating the trained model output and the real plasticized slice image using a mean square error loss function; During the training process, a learning rate decay strategy is adopted, and the trained model is used as the deep learning convolutional neural network model for the next round of training for multiple trainings until the evaluation results do not improve for multiple consecutive times, and the trained bone image region segmentation network is output.

3. The interactive intelligent plasticization tomographic image feature extraction method according to claim 2, characterized in that: Performing enhancement processing on the segmentation result to obtain an enhanced image, including: Converting the segmentation result into a grayscale image; Performing local histogram equalization on the grayscale image to obtain an enhanced image.

4. The interactive intelligent plasticization tomographic image feature extraction method according to claim 1, characterized in that: After obtaining the trained bone image region segmentation network, the following steps are also included: The trained bone image region segmentation network is used to perform region segmentation on the collected plasticized tomographic images to obtain the segmentation results; performing enhancement processing on the segmentation result to obtain an enhanced image; converting the enhanced image into a CT image; Obtain CT images of healthy human bones; Converting the CT image into a solid through a solid operation; Performing meshing processing on the entity and assigning material properties; performing static mechanics analysis and gait mechanics analysis on the entity respectively; Comparing and analyzing the static mechanics analysis results and the gait mechanics analysis results respectively, and obtaining a similarity value between the enhanced image and a healthy human bone; The feasibility of the trained bone image region segmentation network is determined based on the similarity.

5. The interactive intelligent plasticization tomographic image feature extraction method according to claim 1, characterized in that: Performing feature extraction on the enhanced image to obtain image features of the plasticized tomographic image includes: Using the DoG operator to enhance the detail features of the enhanced image to obtain a detail enhanced image; The detail enhanced image is converted into a binary image through Otsu threshold segmentation; An interactive feature extraction interface is constructed to perform feature extraction on the binary image to obtain image features of the plasticized tomographic image.

6. The interactive intelligent plasticization tomographic image feature extraction method according to claim 5, characterized in that: Performing feature extraction on the binary image to obtain image features of the plasticized tomographic image includes: Selecting a target area in the binary image and determining the coordinates and size of the target area; Calculate the area of ​​the target area according to the coordinates and size; determining the porosity of the trabecular bone according to the ratio of background pixels in the target area; calculating the structural direction entropy of the target area, and obtaining anisotropy based on the structural direction entropy; Using a contour retrieval method to obtain the outermost contour of the foreground area within the target area; compressing the outermost contour using a contour approximation algorithm, and using the number of compressed contour lists as the number of trabeculae in the target area; The Hessian matrix and the eigenvector of the target area are calculated, and the trabecular direction information is obtained based on the Hessian matrix and the eigenvector.

7. An interactive intelligent plasticization tomographic image feature extraction system, characterized in that: The system is used to implement the interactive intelligent plasticization tomographic image feature extraction method according to any one of claims 1 to 6, comprising: The dataset construction module is used to collect multiple plasticized tomographic images and construct a plasticized slice semantic segmentation dataset; A model construction module is used to construct a deep learning convolutional neural network and train the deep learning convolutional neural network using the plasticized slice semantic segmentation dataset to obtain a trained bone image region segmentation network; A segmentation module is used to perform regional segmentation on the collected plasticized tomographic image using a trained bone image region segmentation network to obtain a segmentation result; The feature extraction module is used to perform enhancement processing on the segmentation result to obtain an enhanced image, and perform feature extraction on the enhanced image to obtain image features of the plasticized tomographic image.

8. The interactive intelligent plasticization tomographic image feature extraction system according to claim 7, characterized in that: The system further comprises: The verification module is used to verify the feasibility of the trained bone image region segmentation network.

Images