Bone tissue pathology identification method and system

Abstract

The present application provides a bone tissue pathology recognition method and system, which can perform restoration mapping of different descriptor restoration channels on image embedding descriptors generated by image embedding neural networks based on multiple image restoration neural networks, generate descriptor restoration losses of each image restoration neural network under their own descriptor restoration channels, and thus complete collaborative reinforcement debugging of the image embedding neural network based on the descriptor restoration losses of the multiple image restoration neural networks under multiple descriptor restoration channels, so that the network parameters of the image embedding neural network can be optimized with better quality under multiple descriptor restoration channels, thereby improving the debugging quality of the image embedding neural network, and the image embedding neural network obtained based on the debugging can generate more accurate image embedding descriptors for bone tissue medical images to be processed.

Description

Technical Field

[0001] The present application relates to but is not limited to the field of machine learning technology, and in particular to a method and system for identifying bone tissue pathology. Background Art

[0002] In the field of bone tissue pathology recognition, accurately extracting key information from medical images is very important for pathology recognition. Traditional bone tissue pathology recognition methods mainly rely on the experience and expertise of doctors, but this method is limited by the subjective judgment and personal experience of doctors, and it is difficult to ensure the accuracy and consistency of recognition. With the development of computer vision and deep learning technology, more and more researchers have begun to explore bone tissue pathology recognition methods based on machine learning models. Existing deep learning-based bone tissue pathology recognition methods usually use convolutional neural networks (CNNs) to extract and classify medical images. However, these methods still face some challenges when dealing with complex and changeable bone tissue medical images. First, there may be interference factors such as noise and artifacts in medical images, which will affect the accuracy of feature extraction. Secondly, bone tissue medical images under different pathological types vary greatly, and a single neural network model is difficult to adapt to all situations. Finally, existing methods usually only focus on the accuracy of classification results, while ignoring the problem of information loss in the feature extraction process. In order to solve the above problems, researchers have proposed a series of improvement methods. For example, some methods use data enhancement technology to increase the diversity of training samples, thereby improving the generalization ability of the model. Other methods extract richer feature information by designing more complex neural network structures. However, these methods still have certain limitations, such as data enhancement may lead to model overfitting, and complex network structures increase the computational complexity and training difficulty of the model. Summary of the invention

[0003] In view of this, the embodiment of the present application at least provides a method and system for identifying bone tissue pathology. The technical solution of the present application is implemented as follows:

[0004] On the one hand, the present application provides a bone tissue pathology recognition method, the method comprising: obtaining a first bone tissue medical image, and performing an image embedding operation on the first bone tissue medical image through an image embedding neural network to generate a target image embedding descriptor of the first bone tissue medical image; performing restoration mapping on the target image embedding descriptor from X descriptor restoration channels through X image restoration neural networks to obtain a descriptor restoration result of each image restoration neural network; X≥1, an image restoration neural network is used to restore and map the target image embedding descriptor in a corresponding descriptor restoration channel; according to the descriptor restoration result of each image restoration neural network, obtaining the descriptor restoration loss of each image restoration neural network for the target image embedding descriptor; adjusting the network parameter of the image embedding neural network according to the obtained X descriptor restoration losses to obtain a debugged image embedding neural network; the debugged image embedding neural network is used to generate a dense image embedding descriptor of the bone tissue medical image to be processed; obtaining the image embedding descriptor uploaded by the bone tissue terminal recognition device The bone tissue medical image to be identified is used as a bone tissue medical image, and an image embedding operation is performed on the bone tissue medical image to be identified through the debugged image embedding neural network to generate a dense image embedding descriptor of the bone tissue medical image to be identified; descriptor commonality measurement results between the dense image embedding descriptor of the bone tissue medical image to be identified and each dense image embedding descriptor in the image embedding descriptor set are obtained; a matching image embedding descriptor is selected from the image embedding descriptor set according to the descriptor commonality measurement results between the dense image embedding descriptor of the bone tissue medical image to be identified and each dense image embedding descriptor in the image embedding descriptor set; the matching bone tissue medical image to which the matching image embedding descriptor belongs is determined as the matching bone tissue medical image of the bone tissue medical image to be identified, and the matching bone tissue medical image of the bone tissue medical image to be identified is sent to the bone tissue terminal identification device, so that the bone tissue terminal identification device outputs the matching bone tissue medical image of the bone tissue medical image to be identified and the pathological information contained in the matching bone tissue medical image.

[0005] On the other hand, the present application provides a computer system, including a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor implements the steps in the above method when executing the program.

[0006] The beneficial effects of the present application include at least: the present application obtains a first bone tissue medical image, and performs an image embedding operation on the first bone tissue medical image through an image embedding neural network to generate a target image embedding descriptor of the first bone tissue medical image; restores and maps the target image embedding descriptor from X descriptor restoration channels through X image restoration neural networks to obtain a descriptor restoration result of each image restoration neural network; X≥1, an image restoration neural network is used to restore and map the target image embedding descriptor in a corresponding descriptor restoration channel; at the same time, according to the descriptor restoration result of each image restoration neural network, the descriptor restoration loss of each image restoration neural network for the target image embedding descriptor is obtained; the network parameters of the image embedding neural network are adjusted according to the obtained X descriptor restoration losses to obtain a debugged image embedding neural network; the debugged image embedding neural network is used to generate a dense image embedding descriptor of the bone tissue medical image to be processed. Based on this, the present application can perform restoration mapping of different descriptor restoration channels on the image embedding descriptors generated by the image embedding neural network based on multiple image restoration neural networks, and generate descriptor restoration losses of each image restoration neural network under their respective descriptor restoration channels. In this way, based on the descriptor restoration losses of the multiple image restoration neural networks under multiple descriptor restoration channels, the collaborative reinforcement debugging of the image embedding neural network is completed, so that the network parameters of the image embedding neural network can be optimized with better quality under multiple descriptor restoration channels, thereby improving the debugging quality of the image embedding neural network. The image embedding neural network obtained based on the debugging can generate more accurate image embedding descriptors for the processed bone tissue medical images. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 A schematic diagram of the implementation flow of the bone tissue pathology identification method provided in an embodiment of the present application.

[0008] Figure 2 A schematic diagram of the hardware entity of the computer system provided in an embodiment of the present application.

[0009] Reference numerals: 1000 - computer system; 1001 - processor; 1002 - memory. DETAILED DESCRIPTION

[0010] The embodiment of the present application provides a method for identifying bone tissue pathology, which can be executed by a processor of a computer system, wherein the computer system can refer to a device with data processing capabilities such as a server, a laptop, a tablet computer, and a desktop computer.

[0011] Figure 1 A schematic diagram of the implementation process of a bone tissue pathology identification method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, this method

[0012] It includes two execution phases: neural network training phase and neural network application phase.

[0013] 1. The training phase of the image embedding neural network includes the following steps S100~S400:

[0014] Step S100: Acquire a first bone tissue medical image, and perform an image embedding operation on the first bone tissue medical image through an image embedding neural network to generate a target image embedding descriptor of the first bone tissue medical image.

[0015] In the embodiment of the present application, step S100 involves the acquisition and image embedding operation of bone tissue medical images. First, the computer system needs to acquire a first bone tissue medical image from a medical imaging device or a medical image database. This medical image is usually obtained through imaging technologies such as X-rays, CT scans, MRI, etc., and contains detailed information of bone tissue, such as bone density, bone structure, etc. After acquiring the medical image, the computer system preprocesses the image, such as cropping, scaling, and normalization, for subsequent analysis. Next, the computer system uses an image embedding neural network (i.e., an image encoding network) to perform an image embedding operation on the first bone tissue medical image. This neural network model is usually a deep learning model, such as a convolutional neural network (CNN), which can automatically learn features in the image and encode these features into a compact vector representation, i.e., a target image embedding descriptor. This descriptor contains key information of the image and can be used for tasks such as image matching, classification, or retrieval. Specifically, the image embedding neural network performs multi-level feature extraction and abstraction on the input first bone tissue medical image. In each layer of convolution operation, the network learns and captures local features in the image, such as edges, textures, etc. As the network goes deeper, these local features will gradually be combined into higher-level feature representations, such as shape, structure, etc. Finally, the network will output a fixed-length vector, namely the target image embedding descriptor, which is the encoding feature of the first bone tissue medical image.

[0016] For example, suppose a bone tissue medical image obtained by CT scan shows a fracture site. The computer system first obtains this CT image and inputs it into the trained image embedding neural network, which performs multi-level feature extraction on the image and finally outputs a target image embedding descriptor containing key information of the fracture site. This descriptor can be used to match the descriptors in the existing bone tissue medical image library to find similar cases or pathological information, providing doctors with a basis for auxiliary diagnosis.

[0017] Step S200: Using X image restoration neural networks to restore and map the target image embedding descriptors from X descriptor restoration channels respectively, to obtain a descriptor restoration result of each image restoration neural network; X≥1, an image restoration neural network is used to restore and map the target image embedding descriptor in a corresponding descriptor restoration channel.

[0018] In an embodiment of the present application, step S200 restores and maps the target image embedding descriptor through multiple image restoration neural networks to verify the accuracy and effectiveness of the embedded description. In practical applications, the computer system first embeds the bone tissue medical image through the image embedding neural network to generate the target image embedding descriptor. This descriptor is a highly abstract and generalized version of the original medical image and contains the key information in the image. Then, in step S200, the computer system uses X image restoration neural networks (correspondingly, a decoding network) to restore and map the target image embedding descriptor. The X image restoration neural networks here are independent of each other, and each image restoration neural network is responsible for decoding the descriptor from a specific descriptor restoration channel, where a restoration channel corresponds to a decoded dimension. These image restoration neural networks can be different types of neural networks, such as deconvolution networks, generators of generative adversarial networks (GANs), etc., which are trained to reconstruct the original medical image from the embedded description.

[0019] Specifically, each image restoration neural network takes the target image embedded in the descriptor as input and tries to generate a reconstructed image that is as similar as possible to the original medical image. Since different image restoration neural networks decode the descriptors from different angles and dimensions, they generate different reconstruction results. These reconstruction results reflect the details and features of the original medical image in different aspects.

[0020] For example, suppose there are three image restoration neural networks (X=3), which correspond to different descriptor restoration channels. The first image restoration neural network may focus on reconstructing the overall structure and contour of the medical image, the second image restoration neural network may focus more on the texture and details of the image, and the third image restoration neural network may focus on the local area or specific pathological features of the image. When the target image embedding descriptor is input into the three image restoration neural networks, each restoration neural network generates a reconstructed medical image. By comparing the differences between these reconstructed images and the original medical images, the performance of the image embedding neural network can be evaluated. If the reconstructed image is very similar to the original image, it means that the embedded descriptor contains enough information to accurately represent the original image, otherwise it means that the embedded descriptor may have lost some key information. In this way, step S200 not only helps to verify the performance of the image embedding neural network, but also finds possible problems and deficiencies in the network, providing a basis for further improving and optimizing the network. At the same time, by decoding the descriptor from different angles through multiple image restoration neural networks, more features and information can be provided for subsequent image matching, classification or retrieval tasks.

[0021] Step S300: According to the descriptor restoration results of each image restoration neural network, the descriptor restoration loss of each image restoration neural network for the target image embedding descriptor is obtained.

[0022] In the embodiment of the present application, step S300 calculates the respective descriptor restoration losses according to the restoration results of the target image embedded descriptors by each image restoration neural network.

[0023] In practical applications, after the computer system encodes the bone tissue medical image through the image embedding neural network and generates the target image embedding descriptor, it uses multiple image restoration neural networks to try to reconstruct the original medical image from the descriptor. Each image restoration neural network decodes the descriptor from a specific descriptor restoration channel to generate a reconstructed medical image. In step S300, the computer system needs to evaluate the accuracy of these reconstructed images to quantify the encoding performance of the image embedding neural network. To this end, the descriptor restoration loss of each image restoration neural network is calculated. This loss is obtained by comparing the difference between the reconstructed image and the original medical image, which reflects the extent to which the reconstructed image retains the information of the original image. Specifically, the computer system can use various loss functions to calculate the descriptor restoration loss. For example, it can use the mean square error (MSE) loss function to calculate the difference between the reconstructed image and the original image at the pixel level. For each image restoration neural network, it compares the pixel values ​​at the corresponding positions of the reconstructed image and the original image, calculates the square difference between them, and averages the square differences of all pixels to obtain the descriptor restoration loss of the network.

[0024] For example, there is a medical image of bone tissue with a clear fracture line. After being encoded by the image embedding neural network, it is decoded using two image restoration neural networks. The first network focuses on reconstructing the overall structure of the image, while the second network focuses more on the local details of the image.

[0025] After the two networks generate reconstructed images respectively, the computer system compares the differences between these reconstructed images and the original images. If the image reconstructed by the first network is very similar to the original image in terms of overall structure, but the details of the fracture line are blurred, then its descriptor restoration loss may be relatively low, but not enough to fully capture the information of the fracture line. If the second network can clearly restore the details of the fracture line in the reconstructed image, then its descriptor restoration loss may be lower, indicating that it does a better job in preserving local details. By calculating the descriptor restoration loss of each image restoration neural network, the computer system can comprehensively evaluate the encoding performance of the image embedding neural network. This helps to discover possible problems in the network during the encoding process and provide guidance for subsequent optimization and improvement. At the same time, these loss values ​​can also be used in the backpropagation algorithm during the training process to update the network parameters and further improve the encoding accuracy of the network.

[0026] Step S400: adjusting the network parameters of the image embedding neural network according to the obtained X descriptor restoration losses to obtain a debugged image embedding neural network; the debugged image embedding neural network is used to generate a dense image embedding descriptor of the bone tissue medical image to be processed.

[0027] In the embodiment of the present application, step S400 uses the descriptor restoration losses generated by multiple image restoration neural networks to adjust the network parameters of the image embedding neural network, thereby obtaining a performance-optimized network model.

[0028] In practical applications, when a computer system embeds bone tissue medical images through an image embedding neural network, and uses multiple image restoration neural networks to decode and reconstruct the encoded target image embedding descriptors, multiple descriptor restoration losses are generated. These loss values ​​quantify the difference between the reconstructed image and the original image, reflecting the performance of the image embedding neural network in the encoding process. In step S400, the computer system first collects the X descriptor restoration losses. These loss values ​​come from different image restoration neural networks, each of which decodes the target image embedding descriptor from different angles and dimensions, thus providing a multi-dimensional evaluation of the encoding performance. Next, the computer system uses these loss values ​​to adjust the network parameters of the image embedding neural network. This process is usually implemented by a back propagation algorithm. The back propagation algorithm calculates the gradient based on the loss value, and then updates the network parameters such as weights and biases through methods such as gradient descent to minimize the descriptor restoration loss. In the process of adjusting the network parameters, the computer system comprehensively considers all X descriptor restoration losses. This can be achieved by simple averaging or weighted averaging, or more complex optimization strategies such as adaptive learning rate, momentum, etc. Through continuous iteration and optimization, the performance of the image embedding neural network will gradually improve until the preset stopping conditions are reached (such as loss value convergence, number of iterations reaching the upper limit, etc.).

[0029] Finally, after debugging and optimization in step S400, the computer system will obtain a debugged image embedding neural network. This network model can generate more accurate and richer dense image embedding descriptors, that is, dense image features, when encoding bone tissue medical images.

[0030] For example, suppose there is an initial image embedding neural network, and the descriptors it generates when encoding bone tissue medical images are not accurate enough in some details. Through the debugging and optimization of step S400, the descriptor restoration losses generated by multiple image restoration neural networks can be used to adjust the parameters of the network. After multiple rounds of iterations, a debugged image embedding neural network is obtained. This new network model can generate more accurate descriptors when encoding the same bone tissue medical images. These improvements in details will help improve the accuracy of subsequent pathology recognition tasks.

[0031] Second, the application stage of the image embedding neural network includes the following steps S500~S800:

[0032] Step S500: Obtain the medical image of the bone tissue to be identified uploaded by the bone tissue terminal identification device, and perform image embedding operation on the medical image of the bone tissue to be identified through the debugged image embedding neural network to generate a dense image embedding descriptor of the medical image of the bone tissue to be identified.

[0033] In an embodiment of the present application, step S500 obtains the bone tissue medical image to be identified and performs an image embedding operation through the debugged image embedding neural network, thereby generating a dense image embedding descriptor of the image to be identified. In practical applications, a bone tissue terminal recognition device, such as a medical imaging workstation or mobile diagnostic equipment in a hospital, will collect the patient's bone tissue medical images. These images may contain various bone tissue pathological features, such as fractures, osteoporosis, bone tumors, etc. Once the device collects these bone tissue medical images to be identified, they are uploaded to the bone tissue pathology recognition system through a secure data transmission method, such as an encrypted network connection. After receiving the uploaded bone tissue medical images to be identified, the computer system will use the previously debugged image embedding neural network to perform image embedding operations on these images. This network model has been trained and optimized, and can accurately extract key features from bone tissue medical images and encode these features into a dense image embedding descriptor.

[0034] Specifically, the image embedding neural network will perform multi-level feature extraction and abstraction on the input bone tissue medical image to be identified. In each layer of convolution operation, the image embedding neural network will automatically learn and capture local features in the image, such as edges, textures, etc. As the network goes deeper, these local features will gradually be combined into higher-level feature representations, such as shape, structure, etc. Finally, the image embedding neural network will output a fixed-length vector, namely the dense image embedding descriptor. This descriptor contains the key information of the bone tissue medical image to be identified, which can be used for subsequent image matching, classification or retrieval tasks.

[0035] For example, suppose a hospital uses a bone tissue pathology recognition system that has trained an image embedding neural network. When an X-ray of a patient with a fracture is uploaded to the system, the computer system will use this trained network to perform image embedding operations on the X-ray. The image embedding neural network will automatically extract the features of the fracture site from the X-ray, such as the location and morphology of the fracture line, and encode these features into a dense image embedding descriptor. This descriptor will serve as the basis for subsequent pathology recognition, helping doctors to quickly and accurately diagnose the patient's condition.

[0036] Step S600: Obtain descriptor commonality measurement results between the dense image embedding descriptor of the bone tissue medical image to be identified and each dense image embedding descriptor in the image embedding descriptor set.

[0037] In the embodiment of the present application, step S600 measures the commonality between the dense image embedding descriptor of the bone tissue medical image to be identified and each descriptor in the pre-constructed image embedding descriptor set to determine the similarity or matching degree between them.

[0038] First, when the computer system performs image embedding operations on the bone tissue medical image to be identified through the debugged image embedding neural network, a dense image embedding descriptor will be generated. This descriptor is a high-dimensional vector that contains the key feature information of the medical image to be identified. Next, the computer system needs to find the descriptor that is most similar to the descriptor of the medical image to be identified from this descriptor set. This descriptor set usually contains a large number of dense image embedding descriptors of bone tissue medical images of known pathological types. These descriptors are obtained by encoding known medical images through the same image embedding neural network.

[0039] To achieve this goal, the computer system needs to calculate the commonality measurement results between the dense image embedding descriptor of the medical image to be identified and each descriptor in the descriptor set. This commonality measurement result is usually obtained by calculating the similarity score between the two descriptors. Commonly used similarity calculation methods include cosine similarity and Euclidean distance. Taking cosine similarity as an example, the computer system regards the dense image embedding descriptor of the medical image to be identified as a vector, and then regards each descriptor in the descriptor set as a vector. Next, the cosine value of the angle between the descriptor vector of the medical image to be identified and each descriptor vector in the descriptor set is calculated. The size of this cosine value reflects the similarity between the two descriptors. The closer the cosine value is to 1, the more similar the two descriptors are; the closer the cosine value is to -1, the less similar the two descriptors are.

[0040] In practical applications, computer systems usually set a threshold. Only when the commonality measurement result (such as cosine similarity) exceeds this threshold, the medical image to be identified is considered similar to a descriptor in the descriptor set. In this way, the computer system can quickly filter out pathological types similar to the medical image to be identified from a large number of known pathological types, providing doctors with a basis for auxiliary diagnosis.

[0041] For example, suppose the medical image to be identified is an X-ray showing a fracture. After encoding through the image embedding neural network, a dense image embedding descriptor is obtained. The descriptor set contains descriptors for various bone tissue pathology types, such as fractures, osteoporosis, bone tumors, etc. The computer system calculates the cosine similarity between the descriptor of the medical image to be identified and each descriptor in the descriptor set, and finds that the cosine similarity with the descriptor of the fracture type is the highest, exceeding the preset threshold. Therefore, the computer system can determine that the medical image to be identified is likely to be an X-ray showing a fracture, and output this result to the doctor for reference.

[0042] Step S700: Select a matching image embedding descriptor from the image embedding descriptor set based on the descriptor commonality measurement results between the dense image embedding descriptor of the bone tissue medical image to be identified and each dense image embedding descriptor in the image embedding descriptor set.

[0043] In the embodiment of the present application, step S700 selects a matching image embedding descriptor based on the commonality measurement result between the dense image embedding descriptor of the bone tissue medical image to be identified and each descriptor in the image embedding descriptor set. The purpose of this step is to find the known case that is most similar to the pathological features of the medical image to be identified, so as to provide a basis for auxiliary diagnosis for doctors.

[0044] Specifically, when the computer system encodes the bone tissue medical image to be identified through the image embedding neural network and generates a dense image embedding descriptor, it will measure the commonality with each descriptor in the pre-built image embedding descriptor set. This descriptor set contains a large number of dense image embedding descriptors of bone tissue medical images of known pathological types, which are obtained by encoding the known medical images through the same image embedding neural network. Commonality measurement is usually achieved by calculating the similarity score between two descriptors. Commonly used similarity calculation methods include cosine similarity, Euclidean distance, etc. In the scenario of bone tissue pathology recognition, assuming that cosine similarity is used as the commonality measurement method, the closer the calculated cosine value is to 1, the more similar the two descriptors are; the closer the cosine value is to -1, the less similar the two descriptors are.

[0045] Next, the computer system selects a matching image embedding descriptor from the image embedding descriptor set based on the commonality measurement results. There are many ways to implement this step. Two common strategies are listed below:

[0046] Select the descriptor with the highest similarity: The computer system sorts the commonality measurement results (cosine similarity) of the descriptor of the medical image to be identified and each descriptor in the descriptor set, and selects the descriptor with the highest similarity as the matching result. This method is suitable for scenarios where only the most similar cases need to be found.

[0047] Set similarity threshold: The computer system first sets a similarity threshold (such as 0.8), then compares the commonality measurement results of the descriptor of the medical image to be identified and each descriptor in the descriptor set with the threshold, and selects the descriptor with a similarity exceeding the threshold as the matching result. This method is suitable for scenarios where multiple similar cases need to be found or where there are certain requirements for similarity.

[0048] For example, suppose the medical image to be identified is an X-ray showing a fracture. After encoding through the image embedding neural network, a dense image embedding descriptor is obtained. The descriptor set contains descriptors for various bone tissue pathology types, such as fractures, osteoporosis, bone tumors, etc. The computer system calculates the cosine similarity between the descriptor of the medical image to be identified and each descriptor in the descriptor set, and finds that the descriptor similarity with the fracture type is the highest, exceeding the preset threshold. Therefore, the computer system selects the descriptor of this fracture type as the matching result and outputs the matching result to the doctor, prompting the doctor that the medical image to be identified may be related to fractures.

[0049] Step S800: Determine the matching bone tissue medical image to which the matching image embedding descriptor belongs as the matching bone tissue medical image of the bone tissue medical image to be identified, and send the matching bone tissue medical image of the bone tissue medical image to be identified to the bone tissue terminal identification device, so that the bone tissue terminal identification device outputs the matching bone tissue medical image of the bone tissue medical image to be identified and the pathological information contained in the matching bone tissue medical image.

[0050] In an embodiment of the present application, step S800 determines the matching bone tissue medical image corresponding to the matching image embedded descriptor as a similar case of the bone tissue medical image to be identified, and sends this information back to the bone tissue terminal identification device so that doctors or relevant experts can refer to similar cases for a more accurate diagnosis.

[0051] Specifically, after the computer system selects the matching image embedding descriptor from the image embedding descriptor set through step S700, it will further determine the original bone tissue medical image corresponding to this descriptor, that is, the matching bone tissue medical image. This matching bone tissue medical image is the known case that is most similar to the bone tissue medical image to be identified in the descriptor set, so it contains pathological features similar to the medical image to be identified. Next, the computer system sends this matching bone tissue medical image and the pathological information it contains to the bone tissue terminal identification device. This device may be a medical imaging workstation, mobile diagnostic equipment or other medical professional equipment in a hospital, which can receive and display the information sent by the computer system.

[0052] On the bone tissue terminal recognition device, doctors or relevant experts can see the matching bone tissue medical image of the bone tissue to be recognized, as well as the pathological information contained in the matching image. This information can help doctors understand the pathological features in the medical image to be recognized more quickly and accurately, thereby making a more accurate diagnosis.

[0053] Through step S800, the computer system not only realizes automatic recognition and matching of the medical images to be identified, but also provides doctors with a powerful auxiliary diagnosis tool by sending matching cases and their pathological information to the bone tissue terminal recognition device, thereby improving the accuracy and efficiency of diagnosis.

[0054] As an implementation manner, the step S700, based on the descriptor commonality measurement results between the dense image embedding descriptor of the bone tissue medical image to be identified and each dense image embedding descriptor in the image embedding descriptor set, selects a matching image embedding descriptor in the image embedding descriptor set, which may specifically include:

[0055] Step S710a: Arrange the dense image embedding descriptors in the image embedding descriptor set in sequence according to the decreasing direction of the descriptor commonality measurement results between the dense image embedding descriptor of the bone tissue medical image to be identified and each dense image embedding descriptor in the image embedding descriptor set to obtain arranged dense image embedding descriptors.

[0056] In an embodiment of the present application, step S710a sorts the descriptors in the descriptor set based on the commonality measurement results between the dense image embedding descriptor of the bone tissue medical image to be identified and each descriptor in the image embedding descriptor set. First, when the computer system receives the bone tissue medical image to be identified, the image is encoded by the previously trained image embedding neural network to generate a dense image embedding descriptor. This descriptor is a high-dimensional vector that contains key feature information of the medical image to be identified. Next, the computer system takes out the dense image embedding descriptor of each known case from the pre-constructed image embedding descriptor set. This descriptor set is obtained by encoding a large number of bone tissue medical images of known pathological types through the same image embedding neural network, and each descriptor corresponds to a specific case.

[0057] Then, the computer system uses a commonality measurement algorithm to calculate the commonality measurement result between the descriptor of the medical image to be identified and each descriptor in the descriptor set. This commonality measurement result reflects the similarity between the two descriptors. Common commonality measurement algorithms include cosine similarity, Euclidean distance, etc. Taking cosine similarity as an example, the closer the calculated value is to 1, the more similar the two descriptors are; the closer the value is to -1, the less similar they are. After obtaining all the commonality measurement results, the computer system sorts them in descending order according to the size of these results. In other words, the descriptors with the highest similarity will be ranked in front, and the descriptors with lower similarity will be ranked in the back. This sorting process is automatically completed based on the calculation results of the commonality measurement algorithm without manual intervention. Finally, the computer system will obtain a well-arranged list of dense image embedding descriptors. The descriptors in this list are arranged from high to low according to their similarity with the descriptors of the medical image to be identified, which provides a basis for the subsequent selection of matching image embedding descriptors.

[0058] For example, suppose the medical image to be identified is an X-ray showing a fracture, and the image embedding descriptor set contains a large number of descriptors of known cases. The computer system first calculates the cosine similarity between the descriptor of the image to be identified and each descriptor in the set, and then sorts them from high to low according to the similarity. In the sorted list, the descriptor with the highest similarity to the fracture case descriptor will be ranked first, while the descriptors with lower similarity to other case descriptors such as osteoporosis and bone tumors will be ranked at the back. In this way, doctors can quickly find the known case that is most similar to the image to be identified based on the sorted list, thereby assisting in diagnosis.

[0059] Step S720a: Determine the first A dense image embedding descriptors among the arranged dense image embedding descriptors as the matching image embedding descriptors; A≥1.

[0060] In an embodiment of the present application, step S720a selects a known case that is most similar to the medical image of bone tissue to be identified, providing a basis for auxiliary diagnosis for doctors. First, according to step S710a, the computer system has calculated the similarity between the dense image embedding descriptor of the medical image to be identified and each descriptor in the image embedding descriptor set through a commonality measurement algorithm, and sorted them in descending order according to the similarity. This sorted list reflects the similarity relationship between the descriptors, and the descriptors with higher similarity are ranked higher. Next, in step S720a, the computer system will select the first A dense image embedding descriptors from this sorted list as matching image embedding descriptors. Here A is a preset positive integer representing the number of matching descriptors to be obtained. The value of A can be adjusted according to actual needs to balance the accuracy and quantity of matching descriptions.

[0061] Specifically, the computer system starts from the top of the sorted list and selects A descriptors as matching items in turn. Since the list is sorted in descending order of similarity, the selected descriptors have the highest similarity with the descriptors of the medical image to be identified. The original bone tissue medical images corresponding to these matching descriptors, i.e., the matching bone tissue medical images, will be used as the known cases most similar to the image to be identified.

[0062] Or; in another embodiment, the step S700, based on the descriptor commonality measurement results between the dense image embedding descriptor of the bone tissue medical image to be identified and each dense image embedding descriptor in the image embedding descriptor set, selects a matching image embedding descriptor in the image embedding descriptor set, which may specifically include:

[0063] Step S710b: Obtain a commonality measurement threshold for descriptor commonality measurement results between dense image embedding descriptors.

[0064] In an embodiment of the present application, step S710b determines a commonality metric threshold, which is used to subsequently screen out matching image embedding descriptors that are highly similar to the bone tissue medical image to be identified from the image embedding descriptor set. This step ensures that the selected matching items have sufficient similarity, thereby improving the accuracy and reliability of the diagnosis. Specifically, in step S710b, the computer system first determines a commonality metric threshold. This threshold represents the minimum requirement for the similarity between descriptors, and only descriptors whose commonality metric results exceed this threshold will be considered as potential matches. The setting of the commonality metric threshold is usually based on domain knowledge and experience, as well as the requirements for diagnostic accuracy.

[0065] In practical applications, the determination of the commonality metric threshold may involve multiple factors. First, the specific needs and scenarios of bone tissue pathology recognition need to be considered. For example, in the application of fracture recognition, due to the variety of fracture types and locations, a lower commonality metric threshold may need to be set to ensure that as many similar cases as possible can be covered. In the recognition of certain types of bone tumors, due to the clear characteristics of the lesions, a higher commonality metric threshold can be set to improve the accuracy of matching. Secondly, the setting of the commonality metric threshold also needs to consider the distribution of similarity between descriptors. If most of the descriptors in the descriptor set have low similarity with the descriptors of the medical image to be identified, then the threshold may need to be appropriately lowered to avoid missing potential matches. On the contrary, if there are a large number of descriptors in the descriptor set that are highly similar to the medical image to be identified, then the threshold can be appropriately increased to improve the accuracy of matching. Finally, the setting of the commonality metric threshold also needs to be adjusted in combination with experimental verification and actual application effects. In practical applications, multiple experiments can be conducted to verify the matching effect and diagnostic accuracy under different thresholds, so as to determine a suitable commonality metric threshold.

[0066] For example, suppose that in the application of fracture recognition, the computer system has determined a commonality measurement threshold of 0.8 through preliminary experiments and verification (using cosine similarity as the commonality measurement indicator). This means that only when the cosine similarity between the descriptor of the medical image to be recognized and a descriptor in the descriptor set is greater than 0.8, the descriptor will be considered as a potential match. By setting such a threshold, it can be ensured that the selected match has a high similarity with the medical image to be recognized, thereby improving the accuracy and reliability of fracture recognition.

[0067] Step S720b: Determine the dense image embedding descriptor in the image embedding descriptor set whose descriptor commonality measurement result with the dense image embedding descriptor of the bone tissue medical image to be identified is not less than the commonality measurement threshold as the matching image embedding descriptor.

[0068] In an embodiment of the present application, step S720b selects descriptors having sufficient similarity to the dense image embedding descriptors of the bone tissue medical image to be identified from the image embedding descriptor set based on the commonality metric threshold set previously, and these descriptors are subsequently determined as matching image embedding descriptors.

[0069] Specifically, after the commonality metric threshold has been determined in step S710b, the computer system traverses each dense image embedding descriptor in the image embedding descriptor set. For each descriptor in the set, the computer calculates the commonality metric result, such as cosine similarity, between it and the dense image embedding descriptor of the medical image to be identified. If the calculated commonality metric result is not less than the preset commonality metric threshold, then the descriptor is considered to be sufficiently similar to the medical image to be identified, and is thus marked as a matching image embedding descriptor. The advantage of this method is that it allows the computer system to flexibly adjust the strictness of the match according to specific similarity requirements. For example, in a scenario where highly accurate matching is required, a higher commonality metric threshold can be set to ensure that only descriptors that are very similar to the medical image to be identified will be selected. On the contrary, in a scenario where a wider match is required, a lower commonality metric threshold can be set to cover more possible similar cases. For example, assuming that the medical image to be identified is an X-ray showing a specific type of fracture, the image embedding descriptor set contains a large number of descriptors of known fracture cases. If the commonality metric threshold is set to 0.8 (using cosine similarity as the metric), the computer system will traverse each descriptor in the set and calculate the cosine similarity between them and the descriptors of the medical image to be identified. Any descriptor with a similarity greater than or equal to 0.8 will be considered a matching image embedding descriptor, and the original bone tissue medical images corresponding to them will be considered as known cases similar to the X-rays to be identified. These similar cases can provide doctors with valuable reference information to help them more accurately diagnose the pathological features in the medical images to be identified.

[0070] As an implementation manner, the first bone tissue medical image is obtained by performing a masking operation on the initial bone tissue medical image based on a first masking percentage, and the X image restoration neural networks include an image reconstruction restoration neural network. Based on this, the step S200, using the X image restoration neural networks to respectively restore and map the target image embedding descriptors from the X descriptor restoration channels, to obtain the descriptor restoration results of each image restoration neural network, may specifically include:

[0071] Step S210: Acquire a second bone tissue medical image; the second bone tissue medical image is obtained by shielding the initial bone tissue medical image based on a second shielding percentage, the second shielding percentage is greater than the first shielding percentage, and the second bone tissue medical image includes a second shielded bone tissue medical image obtained by shielding a partial image area of ​​the initial bone tissue medical image according to the second shielding percentage.

[0072] In the embodiment of the present application, step S210 is intended to generate a second bone tissue medical image for further evaluating the performance of the image embedding neural network. The core of this step is to simulate the complex situations that may be encountered in actual applications by performing different degrees of shielding operations on the initial bone tissue medical image, so as to more comprehensively evaluate the robustness and generalization ability of the neural network.

[0073] Specifically, the execution process of step S210 is as follows: First, the computer system has an initial bone tissue medical image, which contains a complete bone tissue structure and possible pathological features. In order to simulate the occlusion or damage that may occur in the medical image during the acquisition or transmission process, the initial image will be shielded. This operation is essentially to replace a part of the image with a mask, that is, to cover these areas with a specific masking symbol (such as a black square or pixel value) to make them invisible. In step S210, this shielding operation is performed based on a second shielding percentage, which is greater than the first shielding percentage previously used to generate the first bone tissue medical image. For example, if the first shielding percentage is 20%, it means that 20% of the area in the initial image is shielded, then in step S210, the second shielding percentage may be set to 30% or higher, resulting in more image areas being shielded. By increasing the shielding percentage, a second bone tissue medical image, also called a second shielded bone tissue medical image, is generated. The shielded area in this image is larger, simulating a more serious data loss or damage situation. This processing method can help the computer system evaluate the performance of the image embedding neural network when facing more complex and extreme data corruption. This evaluation method is particularly important in actual application scenarios. Because in actual medical image analysis, due to equipment failure, patient movement or other reasons, the acquired medical images may be partially blurred, blocked or damaged. If the image embedding neural network cannot accurately extract key features in this case, the subsequent pathology recognition task may be affected. Therefore, by introducing this simulated data corruption situation in the training stage and evaluating the performance of the neural network, it can help researchers identify potential problems and make improvements, thereby improving the robustness and accuracy of the entire system.

[0074] Step S220: Reconstruct the second shielded bone tissue medical image through the image reconstruction and restoration neural network according to the target image embedding descriptor and the second bone tissue medical image, and generate a reconstruction confidence of the second shielded bone tissue medical image for the second reference bone tissue medical image; the second reference bone tissue medical image is a partial image area in the initial bone tissue medical image corresponding to the second shielded bone tissue medical image.

[0075] In an embodiment of the present application, step S220 uses an image reconstruction restoration neural network to reconstruct the second shielded bone tissue medical image and generate a reconstruction confidence. This step not only demonstrates the effectiveness of the image embedding descriptor, but also evaluates the robustness of the image embedding neural network by simulating more complex data corruption situations. Specifically, in step S220, the computer system first obtains the target image embedding descriptor and the second bone tissue medical image. The target image embedding descriptor is obtained by encoding the initial bone tissue medical image through the image embedding neural network, and it contains the key information of the image. The second bone tissue medical image is obtained by performing a higher proportion of shielding operations on the initial bone tissue medical image (that is, the second shielding percentage is greater than the first shielding percentage), simulating a more serious image damage situation.

[0076] Next, the image reconstruction restoration neural network starts working. This neural network is trained to reconstruct the original medical image based on the given image embedding descriptor. In step S220, the network takes the target image embedding descriptor and the second bone tissue medical image as input. Among them, the target image embedding descriptor provides information about the overall structure and key features of the original medical image, while the second bone tissue medical image indicates which areas need to be reconstructed (i.e., the masked areas).

[0077] The image reconstruction and restoration neural network attempts to reconstruct the masked area in the second masked bone tissue medical image based on this information. This process is achieved through multi-layer convolution, pooling and other operations within the network, and finally outputs a reconstructed medical image. This reconstructed image covers the masked area in the second masked bone tissue medical image and restores the content of the original image as much as possible. In addition to the reconstructed medical image, the image reconstruction and restoration neural network also outputs a reconstruction confidence. This confidence is a quantitative indicator that reflects the degree of confidence in the reconstruction result. Specifically, it can be a value between 0 and 1, and the higher the value, the more confident the network is in the reconstruction result. This reconstruction confidence is based on the evaluation of the reconstruction process within the network, and can be used to measure the effectiveness of the image embedding descriptor in guiding the network to reconstruct the image.

[0078] It should be noted that the second reference bone tissue medical image in step S220 refers to a partial area of ​​the initial bone tissue medical image corresponding to the second shielded bone tissue medical image. This reference image provides the original content of the shielded area for comparison and evaluation with the reconstructed image. By comparing the difference between the reconstructed image and the second reference bone tissue medical image, the accuracy and effectiveness of the image embedding descriptor can be further verified.

[0079] Step S230: Determine the reconstruction confidence of the second shielded bone tissue medical image with respect to the second reference bone tissue medical image as the descriptor restoration result of the image reconstruction restoration neural network. Step S230 not only summarizes the effect of the network on the reconstruction of the second shielded bone tissue medical image, but also quantifies the effectiveness of the image embedding descriptor in guiding the network to perform image reconstruction.

[0080] Specifically, in step S230, the computer system first receives the reconstruction confidence output by the image reconstruction restoration neural network. This confidence is generated based on the process of the network reconstructing the second shielded bone tissue medical image, and it reflects the network's confidence in the reconstruction result. Since the second shielded bone tissue medical image is obtained from the initial bone tissue medical image by increasing the shielding percentage, it simulates a more complex image damage situation. Therefore, the level of reconstruction confidence not only reflects the network's ability to handle simple damage situations, but also reflects the network's robustness in dealing with more complex damage situations.

[0081] In order to evaluate the effectiveness of the reconstruction confidence, the computer system compares it with the second reference bone tissue medical image. The second reference bone tissue medical image refers to the partial area of ​​the initial bone tissue medical image corresponding to the second shielded bone tissue medical image, which provides the original content of the shielded area. By comparing the difference between the reconstructed medical image and the second reference bone tissue medical image, the accuracy of the reconstruction confidence can be further verified. In the actual application scenario, if the reconstruction confidence is high, it means that the image reconstruction restoration neural network can accurately reconstruct the shielded area according to the target image embedding descriptor, and the reconstruction result is highly consistent with the original content. This shows that the image embedding descriptor contains enough information to guide the network to accurately reconstruct the image, and it also shows that the image embedding neural network effectively retains the key features of the original image during the encoding process. On the contrary, if the reconstruction confidence is low, it means that the network encountered difficulties in the reconstruction process, and the reconstruction result is quite different from the original content. This may be because the image embedding descriptor has lost some key information, or the network has insufficient performance when dealing with complex damage. In this case, the computer system can identify potential problems based on the reconstruction confidence and further optimize and improve the image embedding neural network. For example, suppose there is an X-ray showing a fracture as the initial bone tissue medical image, and a second shielded bone tissue medical image is obtained by increasing the shielding percentage, in which most of the fracture site is shielded. Then, the target image embedding descriptor and the second shielded bone tissue medical image are input into the image reconstruction restoration neural network. The network attempts to reconstruct the shielded fracture site and outputs a reconstruction confidence. If this confidence is high (such as close to 1), it means that the network has successfully reconstructed the fracture site and is highly consistent with the fracture site in the original X-ray. This shows that the image embedding descriptor effectively retains the key information of the fracture feature and can be used for subsequent pathology recognition tasks. On the contrary, if the confidence is low, it means that the network has encountered difficulties in the reconstruction process, and it may be necessary to further adjust the network structure or optimize the parameters to improve the reconstruction performance.

[0082] At this time, as an implementation method, the second shielded bone tissue medical image and the second reference bone tissue medical image both include multiple pixels, each pixel in the second shielded bone tissue medical image corresponds one-to-one to each pixel in the second reference bone tissue medical image, and a pixel in the second shielded bone tissue medical image includes a reconstruction confidence for the corresponding pixel in the second reference bone tissue medical image. Then, step S300, based on the descriptor restoration results of each image restoration neural network, obtains the descriptor restoration loss of each image restoration neural network for the target image embedding descriptor, which may specifically include:

[0083] Step S310: Generate a descriptor restoration sub-loss corresponding to each pixel in the second shielded bone tissue medical image according to the reconstruction confidence of each pixel in the second shielded bone tissue medical image for the corresponding pixel in the second reference bone tissue medical image.

[0084] In the embodiment of the present application, step S310 generates a descriptor restoration sub-loss for each pixel according to the reconstruction confidence. The purpose of this step is to quantitatively evaluate the reconstruction effect of the image reconstruction restoration neural network on each pixel, so as to provide accurate feedback for subsequent adjustment of the image embedding neural network.

[0085] Specifically, in step S310, the computer system first focuses on each pixel in the second shielded bone tissue medical image. Since these pixels are attempted to be restored by the image reconstruction restoration neural network during the reconstruction process, they all have a corresponding reconstruction confidence. This reconstruction confidence is calculated by the network based on the target image embedding descriptor and the second shielded bone tissue medical image, which reflects the network's confidence in the reconstruction result of the pixel. Next, the computer system generates a corresponding descriptor restoration sub-loss based on the reconstruction confidence of each pixel. This process actually converts the reconstruction confidence into a loss value for use in the back propagation algorithm. Specifically, a loss function (such as a cross entropy loss function or a mean square error loss function) can be used to achieve this conversion. For example, if the mean square error loss function is used, the reconstruction confidence can be regarded as a predicted value, 1 (indicating complete reconstruction success) can be regarded as a true value, and then the mean square error between the predicted value and the true value is calculated as the descriptor restoration sub-loss.

[0086] It should be noted that since each pixel in the second shielded bone tissue medical image has a corresponding reconstruction confidence, there will also be a corresponding descriptor reduction sub-loss. These sub-losses reflect the difference in the reconstruction effect of the network on different pixels. For example, on the pixels of the fracture site, if the network can accurately reconstruct key features such as the fracture line, then the reconstruction confidence of these pixels will be higher and the descriptor reduction sub-loss will be smaller; conversely, if the network has deviations or errors in the reconstruction process, then the reconstruction confidence of these pixels will be lower and the descriptor reduction sub-loss will be larger.

[0087] Through step S310, the computer system can generate a set of all pixel descriptor restoration losses. This set provides basic data for calculating the total descriptor restoration loss in the subsequent step S320. At the same time, since the descriptor restoration loss of each pixel reflects the reconstruction effect of the network on the pixel, this set can also be used to further analyze the performance of the network on a specific area or feature, thereby providing valuable reference information for optimizing the network structure or parameters.

[0088] For example, suppose there is an X-ray showing a fracture as the initial bone tissue medical image, and a second shielded bone tissue medical image is obtained by increasing the shielding percentage, in which most of the fracture site is shielded. Then, the shielded area is reconstructed using the image reconstruction and restoration neural network, and the reconstruction confidence of each pixel is obtained. In step S310, a descriptor restoration sub-loss is generated for each reconstructed pixel (i.e., the pixel shielded at the fracture site) according to its reconstruction confidence. For example, if the reconstruction confidence of a pixel is 0.9 (indicating a good reconstruction effect), then its descriptor restoration sub-loss may be a small value; and if the reconstruction confidence of another pixel is 0.5 (indicating a general reconstruction effect), then its descriptor restoration sub-loss will be relatively large. In this way, the performance of the network in the reconstruction process can be comprehensively evaluated.

[0089] Step S320: superimpose the descriptor restoration sub-loss corresponding to each pixel in the second shielded bone tissue medical image to generate the descriptor restoration loss of the image reconstruction restoration neural network for the target image embedding descriptor.

[0090] In the embodiment of the present application, step S320 is a key step in calculating the performance of the image reconstruction and restoration neural network. In step S310, corresponding descriptor restoration sub-losses have been generated for each pixel in the second shielded bone tissue medical image, and these sub-losses reflect the accuracy of the network in reconstructing each pixel. In step S320, the computer system superimposes these pixel-level sub-losses to obtain a total descriptor restoration loss, which will be used to quantitatively evaluate the performance of the entire image reconstruction and restoration neural network.

[0091] Specifically, in this step, the computer system traverses each pixel in the second shielded bone tissue medical image and obtains the descriptor restoration sub-loss corresponding to each pixel. These sub-losses are calculated based on the reconstruction confidence of the pixel, reflecting the accuracy of the network in reconstructing the pixel. Since the second shielded bone tissue medical image usually contains a large number of pixels, there will be a large number of descriptor restoration sub-losses that need to be processed. Then, the computer system superimposes these pixel-level descriptor restoration sub-losses. The superposition operation can be simply understood as adding all the sub-loss values ​​to get a sum. This sum is the descriptor restoration loss of the image reconstruction and restoration neural network for the target image embedded descriptor. This loss value quantifies the accuracy of the entire network in reconstructing the second shielded bone tissue medical image, and is an important indicator for network performance evaluation.

[0092] The size of the descriptor restoration loss directly reflects the performance of the image reconstruction restoration neural network. If the loss value is small, it means that the network can accurately restore the pixel values ​​of the masked area during the reconstruction process, and the network performance is good; on the contrary, if the loss value is large, it means that there are many problems in the network reconstruction process, and the network structure and parameters need to be further optimized.

[0093] For example, suppose there is an X-ray showing a fracture as the initial bone tissue medical image, and a second shielded bone tissue medical image is obtained by increasing the shielding percentage, in which most of the fracture site is shielded. Then, the shielded area is reconstructed using the image reconstruction and restoration neural network, and the descriptor restoration sub-loss of each pixel is calculated. In step S320, the computer will superimpose these sub-losses to obtain a total descriptor restoration loss. If this loss value is small (such as less than 0.1), it means that the network performs well in reconstructing the fracture site and can accurately restore key features such as the fracture line; conversely, if the loss value is large (such as greater than 0.5), it means that there are problems in the network reconstruction process, and it is necessary to further adjust the network parameters or optimize the network structure.

[0094] As an implementation mode, the first bone tissue medical image is obtained after performing a masking operation on the initial bone tissue medical image, and the X image restoration neural networks include a core region inference restoration neural network. Then, at this time, the step S200, through the X image restoration neural networks, respectively restores and maps the target image embedding descriptor from the X descriptor restoration channels to obtain the descriptor restoration results of each image restoration neural network, may specifically include:

[0095] Step S201: Acquire the core area of ​​the initial bone tissue medical image.

[0096] In the embodiment of the present application, the core area usually contains the most critical information in the medical image, such as the lesion site, fracture line, etc., which is crucial for subsequent pathological identification and analysis. Specifically, in step S201, the computer system first performs a comprehensive analysis on the initial bone tissue medical image. This analysis process can be based on various image processing techniques and algorithms, such as edge detection, threshold segmentation, regional growth, etc., to identify different regions and features in the image. Taking fracture identification as an example, the initial bone tissue medical image may be an X-ray film that shows the patient's bone structure and potential fracture sites. In this scenario, the fracture line and the area around it are the core area. In order to extract this core area, the computer system may use an edge detection algorithm to identify the edge of the fracture line, or use a regional growth algorithm to expand the area around the fracture line until a complete core area is formed. In addition, in order to improve the accuracy and efficiency of core area extraction, the computer system can also combine the doctor's prior knowledge and annotation data to assist the identification process. For example, a doctor can manually annotate some typical fracture cases during the training phase, and these annotation data can be used as input to the machine learning model to help the model learn how to identify different types of fractures and their core areas. After extracting the core region, the computer system uses this region as input for subsequent steps, such as inputting it into the core region inference and restoration neural network for further inference and restoration. In this way, the computer system can focus more precisely on the key information in the medical image, thereby improving the accuracy and efficiency of pathology recognition.

[0097] Step S202: inputting the core region and the target image embedding descriptor into the core region inference restoration neural network.

[0098] In step S201, the computer system has successfully extracted the core area from the initial bone tissue medical image, which usually contains important information about lesions or key features. Next, in step S202, the core area is input into the core area reasoning and restoration neural network together with the target image embedding descriptor. The target image embedding descriptor is obtained by encoding the initial bone tissue medical image through the image embedding neural network, which contains the key feature information of the image. This descriptor can be regarded as a high-level feature representation that captures the overall structure and key details of the image, providing a basis for subsequent reasoning and restoration. The core area reasoning and restoration neural network is a neural network model specially designed to process core area information. It accepts the core area image and the target image embedding descriptor as input, and then uses the internal convolutional layer, pooling layer, fully connected layer and other structures to extract features, fuse information, and finally generate the reasoning confidence for each pixel in the core area.

[0099] In practical applications, this inference confidence reflects the confidence of the neural network in the inference results of each pixel in the core area. For the fracture recognition scenario, if the core area is the fracture line and its surrounding area, then the level of inference confidence reflects the accuracy of the neural network's inference of key features such as the position and shape of the fracture line.

[0100] For example, assuming that the initial bone tissue medical image is an X-ray showing a fracture, the core area extracted by step S201 includes the fracture line and its surrounding parts. Then, this core area image and the target image embedding descriptor generated by the image embedding neural network are input into the core area inference restoration neural network. Based on this information, the position and shape of the fracture line are inferred, and an inference confidence map is generated. This confidence map not only shows the network's inference results for the fracture line, but also reflects the network's confidence in the inference results.

[0101] Step S203: performing core area inference based on the target image embedding descriptor and the core area through the core area inference restoration neural network, and generating an inference confidence of the core area inference restoration neural network for each pixel in the core area.

[0102] In an embodiment of the present application, step S203 utilizes a core area reasoning and restoration neural network, combines the target image embedding descriptor and the core area image, performs accurate reasoning on the core area, and generates reasoning confidence for each pixel. First, the core area reasoning and restoration neural network is a specially trained neural network model that can accurately reason and restore the core area in medical images. During the training process, this network learns how to extract key feature information from the target image embedding descriptor and perform reasoning in combination with the core area image. The target image embedding descriptor is obtained by encoding the initial bone tissue medical image with the image embedding neural network, which contains the overall structure and key feature information of the image.

[0103] In step S203, the core area reasoning and restoration neural network receives the target image embedding descriptor and the core area image as input. Then, the target image embedding descriptor is analyzed to extract the key feature information related to the core area. These feature information may include the shape, position, angle, etc. of the fracture line, which are the basis for subsequent reasoning. Next, reasoning is performed in conjunction with the core area image. The core area image shows the specific location and morphology of the lesion or key features, which provides specific contextual information for the target image embedding descriptor. Taking into account the feature information in the target image embedding descriptor and the contextual information in the core area image, reasoning is performed on each pixel in the core area through multi-layer convolution, pooling and other operations.

[0104] During the reasoning process, an inference confidence map is generated. Each pixel value on this map represents the network's confidence in the inference result of the pixel, that is, the inference confidence. The level of inference confidence reflects the accuracy of the network's inference results for each pixel in the core area. For example, in the scene of fracture recognition, if the network infers the position and morphology of the fracture line very accurately, then the pixels in the fracture line area will have a higher inference confidence; conversely, if there is a deviation or error in the inference result, then the pixels in the corresponding area will have a lower inference confidence. Through the execution of step S203, the computer system can obtain an accurate inference confidence map, which not only shows the inference results of each pixel in the core area, but also quantifies the network's confidence in the inference results. This provides strong support for subsequent pathological identification and analysis, allowing doctors to more accurately judge the type, location and severity of lesions.

[0105] Step S204: using the inference confidence corresponding to each pixel in the core area as the descriptor restoration result of the core area inference restoration neural network.

[0106] In the embodiment of the present application, step S204 uses the inference confidence generated by the core area inference restoration neural network for each pixel in the core area as the descriptor restoration result of the network.

[0107] First, the core area is the area where the lesions or key features are located in the medical image. For bone tissue pathology recognition, it may contain key information such as fracture lines and tumors. In step S203, the core area reasoning and restoration neural network has accurately reasoned about the core area and generated a reasoning confidence for each pixel therein. This reasoning confidence reflects the network's confidence in the reasoning result of the pixel, which is a value between 0 and 1. The higher the value, the more confident the network is in the reasoning result of the pixel.

[0108] In step S204, the computer system uses these inference confidences as the descriptor restoration results of the core area inference restoration neural network. These results not only reflect the accuracy of the network's inference of the core area pixels, but also indirectly evaluate the richness and effectiveness of the information contained in the target image embedding descriptor. Because the target image embedding descriptor is the basis for the network to reason, if the descriptor contains enough information, then the network can more accurately infer the content of the core area.

[0109] For example, suppose there is an X-ray showing a fracture as the initial bone tissue medical image, and the fracture site is extracted as the core area through a series of processing. Then, the core area image and the target image are embedded in the descriptor and input into the core area reasoning and restoration neural network. The network infers each pixel of the fracture site and generates the corresponding reasoning confidence. In step S204, these reasoning confidences are output as descriptor restoration results. If the network's reasoning result for the fracture line is very accurate, then the pixels in the fracture line area will have a higher reasoning confidence; conversely, if there is a deviation or error in the reasoning result, then the pixels in the corresponding area will have a lower reasoning confidence. These descriptor restoration results are of great significance for subsequent pathological identification and analysis. Doctors can judge the type, location, severity and other information of the fracture based on these results, so as to formulate a more accurate treatment plan. At the same time, these results can also be used to evaluate the performance of the image embedding neural network, help researchers improve the network structure and parameter settings, and improve the recognition accuracy of the entire system.

[0110] At this time, as an implementation manner, the step S300, according to the descriptor restoration results of each image restoration neural network, obtains the descriptor restoration loss of each image restoration neural network for the target image embedding descriptor, which may include:

[0111] Step S301: Generate a descriptor restoration sub-loss corresponding to each pixel in the core area according to the inference confidence corresponding to each pixel in the core area.

[0112] In the embodiment of the present application, step S301 generates corresponding descriptor restoration sub-losses according to the reasoning confidence corresponding to each pixel in the core area, thereby providing accurate data support for subsequent performance evaluation and optimization. In the previous steps, the core area reasoning restoration neural network has accurately reasoned the core area and generated a reasoning confidence for each pixel therein.

[0113] In step S301, the computer system first traverses each pixel in the core area and obtains its corresponding inference confidence. Then, a descriptor restoration sub-loss is generated based on this inference confidence. This sub-loss is actually a quantitative evaluation of the inference confidence, which is used to measure the inference accuracy of the network on the pixel. Specifically, the calculation of the descriptor restoration sub-loss can be based on a variety of loss functions, such as the cross entropy loss function or the mean square error loss function. Taking the mean square error loss function as an example, assuming that the ideal value of the inference confidence is 1 (indicating completely correct inference), then the descriptor restoration sub-loss can be calculated as the mean square error between the inference confidence and 1. For example, if the inference confidence of a pixel is 0.9, then its descriptor restoration sub-loss is (0.9-1)2 =0.01; if the inference confidence of a pixel is 0.5, then its descriptor restoration loss is (0.5-1) 2 =0.25.

[0114] In this way, the computer system is able to generate a descriptor restoration sub-loss for each pixel in the core region. These sub-losses reflect the accuracy of the network in reasoning about each pixel, and the smaller the value, the more accurate the reasoning. In subsequent steps, these sub-losses can be further summarized and analyzed to evaluate the overall performance of the core region reasoning restoration neural network.

[0115] For example, suppose that in an X-ray showing a fracture, the core area includes the fracture line and its surrounding parts. The core area inference and restoration neural network infers the pixels in the fracture line area and generates corresponding inference confidences. In step S301, the descriptor restoration sub-loss of each pixel is calculated based on these inference confidences. If the network's inference of the fracture line is very accurate, then the pixels in the fracture line area will have a smaller sub-loss; conversely, if there are deviations or errors in the inference, then the pixels in the corresponding area will have a larger sub-loss. These sub-losses provide strong data support for subsequent performance evaluation and optimization.

[0116] Step S302: superimpose the descriptor restoration losses corresponding to each pixel in the core area to generate the descriptor restoration loss of the core area inference restoration neural network for the target image embedding descriptor.

[0117] In the embodiment of the present application, step S302 generates an overall descriptor restoration loss for the target image embedding descriptor by superimposing the descriptor restoration loss of each pixel in the core area. This step not only quantitatively evaluates the performance of the core area inference restoration neural network, but also provides key feedback for subsequent network optimization.

[0118] Specifically, in step S301, the computer system has generated a corresponding descriptor restoration sub-loss for each pixel in the core area. These sub-losses reflect the accuracy of the network when reasoning about each pixel and are important indicators for evaluating network performance. However, since the core area usually contains a large number of pixels, it may be difficult to fully evaluate the overall performance of the network by considering the sub-loss of each pixel alone. Therefore, the role of step S302 is to summarize these sub-losses and generate a descriptor restoration loss so as to more intuitively evaluate the network performance. In step S302, the computer system first traverses each pixel in the core area to obtain its corresponding descriptor restoration sub-loss. Then, these sub-losses are superimposed, that is, all sub-loss values ​​are added together to obtain a sum. This sum is the descriptor restoration loss of the core area reasoning restoration neural network for the target image embedding descriptor. Since this loss value is the sum of all pixel sub-losses, it can fully reflect the overall performance of the network when reasoning about the core area.

[0119] For example, suppose that in an X-ray showing a fracture, the core area includes the fracture line and its surrounding parts. The core area inference and restoration neural network infers the pixels in the fracture line area and generates corresponding descriptor restoration sub-losses. In step S302, these sub-losses are superimposed to obtain a descriptor restoration loss value. The smaller this value is, the more accurate the network is in inferring the fracture line area; conversely, if the loss value is large, it means that there are many problems in the network inference process and further optimization is needed. Through the operation of step S302, the computer system can accurately evaluate the performance of the core area inference and restoration neural network, and provide valuable feedback for subsequent network optimization. This is of great significance for improving the accuracy and efficiency of bone tissue pathology identification.

[0120] As an implementation manner, step S100, obtaining a first bone tissue medical image, may include:

[0121] Step S110: Acquire an initial bone tissue medical image and a first shielding percentage for the initial bone tissue medical image.

[0122] In the embodiment of the present application, step S110 first acquires an initial bone tissue medical image, and then determines a first shielding percentage for the image.

[0123] The computer system retrieves one or more initial bone tissue medical images from the medical image database. These images are usually obtained through medical imaging equipment such as X-rays, CT scans, MRIs, etc., and contain detailed information about the patient's bone tissue. For the task of bone tissue pathology recognition, these initial images may contain a variety of pathological features such as fractures, osteoporosis, and bone tumors. Taking fracture recognition as an example, the initial bone tissue medical image may be an X-ray showing the fracture site. This X-ray clearly shows the location and shape of the fracture line and the condition of the surrounding bone tissue, providing important visual information for subsequent pathology recognition.

[0124] After acquiring the initial bone tissue medical image, the computer system determines a first shielding percentage for the image. This percentage represents the proportion of the area in the image that needs to be shielded (i.e., simulated information loss or damage). The process of determining the shielding percentage is usually based on experimental experience, the importance of pathological features, and the needs of model training. Continuing with the example of fracture recognition, suppose that you want to simulate the partial information loss in the fracture X-ray during the training process. In this case, a lower first shielding percentage (such as 10%) can be set to introduce a certain degree of information loss while retaining most of the fracture features. By doing so, the performance of the image embedding neural network under the condition of partial information loss can be evaluated, thereby improving its robustness in real application scenarios. In actual operation, the determination of the first shielding percentage may also need to consider other factors, such as the complexity of the pathological features, the training progress of the model, etc. By setting the shielding percentage reasonably, the computer system can generate training data that is closer to the actual application scenario, thereby improving the accuracy and reliability of bone tissue pathology recognition.

[0125] Step S120: Shielding the initial bone tissue medical image based on the first shielding percentage to generate the first bone tissue medical image; wherein the first bone tissue medical image includes a first shielded bone tissue medical image obtained by shielding a partial image area of ​​the initial bone tissue medical image with the first shielding percentage.

[0126] In the embodiment of the present application, step S120 performs a shielding operation on the initial bone tissue medical image to simulate the situation where image information is missing or damaged in actual applications. The purpose of this step is to generate a first bone tissue medical image for training the image embedding neural network, thereby enhancing the network's ability to process incomplete image data. Specifically, in step S120, the computer system first determines how much proportion of the area in the initial bone tissue medical image needs to be shielded according to a preset first shielding percentage. This percentage is usually determined according to the specific application scenario and the importance of the pathological characteristics. For example, in the scene of fracture identification, since the position and morphology of the fracture line are the key to diagnosis, it may be necessary to set a relatively low shielding percentage to ensure that the fracture features are still visible in the shielded image. Next, the computer system performs a shielding operation on the initial bone tissue medical image. This operation is usually implemented by random selection or an algorithm based on specific rules to ensure that the distribution of the shielded area in the image is representative and diverse. The specific implementation of the shielding operation may include setting the pixel value of the selected area to 0 (black) or replacing it with a specific shielding symbol (such as a gray square) to simulate different degrees of image damage.

[0127] After the masking operation, the computer system generates a first bone tissue medical image, also called a first masked bone tissue medical image. Compared with the initial bone tissue medical image, the information of some areas of this image is masked or damaged, but the key pathological features are still retained. Such image data is very valuable for training image embedding neural networks because it can simulate image quality problems that may be encountered in practical applications and prompt the network to learn the ability to extract key information from incomplete data.

[0128] For example, suppose there is an X-ray showing a fracture as the initial bone tissue medical image, and a first shielding percentage of 20% is set. In step S120, the computer system randomly selects 20% of the area in the X-ray for shielding. These shielded areas may include part of the fracture line, surrounding bone tissue or other irrelevant areas. After the shielding operation, a new X-ray is obtained, namely the first bone tissue medical image. Although the fracture line in this image is partially shielded, it is still clearly visible and can be used to train the image embedding neural network to identify fracture features.

[0129] As an embodiment, the method further includes:

[0130] Step S10: Reconstruct the first shielded bone tissue medical image based on the first bone tissue medical image through the image embedding neural network to generate a reconstruction confidence of the first shielded bone tissue medical image for the first reference bone tissue medical image; the first reference bone tissue medical image is a partial image area in the initial bone tissue medical image corresponding to the first shielded bone tissue medical image.

[0131] In the embodiment of the present application, step S10 is intended to evaluate the performance of the image embedding neural network by image reconstruction. In this step, the computer system uses the image embedding neural network to reconstruct the first bone tissue medical image after the masking process, and generates a reconstruction confidence to quantify the ability of the network to recover the original image information from the embedded descriptor.

[0132] Specifically, in step S10, the computer system first acquires a first bone tissue medical image, which is obtained by performing a masking operation on the initial bone tissue medical image. The masking operation simulates the situation where image information is missing or damaged in actual applications, such as blurring or occlusion of some areas in the X-ray film due to patient movement or equipment failure. Then, the computer system inputs the first bone tissue medical image into the trained image embedding neural network. The image embedding neural network is responsible for encoding the input medical image into a high-dimensional feature vector (i.e., image embedding descriptor), which contains key information of the image, such as lesion characteristics, structural characteristics, etc. Then, in order to evaluate the information retention ability of the network during the encoding process, an attempt is made to reconstruct the original medical image from the encoded descriptor. This process is implemented by a decoder part of the network (or an independent image restoration neural network). The decoder generates a new medical image based on the encoded descriptor, which should be as close to the original first bone tissue medical image as possible. However, since some areas in the first bone tissue medical image are masked, the reconstructed image will also be missing or blurred in these areas. In order to quantify the degree of this missing or blurred, the concept of reconstruction confidence is introduced. The reconstruction confidence is calculated by comparing the reconstructed image with the unmasked area in the original image (i.e., the first reference bone tissue medical image). Specifically, the similarity or difference between the reconstructed image and the reference image can be calculated to obtain a quantitative index as the reconstruction confidence. The higher the confidence, the closer the reconstructed image is to the original image, and the better the performance of the neural network in the embedding and reconstruction process.

[0133] Step S20: Obtain the first bone tissue medical image reconstruction loss of the image embedded in the neural network according to the reconstruction confidence of the first shielded bone tissue medical image with respect to the first reference bone tissue medical image.

[0134] In the embodiment of the present application, step S20 calculates the first bone tissue medical image reconstruction loss of the image embedding neural network based on the reconstruction confidence generated in step S10. The purpose of this step is to quantitatively evaluate the performance of the image embedding neural network in the encoding and reconstruction process, thereby providing valuable feedback for network optimization.

[0135] Specifically, in step S20, the computer system obtains the reconstruction confidence generated in step S10. This confidence is calculated by comparing the reconstructed first shielded bone tissue medical image with the original first reference bone tissue medical image (i.e., the unshielded portion of the initial bone tissue medical image), which reflects the similarity or difference between the reconstructed image and the original image. Next, the computer system calculates the reconstruction loss of the first bone tissue medical image based on this reconstruction confidence. This loss value is actually a quantitative evaluation of the performance of the network during the encoding and reconstruction process. The choice of loss function is usually based on specific application scenarios and requirements, such as mean square error (MSE), structural similarity index (SSIM), etc. These functions can quantify the difference between the reconstructed image and the original image, thereby deriving a numerical value as the reconstruction loss.

[0136] Taking the mean square error (MSE) as an example, if the pixel value difference between the reconstructed image and the original image is large, the calculated MSE value will also be large, indicating a high reconstruction loss; conversely, if the difference between the two is small, the MSE value will also be small, indicating a low reconstruction loss. This means that the network retains more original information during the encoding and reconstruction process, and has better performance.

[0137] In one example, when calculating the first bone tissue medical image reconstruction loss, calculation formulas based on the two loss functions are given below.

[0138] The mean square error is a commonly used loss function that measures the pixel-level difference between the reconstructed image and the original reference image. Its calculation formula is as follows:

[0139] ;

[0140] Where: M and N are the width and height of the image respectively. I(i,j) is the pixel value of the original reference image (the first reference bone tissue medical image) at position (i, j). is the pixel value of the reconstructed image (the image obtained after reconstructing the first shielded bone tissue medical image) at position (i, j).

[0141] Another feasible calculation method is the structural similarity index (SSIM). Although SSIM itself is not a loss function, it is often used to measure image quality and can be converted into a loss function in some way. The calculation of SSIM involves three aspects: brightness, contrast, and structure. A simplified SSIM calculation formula is as follows:

[0142] ;

[0143] Where: x and y represent the local windows of the original reference image and the reconstructed image respectively; and is the mean of x and y; and is the variance of x and y; is the covariance of x and y; c1 and c2 are preset constants used to avoid division by zero errors and ensure stability.

[0144] As a loss function, it can be: L=1-SSIM(x, y).

[0145] It can be understood that the SSIM here is calculated locally, and usually a window is slid across the entire image, and the average SSIM values ​​of all windows are calculated. In practical applications, the choice of which loss function depends on the specific application scenario and requirements. MSE focuses more on pixel-level differences, while SSIM focuses more on the structural similarity of images. The calculation results of step S20 are of great significance for evaluating the robustness and generalization ability of image embedding neural networks. Since medical images may be affected by various factors (such as patient movement, equipment failure, etc.) in practical applications, some information may be missing or damaged. Therefore, a well-performing image embedding neural network should be able to accurately encode and reconstruct the original image in this case. By calculating the first bone tissue medical image reconstruction loss, the performance of the network in this case can be quantitatively evaluated, thereby guiding the further optimization and improvement of the network.

[0146] At this time, as an implementation mode, the step S400, adjusting the network parameters of the image embedding neural network according to the obtained X descriptor restoration losses to obtain the debugged image embedding neural network, may include:

[0147] Step S410: summing the first bone tissue medical image reconstruction loss and the X descriptor restoration losses to generate a target debugging loss;

[0148] Step S420: adjusting the network parameters of the image embedding neural network according to the target debugging loss to obtain the debugged image embedding neural network.

[0149] In step S410, the computer system obtains the reconstruction loss of the first bone tissue medical image and the X descriptor restoration losses calculated by multiple image restoration neural networks. The reconstruction loss of the first bone tissue medical image is obtained by reconstructing and calculating the initial bone tissue medical image using an image embedding neural network after performing a masking operation, and it reflects the performance of the network in the reconstruction process. The descriptor restoration loss is obtained by comparing the difference between the output of the image restoration neural network and the original image or the reference image, and is used to evaluate the accuracy of the network in the encoding and restoration process. Then, the computer system performs a summation operation on these losses to generate a target debugging loss. This loss value comprehensively considers the performance of the network in both reconstruction and restoration, and is a key indicator for subsequent optimization of network parameters. The specific implementation method of the summation operation can be a simple numerical addition, or a weighted summation based on the network structure and training requirements.

[0150] In step S420, the computer system adjusts the network parameters of the image embedding neural network according to the target debugging loss generated in step S410. This process is usually implemented by a back propagation algorithm, which uses gradient descent or other optimization algorithms to update network parameters such as weights and biases to minimize the target debugging loss. Specifically, the back propagation algorithm first calculates the gradient of the target debugging loss with respect to the network output layer, and then back propagates these gradients layer by layer to the first few layers of the network to calculate the gradient of the parameters of each layer. Then, the network parameters are updated according to these gradient values ​​so that the output of the network is closer to the desired result. This process is iterated multiple times until the target debugging loss converges to a smaller value or reaches a preset number of training rounds. Through the adjustment process of step S420, the performance of the image embedding neural network is optimized. The network can better retain the key information in the original medical image and improve the accuracy of the encoding. This is crucial for subsequent pathology recognition tasks because it directly affects the accuracy and reliability of recognition.

[0151] As an implementation manner, the step S410 of summing the first bone tissue medical image reconstruction loss and the X descriptor restoration losses to generate a target debugging loss may specifically include:

[0152] Step S411: Obtain the weights of each image restoration neural network for the image embedding neural network respectively.

[0153] Specifically, in step S411, the computer system obtains the weight of each image restoration neural network to the image embedding neural network. These weights are determined based on the importance and contribution of each image restoration neural network in evaluating the performance of the image embedding neural network. The determination of weights can be based on a variety of factors, such as the structure, function, and application scenario requirements of the network. In practical applications, these weights can be determined through expert experience, experimental verification, or machine learning methods. For example, if an image restoration neural network performs well in identifying specific types of bone tissue pathological features, then its weight for the image embedding neural network should be relatively high. On the contrary, if an image restoration neural network performs averagely or is not very important for a specific task, then its weight should be relatively low.

[0154] Taking bone tissue pathology recognition as an example, suppose there are two image restoration neural networks: network A and network B. Network A focuses on restoring global structural features, such as the overall morphology and layout of bone tissue; while network B focuses more on restoring local detail features, such as the clarity and location of fracture lines. In this case, weights can be assigned to the two networks according to their importance in identifying different pathological features. If global structural features are more critical for diagnosis, the weight of network A should be relatively high; conversely, if local detail features are more important, the weight of network B should be higher. For example, suppose that after experimental verification and expert evaluation, it is decided to assign a weight of 0.6 to network A and a weight of 0.4 to network B. This means that when calculating the target debugging loss, the descriptor restoration loss of network A will be given a larger weight to reflect its importance in evaluating the performance of the image embedding neural network. In this way, the impact of different image restoration neural networks on the performance of the image embedding neural network can be more comprehensively considered, thereby improving the accuracy and reliability of the entire pathology recognition system.

[0155] Step S412: Based on the weight of each image restoration neural network for the image embedding neural network, weight the descriptor restoration loss of each image restoration neural network for the target image embedding descriptor, and generate a weighted debugging loss of each image restoration neural network for the image embedding neural network.

[0156] In an embodiment of the present application, step S412 performs weighted processing on the descriptor restoration loss according to the weight of each image restoration neural network to the image embedding neural network. The purpose of this step is to more accurately reflect the importance of different image restoration neural networks in evaluating the performance of the image embedding neural network when calculating the target debugging loss. Specifically, in step S412, the computer system first obtains the weight of each image restoration neural network for the image embedding neural network. These weights are determined based on factors such as the design purpose of the network, the requirements of the application scenario, and experimental verification, reflecting the importance and contribution of different networks in the evaluation process. Then, for each image restoration neural network, its corresponding descriptor restoration loss is multiplied by the weight of the network to generate a weighted debugging loss. This process is actually a weighted processing of the descriptor restoration loss, so that the impact of different networks on the performance of the image embedding neural network is reasonably quantified.

[0157] For example, suppose there are two image restoration neural networks: Network A and Network B. Network A focuses on restoring global structural features, such as the overall morphology and layout of bone tissue, while Network B focuses more on restoring local detail features, such as the clarity and location of fracture lines. Based on the requirements of the application scenario and the results of experimental verification, it is decided to assign a weight of 0.6 to Network A and a weight of 0.4 to Network B.

[0158] Now, assume that for a certain target image embedding descriptor, the descriptor restoration loss of network A is 0.1, and the descriptor restoration loss of network B is 0.2. In step S412, the computer system performs weighted processing on these two losses. Specifically, the weighted debugging loss of network A is 0.1*0.6=0.06, and the weighted debugging loss of network B is 0.2*0.4=0.08. Through such weighted processing, two weighted debugging losses are obtained, which respectively reflect the contribution of network A and network B in evaluating the performance of image embedding neural network. These two weighted debugging losses will be combined with the first bone tissue medical image reconstruction loss in subsequent steps to generate a comprehensive target debugging loss, which is used to guide the parameter update and optimization of the image embedding neural network.

[0159] This weighted processing method enables the influence of different image restoration neural networks on the performance of image embedding neural networks to be reasonably quantified, thereby improving the accuracy and reliability of the entire pathology recognition system.

[0160] Step S413: superimposing the first bone tissue medical image reconstruction loss and the X weighted debugging losses of the X image restoration neural networks for the image embedding neural network to generate the target debugging loss.

[0161] In the embodiment of the present application, step S413 integrates the loss values ​​from different sources into a unified loss function to generate a key step of the target debugging loss. The computer system first obtains the reconstruction loss of the first bone tissue medical image, which is obtained by inputting the first bone tissue medical image after shielding into the image embedding neural network for reconstruction, and comparing the difference between the reconstruction result and the original medical image. It reflects the performance of the image embedding neural network in the reconstruction process. Then, X weighted debugging losses of X image restoration neural networks for the image embedding neural network are also obtained. These weighted debugging losses are obtained by inputting the target image embedding descriptor generated by the image embedding neural network into different image restoration neural networks, calculating the descriptor restoration loss, and weighting according to the importance of each network. They reflect the performance of the image embedding neural network in the encoding process, especially in terms of retaining key information. After obtaining these loss values, they are superimposed to generate the target debugging loss. The specific implementation method of the superposition operation is to add all the loss values ​​to obtain a sum. This sum is the target debugging loss, which combines the performance of the image embedding neural network in both reconstruction and encoding.

[0162] Taking the scene of bone tissue pathology recognition as an example, assume that there are two image restoration neural networks (network A and network B), and the reconstruction loss of the first bone tissue medical image has been calculated to be 0.05, the weighted debugging loss of network A is 0.06, and the weighted debugging loss of network B is 0.08. In step S413, the computer will superimpose these loss values: target debugging loss = 0.05 (reconstruction loss) + 0.06 (weighted debugging loss of network A) + 0.08 (weighted debugging loss of network B) = 0.19.

[0163] By generating a target debugging loss, a comprehensive metric can be obtained to evaluate the performance of the image embedding neural network. This loss function not only takes into account the performance of the network during reconstruction, but also the ability of the network to retain key information during encoding. In the subsequent parameter update and optimization process, the target debugging loss will be minimized, and the parameters of the image embedding neural network will be updated through the back propagation algorithm to improve its encoding accuracy and provide better support for bone tissue pathology recognition tasks.

[0164] In one implementation, the training process of the image embedding neural network may further include the following steps S1 to S5:

[0165] Step S1: Obtain a bone tissue medical image library and embed an initial image into a neural network; the bone tissue medical image library contains Y full bone tissue medical images under multiple full bone tissue pathologies; Y≥1.

[0166] In the embodiment of the present application, regarding the acquisition of the bone tissue medical image library, the computer system retrieves medical images containing multiple bone tissue pathology types from the medical image database. These images are usually obtained through medical imaging equipment such as X-rays, CT scans, MRI, etc., and they record the structure and pathological characteristics of the patient's bone tissue in detail. A typical bone tissue medical image library may contain hundreds or thousands of full-volume bone tissue medical images, which cover a variety of common bone tissue pathological conditions such as fractures, osteoporosis, and bone tumors. Each full-volume bone tissue medical image is complete and contains detailed information about the lesion site and its surrounding tissues, providing rich data support for subsequent neural network training.

[0167] Taking fracture recognition as an example, the bone tissue medical image library contains a large number of X-rays showing different types and degrees of fractures. These X-rays cover a wide range of situations from minor cracks to complex comminuted fractures, providing a comprehensive data sample for training image embedding neural networks to recognize fracture features.

[0168] The initial image embedding neural network is a pre-designed neural network model used to encode the input medical image into a high-dimensional feature vector (i.e., image embedding descriptor). It can use a convolutional neural network (CNN) or other neural network structures suitable for image processing. Before training begins, this initial network is usually randomly initialized, that is, its parameters such as weights and biases are assigned a set of random initial values.

[0169] After obtaining the bone tissue medical image library and the initial image embedding neural network, the computer system can begin the subsequent training process. Through continuous iteration and optimization, the initial image embedding neural network will gradually learn how to extract key information from medical images and provide strong support for bone tissue pathology identification.

[0170] Step S2: performing a shielding operation on each full bone tissue medical image in the bone tissue medical image library to generate Y shielded full bone tissue medical images; a shielded full bone tissue medical image includes a full shielded bone tissue medical image obtained by shielding a partial image area in the corresponding full bone tissue medical image.

[0171] In the embodiment of the present application, step S2 performs a shielding operation on each full bone tissue medical image in the bone tissue medical image library to simulate the situation where image information is missing or damaged in actual applications. The purpose of this step is to enhance the robustness of the image embedding neural network to incomplete images and improve its performance in real scenarios. Specifically, in step S2, the computer system first traverses each full bone tissue medical image in the bone tissue medical image library. These full images contain complete bone tissue structures and pathological features, and are an important data basis for training neural networks. Then, for each full image, a shielding operation is performed. This operation is essentially to randomly or selectively cover or replace a part of the image area to generate a new full bone tissue medical image with some information shielded, that is, a full shielded bone tissue medical image. The specific implementation method of the shielding operation can be adjusted according to actual needs. For example, a random shielding method can be used, that is, a certain proportion of pixels in the image can be randomly selected for shielding; a pathological feature-based shielding method can also be used, that is, targeted shielding is performed according to the location and range of the pathological features. Regardless of which method is used, the purpose of the shielding operation is to simulate the situation where image information is missing or damaged that may occur in actual applications.

[0172] Taking fracture recognition as an example, suppose there is an X-ray showing a fracture as a full-volume bone tissue medical image. In step S2, this X-ray may be shielded to simulate the situation where the fracture site is blocked or blurred. Specifically, a portion of the area around the fracture site may be randomly selected for shielding, or the pixel values ​​of these areas may be replaced with specific shielding symbols. After such processing, a new full-volume shielded bone tissue medical image is obtained, which contains part of the information of the fracture characteristics, but also lacks some key information. By shielding each full-volume image in the bone tissue medical image library, the computer system generates a data set containing multiple full-volume shielded bone tissue medical images. This data set will be used in the subsequent training process of the image embedding neural network to help the network learn how to extract key information from incomplete images and improve its performance in real scenarios.

[0173] Step S3: Reconstruct the covered full-scale shielded bone tissue medical images according to the full-scale bone tissue medical images after each shielding operation through the initial image embedding neural network, and generate a reconstruction confidence of each full-scale shielded bone tissue medical image for the corresponding full-scale reference bone tissue medical image; the full-scale reference bone tissue medical image corresponding to a full-scale shielded bone tissue medical image is the original partial image area of ​​the full-scale shielded bone tissue medical image in the corresponding full-scale bone tissue medical image.

[0174] In the embodiment of the present application, step S3 reconstructs the full bone tissue medical image after the shielding operation and generates a reconstruction confidence. This step not only tests the encoding and decoding capabilities of the initial image embedding neural network for incomplete medical images, but also provides key feedback information for subsequent network optimization.

[0175] Specifically, in step S3, the computer system first obtains the full-masked bone tissue medical image after the masking operation. These images are obtained by randomly or selectively masking some areas in the original full-masked bone tissue medical image, simulating the situation where image information is missing or damaged in actual applications. Then, the initial image embedding neural network is used to reconstruct these full-masked bone tissue medical images. This process is actually a test of the network encoding and decoding capabilities. The initial image embedding neural network first encodes the full-masked bone tissue medical image into a high-dimensional feature vector (i.e., image embedding descriptor), which contains the key information of the image. Then, try to decode the original medical image from this feature vector. Since the input image is incomplete, this decoding process is actually simulating the scene of restoring the original image from incomplete information. After the restoration is completed, a reconstruction confidence is generated. This confidence is calculated by comparing the reconstructed image with the unmasked part of the original image (i.e., the full reference bone tissue medical image). Specifically, the similarity or difference between the reconstructed image and the reference image can be calculated to obtain a quantitative index as the reconstruction confidence. The higher the confidence level, the closer the reconstructed image is to the original image, and the better the network performance in the encoding and reconstruction process.

[0176] Step S4: generating a second bone tissue medical image reconstruction loss for embedding the initial image into a neural network based on the reconstruction confidence of each full-scale shielded bone tissue medical image with respect to the corresponding full-scale reference bone tissue medical image.

[0177] In the embodiment of the present application, step S4 generates a second bone tissue medical image reconstruction loss according to the reconstruction confidence. Specifically, in step S4, the computer system first obtains the reconstruction confidence of each full-masked bone tissue medical image for its corresponding full-masked reference bone tissue medical image. This reconstruction confidence is calculated by comparing the reconstructed image with the unmasked part of the original image (i.e., the full-masked reference bone tissue medical image), which reflects the ability of the network to restore the original image from incomplete information.

[0178] Next, the computer system generates a second bone tissue medical image reconstruction loss based on these reconstruction confidences. This loss is actually a quantitative evaluation of the performance of the network during the encoding and reconstruction process. The choice of loss function is usually based on specific application scenarios and requirements. Commonly used loss functions include mean square error (MSE), structural similarity index (SSIM), etc. These functions can quantify the difference between the reconstructed image and the original image, thereby deriving a numerical value as the reconstruction loss. Taking the mean square error (MSE) as an example, if the pixel value difference between the reconstructed image and the original image is large, then the calculated MSE value will also be large, indicating that the reconstruction loss is high; conversely, if the difference between the two is small, the MSE value will also be small, indicating that the reconstruction loss is low. This means that the network retains more original information during the encoding and reconstruction process and has better performance.

[0179] By calculating the second bone tissue medical image reconstruction loss, the computer system can quantitatively evaluate the performance of the initial image embedding neural network and provide valuable feedback for subsequent network optimization. This is of great significance for improving the accuracy and reliability of bone tissue pathology recognition.

[0180] Step S5: adjusting the network parameters of the initial image embedding neural network according to the second bone tissue medical image reconstruction loss to obtain the image embedding neural network.

[0181] In the embodiment of the present application, step S5 adjusts the network parameters of the initial image embedding neural network based on the second bone tissue medical image reconstruction loss, thereby obtaining an optimized image embedding neural network.

[0182] In step S4, by comparing the reconstructed image with the unshielded part of the original image (i.e., the full reference bone tissue medical image), the reconstruction confidence is calculated, and the reconstruction loss is generated based on this. This loss value actually reflects the performance of the initial image embedding neural network in the encoding and reconstruction process, that is, how much original image information it can accurately restore from the encoded feature vector. Next, in step S5, the computer system uses the reconstruction loss to adjust the network parameters of the initial image embedding neural network. This process is usually implemented by a back propagation algorithm. Specifically, the algorithm first calculates the gradient of the reconstruction loss with respect to the network output layer, and then back-propagates these gradients layer by layer to the first few layers of the network to calculate the gradient of the parameters of each layer. Then, the network's weights, biases and other parameters are updated according to these gradient values, so that the network's output is closer to the desired result. This process will be iterated multiple times until the reconstruction loss converges to a smaller value or reaches a preset number of training rounds.

[0183] Taking the scene of bone tissue pathology recognition as an example, suppose there is an X-ray showing a fracture as the original image. After the masking operation, part of the fracture site is masked. During the training process, the initial image embedding neural network attempts to reconstruct the complete X-ray from this masked image and generates a reconstruction loss. If the reconstruction loss is high, it means that there are large errors in the network during the encoding and reconstruction process, and key information such as fracture characteristics cannot be accurately restored. At this time, the network parameters will be adjusted according to this reconstruction loss, so that the network can better learn and optimize in subsequent iterations.

[0184] Through the adjustment process in step S5, the performance of the initial image embedding neural network has been improved. It can more accurately extract key information from medical images and maintain high encoding accuracy even when information is missing or damaged. This is crucial for subsequent bone tissue pathology recognition tasks because it directly affects the accuracy and reliability of recognition.

[0185] In one implementation, the method may further include a process of pre-generating an image embedding descriptor set, specifically including the following steps:

[0186] Step S1000: Acquire a set of matched bone tissue medical images; the set of matched bone tissue medical images includes a plurality of matched bone tissue medical images in a pathology matching process;

[0187] Step S2000: performing an image embedding operation on each matching bone tissue medical image in the matching bone tissue medical image set through the debugged image embedding neural network to generate a dense image embedding descriptor for each matching bone tissue medical image;

[0188] Step S3000: Generate an image embedding descriptor set based on the dense image embedding descriptor of each matching bone tissue medical image.

[0189] In step S1000, the computer system obtains a set of matched bone tissue medical images. This set contains multiple matched bone tissue medical images in the pathology matching process, and these images are usually screened from a large number of medical images based on certain matching criteria (such as pathology type, degree of lesion, etc.). These matched images represent typical cases under different pathological characteristics, providing a rich data basis for the subsequent generation of image embedding descriptors. Taking fracture recognition as an example, the set of matched bone tissue medical images may contain X-rays of fracture cases of various types and degrees. These X-rays cover a variety of situations from minor cracks to complex comminuted fractures, providing a comprehensive data sample for the subsequent generation of image embedding descriptors.

[0190] After obtaining a set of matching bone tissue medical images, in step S2000, the computer system uses the debugged image embedding neural network to perform image embedding operations on each matching image. This process actually encodes each matching image into a high-dimensional feature vector, namely a dense image embedding descriptor. This descriptor contains key information of the matching image, such as lesion characteristics, structural characteristics, etc., providing a powerful feature representation for subsequent pathological identification. In the scenario of fracture identification, the image embedding neural network will encode the X-ray of each fracture case into a dense image embedding descriptor. This descriptor contains key information such as the shape, position, angle, etc. of the fracture site, providing rich feature support for subsequent fracture identification.

[0191] Finally, after obtaining the dense image embedding descriptors of each matching bone tissue medical image, the computer system integrates these descriptors into an image embedding descriptor set. This set contains the feature representations of all matching images and provides comprehensive feature support for subsequent pathology recognition tasks. In the process of generating the image embedding descriptor set, the computer system can also perform normalization, dimensionality reduction and other processing on each descriptor to improve its applicability and performance in subsequent tasks. Taking fracture recognition as an example, the generated image embedding descriptor set will contain dense image embedding descriptors for all fracture cases. This set provides comprehensive feature support for subsequent fracture recognition tasks, enabling the computer system to more accurately identify fracture cases and determine their type and degree.

[0192] Figure 2 A hardware entity diagram of a computer system provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the hardware entity of the computer system 1000 includes: a processor 1001 and a memory 1002, wherein the memory 1002 stores a computer program that can be run on the processor 1001, and the processor 1001 implements the steps in the method of any of the above embodiments when executing the program.

Claims

1. A method for identifying bone tissue pathology, characterized in that: The method comprises: Acquire a first bone tissue medical image, and perform an image embedding operation on the first bone tissue medical image through an image embedding neural network to generate a target image embedding descriptor of the first bone tissue medical image; Using X image restoration neural networks to restore and map the target image embedding descriptor from X descriptor restoration channels, respectively, to obtain a descriptor restoration result of each image restoration neural network; X≥1, an image restoration neural network is used to restore and map the target image embedding descriptor in a corresponding descriptor restoration channel; According to the descriptor restoration results of each image restoration neural network, obtaining the descriptor restoration loss of each image restoration neural network for the target image embedding descriptor; According to the obtained X descriptor restoration losses, the network parameters of the image embedding neural network are adjusted to obtain a debugged image embedding neural network; the debugged image embedding neural network is used to generate a dense image embedding descriptor of the bone tissue medical image to be processed; Acquire a bone tissue medical image to be identified uploaded by a bone tissue terminal identification device, and perform an image embedding operation on the bone tissue medical image to be identified through the debugged image embedding neural network to generate a dense image embedding descriptor of the bone tissue medical image to be identified; Obtaining descriptor commonality measurement results between the dense image embedding descriptor of the bone tissue medical image to be identified and each dense image embedding descriptor in the image embedding descriptor set; According to the descriptor commonality measurement results between the dense image embedding descriptor of the bone tissue medical image to be identified and each dense image embedding descriptor in the image embedding descriptor set, a matching image embedding descriptor is selected from the image embedding descriptor set; Determine the matching bone tissue medical image to which the matching image embedding descriptor belongs as the matching bone tissue medical image of the bone tissue medical image to be identified, and send the matching bone tissue medical image of the bone tissue medical image to be identified to the bone tissue terminal identification device, so that the bone tissue terminal identification device outputs the matching bone tissue medical image of the bone tissue medical image to be identified and the pathological information contained in the matching bone tissue medical image; Wherein, the first bone tissue medical image is obtained after performing a shielding operation on the initial bone tissue medical image, and the X image restoration neural networks include a core area inference restoration neural network; The method of restoring and mapping the target image embedding descriptors from the X descriptor restoration channels through the X image restoration neural networks to obtain the descriptor restoration results of each image restoration neural network includes: Acquiring a core area of ​​the initial bone tissue medical image; Inputting the core region and the target image embedding descriptor into the core region inference restoration neural network; Performing core region inference based on the target image embedding descriptor and the core region through the core region inference restoration neural network, and generating inference confidence of the core region inference restoration neural network for each pixel in the core region; Using the inference confidence corresponding to each pixel in the core area as the descriptor restoration result of the core area inference restoration neural network; The step of obtaining the descriptor restoration loss of each image restoration neural network for the target image embedding descriptor according to the descriptor restoration result of each image restoration neural network comprises: Generating a descriptor restoration sub-loss corresponding to each pixel in the core area according to the inference confidence corresponding to each pixel in the core area; The descriptor restoration losses corresponding to each pixel in the core area are superimposed to generate the descriptor restoration loss of the core area inference restoration neural network for the target image embedding descriptor.

2. The method according to claim 1, characterized in that The selecting a matching image embedding descriptor from the image embedding descriptor set according to the descriptor commonality measurement results between the dense image embedding descriptor of the bone tissue medical image to be identified and each dense image embedding descriptor in the image embedding descriptor set includes: According to the decreasing direction of the descriptor commonality measurement result between the dense image embedding descriptor of the bone tissue medical image to be identified and each dense image embedding descriptor in the image embedding descriptor set, the dense image embedding descriptors in the image embedding descriptor set are arranged in sequence to obtain arranged dense image embedding descriptors; Determine the first A dense image embedding descriptors among the arranged dense image embedding descriptors as the matching image embedding descriptors; A≥1; Or, the method of selecting a matching image embedding descriptor from the image embedding descriptor set based on the descriptor commonality measurement results between the dense image embedding descriptor of the bone tissue medical image to be identified and each dense image embedding descriptor in the image embedding descriptor set includes: Obtaining a commonality measurement threshold for descriptor commonality measurement results between dense image embedding descriptors; The dense image embedding descriptor in the image embedding descriptor set whose descriptor commonality measurement result with the dense image embedding descriptor of the bone tissue medical image to be identified is not less than the commonality measurement threshold is determined as the matching image embedding descriptor.

3. The method according to claim 1, characterized in that The first bone tissue medical image is obtained by performing a masking operation on the initial bone tissue medical image based on a first masking percentage, the X image restoration neural networks include an image reconstruction restoration neural network; the X image restoration neural networks are used to restore and map the target image embedding descriptors from the X descriptor restoration channels to obtain the descriptor restoration results of each image restoration neural network, including: Acquire a second bone tissue medical image; the second bone tissue medical image is obtained by shielding the initial bone tissue medical image based on a second shielding percentage, the second shielding percentage is greater than the first shielding percentage, and the second bone tissue medical image includes a second shielded bone tissue medical image obtained by shielding a partial image area of ​​the initial bone tissue medical image at the second shielding percentage; Reconstructing the second shielded bone tissue medical image through the image reconstruction restoration neural network according to the target image embedding descriptor and the second bone tissue medical image, and generating a reconstruction confidence of the second shielded bone tissue medical image for the second reference bone tissue medical image; the second reference bone tissue medical image is a partial image area corresponding to the second shielded bone tissue medical image in the initial bone tissue medical image; Determining the reconstruction confidence of the second shielded bone tissue medical image with respect to the second reference bone tissue medical image as a descriptor restoration result of the image reconstruction restoration neural network; The second shielded bone tissue medical image and the second reference bone tissue medical image both include a plurality of pixels, each pixel in the second shielded bone tissue medical image corresponds one-to-one to each pixel in the second reference bone tissue medical image, and a pixel in the second shielded bone tissue medical image includes a reconstruction confidence for a corresponding pixel in the second reference bone tissue medical image; and obtaining the descriptor restoration loss of each image restoration neural network for the target image embedding descriptor according to the respective descriptor restoration results of each image restoration neural network, comprises: Generate a descriptor restoration sub-loss corresponding to each pixel in the second shielded bone tissue medical image according to the reconstruction confidence of each pixel in the second shielded bone tissue medical image for the corresponding pixel in the second reference bone tissue medical image; The descriptor restoration sub-loss corresponding to each pixel in the second shielded bone tissue medical image is superimposed to generate the descriptor restoration loss of the image reconstruction restoration neural network for the target image embedding descriptor.

4. The method according to claim 1, characterized in that The step of acquiring the first bone tissue medical image comprises: Acquire an initial bone tissue medical image and a first shielding percentage for the initial bone tissue medical image; Performing a shielding operation on the initial bone tissue medical image based on the first shielding percentage to generate the first bone tissue medical image; wherein the first bone tissue medical image includes a first shielded bone tissue medical image obtained by shielding a partial image area of ​​the initial bone tissue medical image with the first shielding percentage; The method further comprises: Reconstructing the first shielded bone tissue medical image based on the first bone tissue medical image through the image embedding neural network to generate a reconstruction confidence of the first shielded bone tissue medical image with respect to a first reference bone tissue medical image; the first reference bone tissue medical image is a partial image region in the initial bone tissue medical image corresponding to the first shielded bone tissue medical image; According to the reconstruction confidence of the first shielded bone tissue medical image with respect to the first reference bone tissue medical image, the reconstruction loss of the first bone tissue medical image of the image embedded in the neural network is obtained.

5. The method according to claim 4, characterized in that The method of adjusting the network parameters of the image embedding neural network according to the obtained X descriptor restoration losses to obtain a debugged image embedding neural network includes: Summing the first bone tissue medical image reconstruction loss and the X descriptor restoration losses to generate a target debugging loss; The network parameters of the image embedding neural network are adjusted according to the target debugging loss to obtain the debugged image embedding neural network.

6. The method according to claim 5, characterized in that The step of summing the first bone tissue medical image reconstruction loss and the X descriptor restoration losses to generate a target debugging loss includes: Obtaining the weights of each image restoration neural network for the image embedding neural network respectively; Based on the weight of each image restoration neural network for the image embedding neural network, weight the descriptor restoration loss of each image restoration neural network for the target image embedding descriptor to generate a weighted debugging loss of each image restoration neural network for the image embedding neural network; The first bone tissue medical image reconstruction loss and the X weighted debugging losses of the X image restoration neural networks for the image embedding neural network are superimposed to generate the target debugging loss.

7. The method according to claim 1, characterized in that The method further comprises: Obtain a bone tissue medical image library and an initial image embedding neural network; the bone tissue medical image library contains Y full bone tissue medical images under multiple full bone tissue pathologies; Y≥1; Performing a shielding operation on each full bone tissue medical image in the bone tissue medical image library to generate Y shielded full bone tissue medical images; a shielded full bone tissue medical image includes a full shielded bone tissue medical image obtained by shielding a partial image area in the corresponding full bone tissue medical image; The initial image is embedded in the neural network, and the covered full-scale shielded bone tissue medical images are reconstructed according to the full-scale bone tissue medical images after each shielding operation, so as to generate a reconstruction confidence of each full-scale shielded bone tissue medical image for the corresponding full-scale reference bone tissue medical image; the full-scale reference bone tissue medical image corresponding to a full-scale shielded bone tissue medical image is the original partial image area of ​​the full-scale shielded bone tissue medical image in the corresponding full-scale bone tissue medical image; Generating a second bone tissue medical image reconstruction loss for embedding the initial image into a neural network according to the reconstruction confidence of each full-amount shielded bone tissue medical image with respect to the corresponding full-amount reference bone tissue medical image; The network parameters of the initial image embedding neural network are adjusted according to the second bone tissue medical image reconstruction loss to obtain the image embedding neural network.

8. The method according to claim 1, characterized in that The method further comprises: Acquire a set of matched bone tissue medical images; the set of matched bone tissue medical images includes a plurality of matched bone tissue medical images in a pathology matching process; Performing an image embedding operation on each matching bone tissue medical image in the matching bone tissue medical image set through the debugged image embedding neural network to generate a dense image embedding descriptor for each matching bone tissue medical image; An image embedding descriptor set is generated based on the dense image embedding descriptor of each matching bone tissue medical image.

9. A computer system comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, wherein: When the processor executes the program, the steps in the method according to any one of claims 1 to 8 are implemented.

Images