Model training method and image processing method

Abstract

The embodiment of the invention provides a model training method and an image processing method, and the model training method comprises the steps: determining a target texture feature of a target skeleton image, and adjusting a pre-training model to an initial prediction model based on the target texture feature; inputting the second skeleton image and the equipment information of the second skeleton image into an initial prediction model for processing to obtain an image embedding vector and an equipment embedding vector; calculating prediction weights of a plurality of prediction modules in the initial prediction model according to the image embedding vector and the equipment embedding vector; and generating prediction diagnosis information of the initial prediction model according to the prediction weights of the plurality of prediction modules, and optimizing the initial prediction model according to the prediction diagnosis information and the image diagnosis information to obtain a target prediction model. The integrity of the bone texture information when the model extracts the texture features is ensured, the generalization ability of the model for diagnosis of images of different devices is improved, and the accuracy of diagnosis of bone images is improved.

Description

Technical Field

[0001] The embodiments of this specification relate to the technical field of osteoporosis, and in particular to a model training method, an image processing method, a computer-aided diagnosis method for osteoporosis, and a computer-aided diagnosis method. Background Art

[0002] In recent years, opportunistic osteoporosis diagnosis based on CT (Computed Tomography) imaging has garnered widespread attention due to its non-invasive nature and high availability. However, current deep learning-based CT imaging diagnostic methods suffer from low diagnostic accuracy due to issues such as insufficient utilization of vertebral data and a lack of cross-device adaptability. Therefore, improving the accuracy of deep learning-based imaging diagnostics is an urgent challenge. Summary of the Invention

[0003] In view of this, embodiments of this specification provide a model training method, an image processing method, a computer-assisted diagnosis method for osteoporosis, and a computer-assisted diagnosis method. One or more embodiments of this specification also relate to a model training device, an image processing device, a computer-assisted diagnosis device for osteoporosis, a computer-assisted diagnosis device, a computing device, a computer-readable storage medium, and a computer program product to address technical deficiencies in the prior art.

[0004] According to a first aspect of the embodiments of this specification, a model training method is provided, including:

[0005] determining target texture features of the target bone image, and adjusting the pre-trained model to an initial prediction model based on the target texture features;

[0006] Inputting the second bone image and the device information of the second bone image into the initial prediction model for processing to obtain an image embedding vector and a device embedding vector;

[0007] Calculating prediction weights of multiple prediction modules in the initial prediction model according to the image embedding vector and the device embedding vector;

[0008] The prediction diagnosis information of the initial prediction model is generated according to the prediction weights of the multiple prediction modules, and the initial prediction model is optimized according to the prediction diagnosis information and the imaging diagnosis information to obtain a target prediction model.

[0009] According to a second aspect of the embodiments of this specification, there is provided an image processing method, which is applied to a target prediction model obtained by training using the above-mentioned model training method, comprising:

[0010] Inputting the bone image to be diagnosed and the target device information of the bone image to be diagnosed into the target prediction model;

[0011] Determining a first embedding vector of the bone image to be diagnosed and a second embedding vector of the target device information by using the target prediction model;

[0012] Calculating target prediction weights of multiple target prediction modules in the target prediction model according to the first embedding vector and the second embedding vector, and predicting the first embedding vector to determine multiple diagnostic sub-results of the bone image to be diagnosed;

[0013] A target diagnosis result of the bone image to be diagnosed is generated based on the target prediction weight and the multiple diagnosis sub-results.

[0014] According to a third aspect of the embodiments of this specification, a computer-aided diagnosis method for osteoporosis is provided, comprising:

[0015] Inputting the bone image to be diagnosed and the target device information of the bone image to be diagnosed into the target prediction model, wherein the target prediction model is obtained by training using the above-mentioned model training method;

[0016] Determining a first embedding vector of the bone image to be diagnosed and a second embedding vector of the target device information by using the target prediction model;

[0017] Calculating target prediction weights of multiple target prediction modules in the target prediction model according to the first embedding vector and the second embedding vector, and predicting the first embedding vector to determine multiple diagnostic sub-results of the bone image to be diagnosed;

[0018] An osteoporosis diagnosis result of the bone image to be diagnosed is generated based on the target prediction weight and the multiple diagnosis sub-results.

[0019] According to a fourth aspect of the embodiments of this specification, a computer-aided diagnosis method is provided, which is applied to a medical system client, comprising:

[0020] Receive a bone image to be diagnosed uploaded by a user, and input the bone image to be diagnosed and the target device information of the bone image to be diagnosed into the target prediction model, wherein the target prediction model is obtained by training using the above-mentioned model training method;

[0021] Determining a first embedding vector of the bone image to be diagnosed and a second embedding vector of the target device information by using the target prediction model;

[0022] Calculating target prediction weights of multiple target prediction modules in the target prediction model according to the first embedding vector and the second embedding vector, and predicting the first embedding vector to determine multiple diagnostic sub-results of the bone image to be diagnosed;

[0023] A target diagnosis result of the bone image to be diagnosed is generated based on the target prediction weight and the multiple diagnosis sub-results, and the target diagnosis result is returned to the user.

[0024] According to a fifth aspect of the embodiments of this specification, a computer-aided diagnosis method is provided, which is applied to a medical system server, including:

[0025] The diagnostic task sent by the receiving terminal device is determined, the skeletal image to be diagnosed corresponding to the diagnostic task is determined, and the skeletal image to be diagnosed and the target device information of the skeletal image to be diagnosed are input into the target prediction model, wherein the target prediction model is obtained by training the above-mentioned model training method;

[0026] Obtain a target diagnosis result output by the target prediction model based on the bone image to be diagnosed, and send the target diagnosis result to the terminal side device.

[0027] According to a sixth aspect of the embodiments of this specification, a model training method is provided, which is applied to a cloud-side device, including:

[0028] receiving a model training task sent by a terminal-side device, and determining a sample bone image corresponding to the model training task;

[0029] Inputting the sample bone image and the device information of the sample bone image into an initial prediction model for processing to obtain an image embedding vector and a device embedding vector;

[0030] Calculating prediction weights of multiple prediction modules in the initial prediction model according to the image embedding vector and the device embedding vector;

[0031] Generating predictive diagnostic information of the initial predictive model according to the prediction weights of the multiple prediction modules, and optimizing the initial predictive model according to the predictive diagnostic information and the imaging diagnostic information to obtain a target predictive model;

[0032] The model parameters of the target prediction model are sent to the end-side device, wherein the model parameters are used to deploy the target prediction model on the medical system client.

[0033] According to a seventh aspect of the embodiments of this specification, a model training device is provided, comprising:

[0034] an adjustment module configured to determine target texture features of a target bone image and adjust the pre-trained model to an initial prediction model based on the target texture features;

[0035] an input module configured to input the second bone image and the device information of the second bone image into the initial prediction model for processing to obtain an image embedding vector and a device embedding vector;

[0036] a calculation module configured to calculate prediction weights of multiple prediction modules in the initial prediction model based on the image embedding vector and the device embedding vector;

[0037] The optimization module is configured to generate predictive diagnosis information of the initial prediction model according to the prediction weights of the multiple prediction modules, and optimize the initial prediction model according to the predictive diagnosis information and the imaging diagnosis information to obtain a target prediction model.

[0038] According to an eighth aspect of the embodiments of this specification, there is provided an image processing device, which is applied to a target prediction model obtained by training using the above-mentioned model training method, comprising:

[0039] An input module configured to input the bone image to be diagnosed and the target device information of the bone image to be diagnosed into the target prediction model;

[0040] a determination module configured to determine a first embedding vector of the bone image to be diagnosed and a second embedding vector of the target device information by using the target prediction model;

[0041] a calculation module configured to calculate target prediction weights of multiple target prediction modules in the target prediction model based on the first embedding vector and the second embedding vector, and predict the first embedding vector to determine multiple diagnostic sub-results of the bone image to be diagnosed;

[0042] A generating module is configured to generate a target diagnostic result of the bone image to be diagnosed based on the target prediction weight and the multiple diagnostic sub-results.

[0043] According to a ninth aspect of the embodiments of this specification, a model training device is provided, comprising:

[0044] An input module is configured to input the bone image to be diagnosed and the target device information of the bone image to be diagnosed into the target prediction model, wherein the target prediction model is obtained by training using the above-mentioned model training method;

[0045] a determination module configured to determine a first embedding vector of the bone image to be diagnosed and a second embedding vector of the target device information by using the target prediction model;

[0046] a calculation module configured to calculate target prediction weights of multiple target prediction modules in the target prediction model based on the first embedding vector and the second embedding vector, and predict the first embedding vector to determine multiple diagnostic sub-results of the bone image to be diagnosed;

[0047] A generating module is configured to generate an osteoporosis diagnosis result of the bone image to be diagnosed based on the target prediction weight and the multiple diagnosis sub-results.

[0048] According to a tenth aspect of the embodiments of this specification, a computer-aided diagnosis device is provided, which is applied to a medical system client, including:

[0049] A receiving module is configured to receive a bone image to be diagnosed uploaded by a user, and input the bone image to be diagnosed and the target device information of the bone image to be diagnosed into the target prediction model, wherein the target prediction model is obtained by training using the above-mentioned model training method;

[0050] a determination module configured to determine a first embedding vector of the bone image to be diagnosed and a second embedding vector of the target device information by using the target prediction model;

[0051] a calculation module configured to calculate target prediction weights of multiple target prediction modules in the target prediction model based on the first embedding vector and the second embedding vector, and predict the first embedding vector to determine multiple diagnostic sub-results of the bone image to be diagnosed;

[0052] A return module is configured to generate a target diagnosis result of the bone image to be diagnosed based on the target prediction weight and the multiple diagnosis sub-results, and return the target diagnosis result to the user.

[0053] According to an eleventh aspect of the embodiments of this specification, a computer-aided diagnosis device is provided, which is applied to a medical system server, including:

[0054] a receiving module configured to receive a diagnostic task sent by an end-side device, determine a bone image to be diagnosed corresponding to the diagnostic task, and input the bone image to be diagnosed and target device information of the bone image to be diagnosed into the target prediction model, wherein the target prediction model is obtained by training using the above-mentioned model training method;

[0055] The sending module is configured to obtain a target diagnosis result output by the target prediction model based on the bone image to be diagnosed, and send the target diagnosis result to the terminal side device.

[0056] According to a twelfth aspect of the embodiments of this specification, a model training device is provided, which is applied to a cloud-side device, including:

[0057] a receiving module configured to receive a model training task sent by a terminal-side device and determine a sample bone image corresponding to the model training task;

[0058] An input module is configured to input the sample bone image and the device information of the sample bone image into an initial prediction model for processing to obtain an image embedding vector and a device embedding vector;

[0059] a calculation module configured to calculate prediction weights of multiple prediction modules in the initial prediction model based on the image embedding vector and the device embedding vector;

[0060] an optimization module configured to generate predictive diagnosis information of the initial prediction model according to the prediction weights of the multiple prediction modules, and optimize the initial prediction model according to the predictive diagnosis information and the imaging diagnosis information to obtain a target prediction model;

[0061] The sending module is configured to send the model parameters of the target prediction model to the end-side device, wherein the model parameters are used to deploy the target prediction model on the medical system client.

[0062] According to a thirteenth aspect of the embodiments of this specification, there is provided a computing device, including:

[0063] memory and processor;

[0064] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the above-mentioned model training method and image processing method are implemented.

[0065] According to the fourteenth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the above-mentioned model training method and image processing method are implemented.

[0066] According to the fifteenth aspect of the embodiments of this specification, a computer program product is provided, including a computer program or instructions, which implement the steps of the above-mentioned model training method and image processing method when executed by a processor.

[0067] One embodiment of this specification implements determining target texture features of a target bone image and using the target texture features to adjust a pre-trained model to an initial prediction model, so that the initial prediction model can subsequently fully extract texture features from the bone image, ensuring the integrity of the bone texture information. By calculating prediction weights for multiple prediction modules based on image embedding vectors and device embedding vectors, generating predictive diagnostic information based on the prediction weights, and optimizing the initial prediction model using the predictive diagnostic information and image diagnostic information, the problem of diagnostic bias caused by differences in measurement data caused by different image acquisition devices is reduced, the model's generalization ability when diagnosing images from different devices is improved, and the accuracy of the diagnosis of bone images is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 A flow chart of a model training method provided according to one embodiment of this specification is shown;

[0069] Figure 2 A flowchart of a model training method according to an embodiment of the present invention is shown.

[0070] Figure 3 A schematic structural diagram of a model training device provided by one embodiment of this specification is shown;

[0071] Figure 4 A flowchart of an image processing method provided according to one embodiment of this specification is shown;

[0072] Figure 5 A schematic structural diagram of an image processing device provided by an embodiment of this specification is shown;

[0073] Figure 6 A flowchart of a computer-aided diagnosis method for osteoporosis provided according to one embodiment of this specification is shown;

[0074] Figure 7 A schematic structural diagram of a computer-aided diagnostic device for osteoporosis provided in one embodiment of this specification is shown;

[0075] Figure 8 A flowchart of a computer-aided diagnosis method according to one embodiment of the present specification is shown;

[0076] Figure 9 A schematic diagram of the structure of a computer-aided diagnosis device provided by one embodiment of this specification is shown;

[0077] Figure 10 A flowchart of a computer-aided diagnosis method according to one embodiment of the present specification is shown;

[0078] Figure 11 A schematic diagram of the structure of a computer-aided diagnosis device provided by one embodiment of this specification is shown;

[0079] Figure 12 A flow chart of a model training method provided according to one embodiment of this specification is shown;

[0080] Figure 13 A schematic structural diagram of a model training device provided by one embodiment of this specification is shown;

[0081] Figure 14 A structural block diagram of a computing device provided according to an embodiment of the present specification is shown. DETAILED DESCRIPTION

[0082] The following description sets forth many specific details to facilitate a thorough understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0083] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a," "the," and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0084] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0085] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0086] First, the terms involved in one or more embodiments of this specification are explained.

[0087] Dual-Energy X-ray Absorptiometry (DXA): This technique accurately measures bone density and is currently the standard method for assessing bone health and diagnosing osteoporosis. DXA offers the advantages of high precision, low radiation dose, and rapid testing. It not only accurately measures bone density but also assesses other body components, such as fat and muscle mass.

[0088] Opportunistic osteoporosis diagnosis: This diagnostic method uses existing CT imaging data to screen patients for osteoporosis without the need for additional imaging tests. This not only improves the efficiency of medical resources but also provides the possibility of early diagnosis without increasing the burden on patients.

[0089] Bone Mineral Density (BMD): This refers to the amount of minerals (primarily calcium and phosphorus) in bone per unit volume or area. BMD is an important indicator for assessing bone strength and health, and is widely used in fracture risk assessment and osteoporosis diagnosis.

[0090] Multi-task learning: This is a machine learning method that aims to optimize multiple related tasks simultaneously. For example, in medical image analysis, tasks such as bone density regression, vertebral location, and osteoporosis classification may be performed simultaneously. This approach can improve overall performance because different tasks can share information and help each other improve accuracy.

[0091] Mixture of Experts (MoE): This is a neural network architecture that dynamically selects an expert subnetwork best suited to the input data to make predictions. MoE uses a gating network to determine how to allocate tasks to different expert models, achieving greater efficiency and cross-device adaptability.

[0092] Texture-preserving Swapping Assignments between Multiple Views (TP-SwAV): This is a self-supervised learning method for image processing and analysis. It uses a sliding window mechanism to extract texture features, such as bones, while ensuring that the original image structure is intact during feature extraction. This method is particularly suitable for tasks that require preserving detailed information in the image.

[0093] Current deep learning-based methods for diagnosing opportunistic osteoporosis using CT images fail to fully utilize vertebral data, focusing only on osteoporosis classification while ignoring supplementary information such as BMD regression and vertebral positioning, resulting in limited diagnostic effectiveness. These methods are also affected by measurement differences between different DXA devices and perform poorly in multi-center or multi-device scenarios. When extracting texture features, random cropping operations destroy bone texture features, affecting diagnostic accuracy. All of these issues will lead to reduced diagnostic accuracy.

[0094] Based on this, this specification provides a model training method, an image processing method, a computer-assisted diagnosis method for osteoporosis, and a computer-assisted diagnosis method. This specification also involves a model training device, an image processing device, a computer-assisted diagnosis device for osteoporosis, a computer-assisted diagnosis device, a computing device, a computer-readable storage medium, and a computer program product, which are described in detail one by one in the following embodiments.

[0095] See also Figure 1 , Figure 1 A flowchart of a model training method provided according to an embodiment of this specification is shown, which specifically includes the following steps.

[0096] Step 102: Determine target texture features of the target bone image, and adjust the pre-trained model to an initial prediction model based on the target texture features.

[0097] The pre-trained model can be understood as a model trained from a large amount of unlabeled data. Its main goal is to learn general feature representations from the data, without relying on specific task objectives (such as bone density prediction). That is, the pre-trained model is not targeted at a specific task, but focuses on learning a general representation of images. The initial prediction model can be understood as a model that further uses input data to learn on the basis of the pre-trained model to better understand and represent CT images. The goal of the initial prediction model is to enhance the ability to understand CT images, but it is still not targeted at a specific task.

[0098] In practical applications, the pre-trained model can adopt unsupervised learning, that is, the training process does not rely on any labeled data for training. Through model training, the pre-trained model can have the ability to universally represent the input image; the initial prediction model can adopt self-supervised learning to learn using a small amount of labeled data, and fine-tune the pre-trained model based on the pre-trained model. The model parameters of the pre-trained model are adjusted to adapt to the data distribution of the target task, providing an encoder with powerful vector expression capabilities, which can map the input image to a low-dimensional latent space. The learned features can capture important information in the image (such as texture, shape, etc.), providing a basis for subsequent stages.

[0099] In the specific implementation, it is necessary to first determine the target texture features of the target bone image. The target texture features can be understood as the texture features of the entire image range corresponding to the target bone image. The target texture features may include autocorrelation features, joint average features, contrast features, and difference average features. In practical applications, the target texture features of the target bone image can be determined by constructing a texture pool, which contains the original image and its texture features. After determining the target texture features of the target bone image, the pre-trained model can be adjusted to the initial prediction model based on the target texture features, so that the initial prediction model can extract texture features from the image more completely.

[0100] Furthermore, in order to pay more attention to the part of the image associated with the DXA measurement data, it is necessary to extract the part of the image associated with the DXA measurement data from the image. Before specifically determining the target texture features of the target bone image, the method also includes: determining an initial bone image and bone measurement data associated with the initial bone image; and determining the target bone image in the initial bone image based on the bone measurement data.

[0101] The initial skeletal image can be understood as a CT image obtained by scanning the user or patient using CT. The initial skeletal image can be a skeletal image obtained by scanning the user's entire spine. In the vertebral data of the initial skeletal image, the corresponding DXA gold standard results may only account for a small part, and more are unlabeled areas. Therefore, in order to enable the model to analyze the vertebral areas associated with DXA measurement data in the image during subsequent diagnosis, it is necessary to extract the target skeletal image from the initial skeletal image and subsequently use the target skeletal image for model training, so that the model can pay more attention to the useful information in the image.

[0102] In practical applications, the bone measurement data associated with the initial bone image can be understood as the DXA measurement data corresponding to the CT image. DXA measurement data includes BMD assessment and is used to diagnose osteoporosis. The bone measurement data associated with the initial bone image can be considered to be the initial bone image obtained via CT scan and the bone measurement data obtained via DXA measurement for the same user or patient. Based on the bone measurement data, regional images related to the bone measurement data can be filtered from the initial bone image.

[0103] In a specific implementation, the positioning model can be used to identify the vertebral area corresponding to the bone measurement data in the initial bone image, thereby associating the bone image at the vertebral level with the bone data.

[0104] In a specific embodiment of the present specification, a trained three-dimensional segmentation model such as 3D U-Net (3D U-Net is a classic deep learning model widely used in medical image segmentation tasks. It can efficiently extract features in three-dimensional space through an encoder-decoder structure and skip connections) is used to identify the various vertebral regions in the initial bone image, i.e., the CT image. Then, a Med-Query method (Med-Query is a query method that combines medical knowledge and can accurately mark the location information of the vertebra) is used to identify the position of the vertebra, and the vertebra is matched with the DXA measurement data, thereby selecting the target bone image in the initial bone image.

[0105] Based on this, the team combined CT images with DXA measurements, leveraging deep learning and image analysis techniques to accurately predict and analyze vertebral bone density. This approach not only provides richer anatomical information but also allows for calibration with DXA measurements, thereby improving prediction accuracy.

[0106] Furthermore, in determining the target texture features of the target bone image, in order to avoid the destruction of the texture features by random cropping operations, a sliding window technique can be used to extract the texture features. Specifically, determining the target texture features of the target bone image includes: performing window sliding processing on the target bone image to obtain multiple local bone images; extracting local texture features of the multiple local bone images, and calculating the target texture features of the target bone image based on the local texture features of the multiple local bone images.

[0107] Among them, performing window sliding processing on the target bone image can be understood as using the sliding window technology to extract the target bone image, obtaining window images corresponding to multiple windows, i.e., local bone images. It can also be understood as using the sliding window technology to segment the target bone image into multiple local bone images. Then, texture features are extracted from the multiple local bone images to obtain local texture features of each local bone image. Subsequently, the target texture features of the target bone image can be calculated based on the local texture features of each local bone image.

[0108] In practical applications, the extraction of feature information within the entire image through sliding window technology can protect the texture features of the image, thereby avoiding the loss of texture features due to random cropping. Specifically, the corresponding local texture features such as mean, contrast, and autocorrelation are calculated for the local bone image of each window, and the local texture features of each local bone image are summarized into a matrix. This matrix contains the texture information of each area in the target bone image, that is, the target bone image is observed from the perspectives of autocorrelation, mean, contrast, etc., and the image texture features of the corresponding perspective are obtained. These features are full-image size, thereby ensuring the integrity of the extracted target texture features.

[0109] In specific implementations, a sliding window technique is used to localize the image, combined with the Pyradiomics toolkit to extract spatially localized texture descriptors. Pyradiomics is an open-source toolkit for extracting quantitative features from medical images. It can be used to analyze radiomic features in oncology. In this example, it is used to extract bone texture features. To ensure consistency between the features obtained from texture analysis and the original image, an adaptive interpolation technique is applied. This step ensures that the texture features accurately reflect the information of the original image even at different resolutions or scales.

[0110] In a specific embodiment of this specification, it is assumed that the sliding window size is 5×5×5 (in three-dimensional space), which means that each window covers a local cubic area of ​​the image. The windows are slid in sequence from top to bottom, from left to right, and from front to back to traverse the entire image. Each time the window is moved, it is usually done with a step size of 1 to ensure that no pixel is missed. Local bone images corresponding to multiple windows are obtained, and texture features including mean, contrast, and autocorrelation are calculated for the local bone images corresponding to each window. The mean can be the average value of all pixel values ​​in the calculation window, representing the general level of brightness or intensity in the area; the contrast can be a measure of the degree of change of pixel values ​​in the window, reflecting the texture roughness inside the area; and the autocorrelation can be an assessment of the similarity between adjacent pixels, revealing the directionality and periodicity of the texture pattern. After the above steps, the local texture features of each local bone image can be obtained. The local texture features will be placed in a matrix to generate a local feature matrix, and then each local feature matrix is ​​fused and calculated to obtain a target feature matrix of the target bone image. The target feature matrix has the same spatial resolution as the target bone image, that is, a full-size representation in the (x, y, z) coordinate system. If the size of the feature matrix does not match the original image due to boundary conditions or other reasons during the feature extraction process, an interpolation algorithm needs to be used to adjust its size to the original image size.

[0111] Based on this, sliding windows, as an effective feature extraction strategy, are particularly useful when processing complex datasets such as medical imaging. Sliding windows cover the entire image, ensuring that every local area is analyzed, avoiding the loss of critical details that might otherwise be lost due to cropping. They not only effectively extract local texture information but also maintain its spatial distribution across the entire image, improving the accuracy of subsequent tasks such as disease diagnosis and lesion detection.

[0112] Furthermore, if the extracted imaging genomics features do not match the size of the original image, the extracted features need to be adjusted according to the size of the original image to ensure that the spatial dimensions of the two are consistent. Specifically, the target texture features of the target bone image are calculated based on the local texture features of the multiple local bone images, including: generating an initial feature matrix based on the local texture features of the multiple local bone images; adjusting the initial feature matrix based on the image attribute information of the target bone image to obtain a target feature matrix, and determining the target texture features of the target bone image based on the target feature matrix.

[0113] The initial feature matrix can be understood as the feature matrix of the target bone image generated based on all local texture features, that is, all local texture features are integrated into a feature pool to form a unified feature representation. However, since the size of the feature matrix may not match the image size of the target bone image due to window boundary conditions during the feature extraction process, an adaptive interpolation algorithm can be used to adjust the target feature matrix based on the image attribute information, that is, the image size information of the target bone image, to ensure that the dimensions of the target feature matrix and the target bone image are consistent. After obtaining the target feature matrix, the target feature matrix can be used as the target texture feature of the target bone image.

[0114] In practical applications, in order to preserve the spatial distribution information of the original image so that the features at each position correspond to their positions in the original image, after generating the initial feature matrix based on the local texture features, it is necessary to adjust the spatial distribution of the initial feature matrix so that the feature matrix is ​​placed in a spatial coordinate system with the same size as the original image, ensuring that the dimensions of the target bone image and the target feature matrix are the same.

[0115] In a specific embodiment of the present specification, an initial feature matrix is ​​generated based on the local texture features of the local bone image corresponding to each window, and the initial feature matrix is ​​adjusted according to the image size of the target bone image to obtain a target feature matrix, which is used as the target texture feature.

[0116] Based on this, by adjusting the size of the initial feature matrix, the initial feature matrix is ​​made to have the same spatial dimension as the target bone image, ensuring that the extracted texture features have a corresponding positional relationship with the target bone image, and improving the model training efficiency when the texture features are subsequently used for model training.

[0117] Furthermore, after extracting the target texture features of the target bone image, the target texture features can be used to adjust the model capabilities of the pre-trained model. Specifically, the pre-trained model is adjusted to the initial prediction model based on the target texture features, including: using the pre-trained model to encode the target texture features to obtain a texture embedding vector, and calculating the prototype distribution information corresponding to the texture embedding vector; adjusting the model parameters of the pre-trained model based on the prototype distribution information to obtain an initial prediction model.

[0118] The texture embedding vector can be understood as the latent embedding vector obtained after encoding the target texture features. The texture embedding vector is a low-dimensional representation vector. In practical applications, the target texture features are mapped to latent embedding vectors by the encoder. These latent embedding vectors will be assigned to a set of K learnable prototypes. For example, for the latent embedding vectors Pt and Ps generated by two different views of the same image, the original distributions Qt and Qs are calculated through transfer matching.

[0119] In practice, the target texture features extracted from the target skeletal image are input to the pre-trained model's universal encoder, which can be a deep neural network. Its role is to convert the input texture features into a low-dimensional representation, called a latent embedding vector. The latent embedding vector captures the core information of the original features, but is typically lower dimensional and more abstract, making it easier to process. A fixed set of learnable prototypes (K) exists. These prototypes are essentially a set of vectors, each representing a typical pattern or category in the data space. They are part of the model and are continuously adjusted during training to better match the data distribution. After the texture features are converted into latent embeddings by the encoder, the next step is to determine how these embeddings relate to the K prototypes. This is typically achieved by calculating the similarity between each embedding and all prototypes, and then assigning embeddings to prototypes that are closest based on this similarity. When two sets of latent embeddings are generated from different views of the same image (for example, through different transformations or perspectives), the assignments between the two sets of embeddings can be compared using the method of optimal transfer matching. Optimal transfer theory provides a way to measure the difference between two probability distributions and identify the "optimal" assignment that minimizes this difference. The prototype distribution information is the distribution information of each embedding for the prototype. The model parameters can be adjusted through the prototype distribution information, so that the model can have a stable and consistent understanding of the image, thereby improving the generalization ability and robustness of the model.

[0120] In one embodiment of this specification, target texture features are input into a pre-trained model. The target texture features are encoded using an encoder within the pre-trained model to obtain a texture embedding vector. The texture embedding vector is then assigned to a prototype, and the corresponding prototype distribution information is calculated. This prototype distribution information is then used to adjust the model parameters of the pre-trained model to obtain an initial prediction model.

[0121] Furthermore, the model parameters of the pre-trained model are adjusted based on the prototype distribution information to obtain an initial prediction model, including: calculating the exchange prediction loss and the clustering regularization loss based on the prototype distribution information; and adjusting the model parameters of the pre-trained model based on the exchange prediction loss and the clustering regularization loss to obtain the initial prediction model.

[0122] The core purpose of the exchange prediction loss is to ensure that the potential embeddings generated from different views can be assigned to consistent prototypes. In other words, even if the input is different views of the same image, their embeddings should be mapped to similar prototypes. Figure 1 The core purpose of the clustering regularization loss is to encourage each prototype to be assigned a certain number of embeddings, thereby preventing some prototypes from being overused or completely ignored.

[0123] In practice, cross-view is enforced by exchanging prediction losses. Figure 1 Consistency ensures that embeddings generated from different views of the same image are assigned to consistent prototypes. This helps the model learn robust feature representations. By introducing a clustering regularization term, we ensure prototype diversity and prevent certain prototypes from being overused or completely ignored, thereby improving the model's coverage and expressiveness of the data distribution.

[0124] In specific implementation, after obtaining the prototype distribution information, the exchange prediction loss and clustering regularization loss can be calculated based on the prototype distribution information, and then the model parameters of the pre-trained model are adjusted based on the exchange prediction loss and clustering regularization loss. The pre-trained model is continued to be trained until the training stop conditions are met, such as reaching the training round or reaching the training goal, and the initial prediction model can be obtained.

[0125] Based on this, self-supervised learning techniques are used to learn useful representations from unlabeled data. By converting texture features into latent embeddings and assigning these embeddings to a set of learnable prototypes, the model is able to capture the essential structure of the data. Comparing the embedding assignments generated by different views of the same image through optimal transfer matching ensures that the model has a stable and consistent understanding of the same object from different perspectives, thereby improving the model's generalization and robustness.

[0126] Step 104: Input the second bone image and the device information of the second bone image into the initial prediction model for processing to obtain an image embedding vector and a device embedding vector.

[0127] The second skeletal image can be understood as training image data used to train the initial prediction model. The second skeletal image can be the same as or different from the target skeletal image. The device information of the second skeletal image can be understood as relevant information of the DXA device. Due to differences in DXA devices, different DXA devices may produce different bone mineral density (BMD) values ​​for the same vertebra in the same patient. In order to reduce diagnostic differences caused by differences in DXA devices, device information can be introduced. The device information includes the serial identification information of the DXA device.

[0128] In practical applications, device information can be associated with a second skeletal image. This second skeletal image and its associated device information are then fed into an initial prediction model for processing. The initial prediction model then generates an image embedding vector and a device embedding vector. The image embedding vector is the latent embedding vector obtained by encoding the second skeletal image, while the device embedding vector is the latent embedding vector obtained by encoding the device information. DXA device embeddings are learnable and are updated as the model trains, capturing the data's underlying structure and semantics.

[0129] Furthermore, the second bone image and the device information of the second bone image are input into the initial prediction model for processing to obtain an image embedding vector and a device embedding vector, including: inputting the second bone image and the device information of the second bone image into the initial prediction model; encoding the second bone image and the device information respectively through the initial prediction model to obtain the image embedding vector corresponding to the second bone image, and the device embedding vector corresponding to the device information.

[0130] After the second bone image and the device information are input into the initial prediction model, they can be encoded respectively to obtain an image embedding vector and a device embedding vector.

[0131] In actual applications, when a CT image is input into the model, the corresponding device embedding vector can be obtained from the encoding layer based on the DXA device information associated with the image (i.e., device ID). The device embedding vector is a learnable parameter, which means that as training progresses, it will gradually adjust to better capture the data structure and semantic information associated with it. For example, during training, if the data from a certain device has specific patterns or deviations, the device embedding will learn these characteristics, thereby helping the model to more accurately process data from the device. It should be noted that the dimensions of the image embedding vector and the device embedding vector are consistent, allowing both to be processed under the same framework.

[0132] In a specific embodiment of the present specification, a CT image and a DXA device identifier associated with the CT image are input into an initial prediction model, and the CT image and the DXA device identifier are respectively encoded through the encoding layer in the initial prediction model to obtain an image embedding vector corresponding to the CT image and a device embedding vector corresponding to the DXA device identifier.

[0133] Based on this, by adding device information to the model input data and using the device embedding vector corresponding to the device information as part of the model training, the model can learn and adapt to the data patterns unique to different devices. This not only improves the model's ability to process data from specific devices, but also enhances its generalization performance across devices.

[0134] Step 106: Calculate prediction weights of multiple prediction modules in the initial prediction model based on the image embedding vector and the device embedding vector.

[0135] Furthermore, in order to solve the inherent heterogeneity problem between different DXA devices, the decoding stage of the initial prediction model integrates an expert hybrid architecture. To this end, it is necessary to determine the corresponding prediction weights for the expert modules. Specifically, based on the image embedding vector and the device embedding vector, the prediction weights of multiple prediction modules in the initial prediction model are calculated, including: fusing the image embedding vector and the device embedding vector to obtain a fused embedding vector; calculating the attention weight corresponding to the initial prediction model based on the fused embedding vector, and determining the prediction weights of multiple prediction modules in the initial prediction model based on the attention weight.

[0136] The fused embedding vector can be understood as the fusion of the image embedding vector and the device embedding vector. The fused embedding vector is the sum of the image embedding vector and the device embedding vector. Based on this fused embedding vector, attention weights corresponding to the initial prediction model are generated. The attention weights reflect the importance and relevance of each prediction module for the current input data. The prediction weights can be understood as the attention prediction weights corresponding to each prediction module.

[0137] In practical applications, the initial prediction model includes a Mixture of Experts (MoE) architecture, which is used to address the heterogeneity problem between different DXA devices and improve the generalization and robustness of the model through dynamic adaptation and cross-device regularization. The expert model under this architecture is the prediction module, and each prediction module focuses on capturing the data features from a specific DXA device. For example, if there are three different DXA devices, there may be three corresponding expert modules, each of which is trained to recognize and process image features unique to the device. This architecture also includes an adaptive fusion mechanism, which is implemented through a gated network. The gated network can generate adaptive attention weights based on the sum of the input image embedding and the embedding of the corresponding DXA device. The calculated weight determines the degree of influence of each expert output on the prediction result.

[0138] In practice, a parameterized gating function is used to generate a series of attention weights based on the sum of the image embedding and the DXA device embedding. These weights reflect the importance or relevance of each expert model to the current input data. The attention weights are used to determine the prediction weight corresponding to each expert model. Subsequently, the prediction weights are used to perform a weighted average summation of the outputs of each expert model to obtain the model output.

[0139] In a specific embodiment of this specification, the latent embedding vector of the CT image (generated by the encoder) and the embedding information of the corresponding DXA device are used as input. The DXA device embedding can be a simple one-hot encoding (indicating the device type). A gated network is used to calculate attention weights, and the prediction weights of each prediction module, i.e., the expert model, are determined based on the attention weights.

[0140] Based on this, the inherent heterogeneity between different DXA devices is cleverly addressed by integrating the MoE architecture into the decoding stage. By utilizing device-specific expert modules and adaptive fusion mechanisms, dynamic adaptation of data from different devices is achieved.

[0141] Step 108: Generate predictive diagnosis information of the initial prediction model according to the prediction weights of the multiple prediction modules, and optimize the initial prediction model according to the predictive diagnosis information and the imaging diagnosis information to obtain a target prediction model.

[0142] The predicted diagnostic information can be understood as the predicted result output by the initial prediction model after diagnostic processing of the second skeletal image, and the imaging diagnostic information can be understood as the actual diagnostic result corresponding to the second skeletal image. That is, the second skeletal image serves as training sample data for model training, and the imaging diagnostic information serves as the training label data for the training sample data. Therefore, the initial prediction model can be optimized based on the predicted diagnostic information (i.e., the predicted value) and the imaging diagnostic information (i.e., the actual value) to obtain the target prediction model. In practical applications, the predicted diagnostic information can include diagnostic information such as the diagnosed bone density value, diagnostic classification, and anatomical location.

[0143] Furthermore, the predictive diagnosis information of the initial prediction model is generated according to the prediction weights of the multiple prediction modules, including: processing the image embedding vector through the multiple prediction modules to obtain the predictive diagnosis sub-information of the multiple prediction modules; and calculating the predictive diagnosis information of the initial prediction model according to the prediction weights of the multiple prediction modules and the predictive diagnosis sub-information of the multiple prediction modules.

[0144] The processing of image embedding vectors by multiple prediction modules can be understood as using each prediction module to predict the same image embedding vector, thereby obtaining the predicted diagnostic sub-information output by each prediction module. Because each prediction module has a different corresponding prediction weight, when subsequently calculating the output of the initial prediction model, it is necessary to perform a weighted average summation of all the predicted diagnostic sub-information based on the prediction weight of each prediction module to obtain the predicted diagnostic information of the initial prediction model.

[0145] In practice, each prediction module, or expert model, processes the image embedding vector and generates an output. Each expert model corresponds to a specific piece of predicted diagnosis information. The outputs of all expert models are weighted and summed using the attention weights calculated previously to form the prediction output of the initial prediction model. This initial prediction model can then be optimized using both the predicted diagnosis information and the imaging diagnosis information to achieve the desired model training results.

[0146] Based on this, the predictive diagnostic information is calculated for all predictive diagnostic sub-information according to the prediction weights of multiple prediction modules, and the model is subsequently optimized using the predictive diagnostic information, so that the model can dynamically select relevant expert models to process the data of specific devices based on the current input (including image features and device information), allowing the model to adapt to the data of different DXA devices and solve the heterogeneity problem between devices.

[0147] Furthermore, the initial prediction model is optimized according to the predictive diagnostic information and the imaging diagnostic information to obtain a target prediction model, including: determining the imaging diagnostic information corresponding to the second bone image, and calculating the prediction task loss and the device regularization loss according to the predictive diagnostic information and the imaging diagnostic information; adjusting the model parameters of the initial prediction model according to the prediction task loss and the device regularization loss to obtain a target prediction model.

[0148] Among them, the imaging diagnostic information is the actual diagnostic information corresponding to the second bone image. The prediction task loss and the device regularization loss can be calculated based on the predicted diagnostic information and the imaging diagnostic information. The prediction task loss can be understood as the loss corresponding to the model's diagnostic prediction task. The prediction task loss can be the mean square error (MSE) or the cross-entropy loss, which is used to measure the gap between the model's predicted value and the true value. The device regularization loss is used to measure the adaptability of the model on different DXA devices. The device regularization loss can be calculated through certain constraints (such as output consistency, feature consistency, expert module diversity, etc.), which is used to measure the performance of the model when processing multi-device data.

[0149] In practical applications, the model parameters of the initial prediction model can be adjusted based on the prediction task loss and device regularization loss. Backpropagation can be used to update the model parameters. For example, the encoder parameters in the model can be adjusted based on the prediction task loss, and the expert module parameters and gating network parameters can be adjusted using the device regularization loss. By adjusting the model parameters of the initial prediction model, the model can better adapt to data from different DXA devices and improve prediction accuracy.

[0150] In a specific embodiment of the present specification, the true diagnostic result of the CT image is obtained, the prediction task loss and the device regularization loss are calculated based on the predicted diagnostic result output by the model and the true diagnostic result, the encoder parameters in the initial prediction model are adjusted according to the prediction task loss, the various expert model parameters and the gating network parameters in the initial prediction model are adjusted according to the device regularization loss, and the initial prediction model is continuously trained until a target prediction model that meets the training stop conditions is obtained. The target prediction model can be used for the diagnosis of opportunistic osteoporosis to improve diagnostic efficiency and accuracy.

[0151] This specification provides a model training method, including: determining the target texture features of the target bone image, and adjusting the pre-trained model to the initial prediction model based on the target texture features; inputting the second bone image and the device information of the second bone image into the initial prediction model for processing to obtain an image embedding vector and a device embedding vector; calculating the prediction weights of multiple prediction modules in the initial prediction model based on the image embedding vector and the device embedding vector; generating the prediction diagnosis information of the initial prediction model based on the prediction weights of the multiple prediction modules, and optimizing the initial prediction model based on the prediction diagnosis information and the image diagnosis information to obtain the target prediction model. By determining the target texture features of the target bone image and using the target texture features to adjust the pre-trained model to the initial prediction model, the initial prediction model can subsequently fully extract the texture features from the bone image, thereby ensuring the integrity of the bone texture information. By calculating the prediction weights of multiple prediction modules based on image embedding vectors and device embedding vectors, generating predictive diagnostic information based on the prediction weights, and optimizing the initial prediction model using the predictive diagnostic information and imaging diagnostic information, the problem of diagnostic bias caused by differences in measurement data caused by different imaging acquisition devices is reduced, the model's generalization ability for diagnosing images from different devices is improved, and the accuracy of diagnosing bone images is improved.

[0152] The following combined Figure 2 , taking the application of the model training method provided in this specification in the osteoporosis diagnosis model as an example, the model training method is further explained. Figure 2 A flowchart of the processing process of a model training method provided by an embodiment of this specification is shown, which specifically includes the following steps.

[0153] Step 202: Determine an initial bone image and bone measurement data associated with the initial bone image, and determine a target bone image in the initial bone image based on the bone measurement data.

[0154] In one achievable manner, the initial bone image is a CT original image, the bone measurement data associated with the initial bone image is DXA measurement data associated with the CT original image, and a matching CT local image is determined in the CT original image according to the DXA measurement data as the target bone image.

[0155] Step 204: Perform window sliding processing on the target bone image to obtain multiple local bone images, extract local texture features of the multiple local bone images, and generate an initial feature matrix based on the local texture features of the multiple local bone images.

[0156] In one feasible method, the target bone image is segmented into multiple local bone images of different perspectives using window sliding technology, and the local texture features of each local bone image are extracted using the Pyradiomics toolkit, and the initial feature matrix of the target bone image is generated based on each local texture feature.

[0157] Step 206: Adjust the initial feature matrix according to the image attribute information of the target bone image to obtain a target feature matrix, and determine the target texture features of the target bone image according to the target feature matrix.

[0158] In one feasible method, the initial feature matrix is ​​adjusted based on an adaptive interpolation algorithm according to the image size of the target bone image, and the matrix size of the initial feature matrix is ​​adjusted so that the dimension of the target bone image is consistent with that of the initial feature matrix, thereby obtaining the target feature matrix, and using the target feature matrix as the target texture feature of the target bone image.

[0159] Step 208: Encode the target texture feature to obtain a texture embedding vector, and calculate prototype distribution information corresponding to the texture embedding vector.

[0160] In one feasible method, the target texture feature is input into a pre-trained model, the target texture feature is encoded by an encoder in the pre-trained model to obtain a texture embedding vector, the texture embedding vector is assigned to a prototype, and the prototype distribution information corresponding to the texture embedding vector is calculated.

[0161] Step 210: Calculate the exchange prediction loss and clustering regularization loss based on the prototype distribution information, adjust the model parameters of the pre-trained model based on the exchange prediction loss and clustering regularization loss, and obtain an initial prediction model.

[0162] In one feasible approach, exchange prediction loss and clustering regularization loss are calculated respectively based on prototype distribution information, and the exchange prediction loss and clustering regularization loss are used to adjust the model parameters of the pre-trained model, so that the pre-trained model can better understand the image representation of the CT image and obtain an initial prediction model.

[0163] Step 212: Input the second bone image and the device information of the second bone image into the initial prediction model for processing to obtain an image embedding vector and a device embedding vector.

[0164] In one feasible method, the second bone image and the device information of the second bone image are input into the initial prediction model, and the second bone image and the device information are respectively encoded by the initial prediction model to obtain the image embedding vector corresponding to the second bone image and the device embedding vector corresponding to the device information.

[0165] Step 214: Fuse the image embedding vector and the device embedding vector to obtain a fused embedding vector, calculate the attention weight corresponding to the initial prediction model based on the fused embedding vector, and determine the prediction weights of multiple prediction modules in the initial prediction model based on the attention weight.

[0166] In one implementation, the sum of the image embedding vector and the device embedding vector is calculated to obtain a fused embedding vector. The attention weight W corresponding to the initial prediction model is calculated based on the fused embedding vector. The prediction weights {w1, w2, ...wn} of each prediction module are determined based on the attention weight W.

[0167] Step 216: Process the image embedding vector through multiple prediction modules to obtain prediction diagnosis sub-information of the multiple prediction modules, and calculate the prediction diagnosis information of the initial prediction model based on the prediction weights of the multiple prediction modules and the prediction diagnosis sub-information of the multiple prediction modules.

[0168] In one feasible manner, the image embedding vector of the input data is processed by each prediction module to obtain the predictive diagnosis sub-information output by each prediction module, and all the predictive diagnosis sub-information is weighted and summed according to the prediction weight of each prediction module to obtain the predictive diagnosis information output by the initial prediction model.

[0169] Step 218: Optimize the initial prediction model based on the predicted diagnosis information and the imaging diagnosis information to obtain a target prediction model.

[0170] In one feasible method, the imaging diagnostic information corresponding to the second bone image is determined, and the prediction task loss and the device regularization loss are calculated based on the predicted diagnostic information and the imaging diagnostic information. The model parameters of the initial prediction model are adjusted based on the prediction task loss and the device regularization loss to obtain the target prediction model. The target prediction model can be used for auxiliary diagnosis of osteoporosis.

[0171] This specification provides a model training method that determines target texture features of a target bone image and uses the target texture features to adjust a pre-trained model to an initial prediction model, enabling the initial prediction model to subsequently fully extract texture features from the bone image, thereby ensuring the integrity of the bone texture information. By calculating prediction weights for multiple prediction modules based on image embedding vectors and device embedding vectors, generating predictive diagnostic information based on the prediction weights, and optimizing the initial prediction model using the predictive diagnostic information and image diagnostic information, the problem of diagnostic bias caused by differences in measurement data from different image acquisition devices is reduced, the model's generalization ability for diagnosing images from different devices is improved, and the accuracy of diagnosing bone images is improved.

[0172] Corresponding to the above method embodiment, this specification also provides a model training device embodiment, Figure 3 FIG1 shows a schematic diagram of the structure of a model training device provided by an embodiment of this specification. Figure 3 As shown, the device includes:

[0173] An adjustment module 302 is configured to determine target texture features of a target bone image and adjust the pre-trained model to an initial prediction model based on the target texture features;

[0174] An input module 304 is configured to input the second bone image and the device information of the second bone image into the initial prediction model for processing to obtain an image embedding vector and a device embedding vector;

[0175] a calculation module 306 configured to calculate prediction weights of multiple prediction modules in the initial prediction model based on the image embedding vector and the device embedding vector;

[0176] The optimization module 308 is configured to generate the prediction diagnosis information of the initial prediction model according to the prediction weights of the multiple prediction modules, and optimize the initial prediction model according to the prediction diagnosis information and the imaging diagnosis information to obtain a target prediction model.

[0177] Optionally, the device further includes a determination module configured to determine an initial bone image and bone measurement data associated with the initial bone image; and determine a target bone image in the initial bone image based on the bone measurement data.

[0178] Optionally, the adjustment module 302 is further configured to perform window sliding processing on the target bone image to obtain multiple local bone images; extract local texture features of the multiple local bone images, and calculate the target texture features of the target bone image based on the local texture features of the multiple local bone images.

[0179] Optionally, the adjustment module 302 is further configured to generate an initial feature matrix based on the local texture features of the multiple local bone images; adjust the initial feature matrix based on the image attribute information of the target bone image to obtain a target feature matrix; and determine the target texture features of the target bone image based on the target feature matrix.

[0180] Optionally, the adjustment module 302 is further configured to use a pre-trained model to encode the target texture features, obtain a texture embedding vector, and calculate the prototype distribution information corresponding to the texture embedding vector; adjust the model parameters of the pre-trained model based on the prototype distribution information to obtain an initial prediction model.

[0181] Optionally, the adjustment module 302 is further configured to calculate the exchange prediction loss and the clustering regularization loss based on the prototype distribution information; adjust the model parameters of the pre-trained model based on the exchange prediction loss and the clustering regularization loss to obtain an initial prediction model.

[0182] Optionally, the input module 304 is further configured to input the second bone image and the device information of the second bone image into the initial prediction model; the second bone image and the device information are respectively encoded through the initial prediction model to obtain the image embedding vector corresponding to the second bone image and the device embedding vector corresponding to the device information.

[0183] Optionally, the calculation module 306 is further configured to fuse the image embedding vector and the device embedding vector to obtain a fused embedding vector; calculate the attention weight corresponding to the initial prediction model based on the fused embedding vector, and determine the prediction weights of multiple prediction modules in the initial prediction model based on the attention weight.

[0184] Optionally, the optimization module 308 is further configured to process the image embedding vector through the multiple prediction modules to obtain the predictive diagnosis sub-information of the multiple prediction modules; and calculate the predictive diagnosis information of the initial prediction model based on the prediction weights of the multiple prediction modules and the predictive diagnosis sub-information of the multiple prediction modules.

[0185] Optionally, the optimization module 308 is further configured to determine the imaging diagnostic information corresponding to the second skeletal image, and calculate the prediction task loss and the device regularization loss based on the predicted diagnostic information and the imaging diagnostic information; adjust the model parameters of the initial prediction model according to the prediction task loss and the device regularization loss to obtain the target prediction model.

[0186] This specification provides a model training device, including: an adjustment module, configured to determine the target texture features of the target bone image, and adjust the pre-trained model to the initial prediction model based on the target texture features; an input module, configured to input the second bone image and the device information of the second bone image into the initial prediction model for processing, and obtain an image embedding vector and a device embedding vector; a calculation module, configured to calculate the prediction weights of multiple prediction modules in the initial prediction model based on the image embedding vector and the device embedding vector; an optimization module, configured to generate the prediction diagnosis information of the initial prediction model based on the prediction weights of the multiple prediction modules, and optimize the initial prediction model based on the prediction diagnosis information and the image diagnosis information to obtain the target prediction model. The target texture features of the target bone image are determined, and the pre-trained model is adjusted to the initial prediction model using the target texture features, so that the initial prediction model can subsequently fully extract the texture features from the bone image, ensuring the integrity of the bone texture information. By calculating the prediction weights of multiple prediction modules based on image embedding vectors and device embedding vectors, generating predictive diagnostic information based on the prediction weights, and optimizing the initial prediction model using the predictive diagnostic information and imaging diagnostic information, the problem of diagnostic bias caused by differences in measurement data caused by different imaging acquisition devices is reduced, the model's generalization ability for diagnosing images from different devices is improved, and the accuracy of diagnosing bone images is improved.

[0187] The above is a schematic scheme of a model training device of this embodiment. It should be noted that the technical scheme of the model training device and the technical scheme of the above-mentioned model training method are of the same concept. For details not described in detail in the technical scheme of the model training device, please refer to the description of the technical scheme of the above-mentioned model training method.

[0188] See also Figure 4 , Figure 4 A flowchart of an image processing method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0189] Step 402: Input the bone image to be diagnosed and the target device information of the bone image to be diagnosed into the target prediction model.

[0190] Step 404: Determine a first embedding vector of the bone image to be diagnosed and a second embedding vector of the target device information through the target prediction model.

[0191] Step 406: Calculate target prediction weights of multiple target prediction modules in the target prediction model based on the first embedding vector and the second embedding vector, and predict the first embedding vector to determine multiple diagnostic sub-results of the bone image to be diagnosed.

[0192] Step 408: Generate a target diagnostic result of the bone image to be diagnosed based on the target prediction weight and the multiple diagnostic sub-results.

[0193] In a specific embodiment of the present specification, the bone image to be diagnosed can be understood as a CT image that needs to be diagnosed for osteoporosis, and the bone image to be diagnosed can be a bone image that has been preprocessed and cropped. The bone image to be diagnosed and its corresponding target device information are input into the target prediction model, and the target device information is the identification information of the DXA device of the DXA measurement data. The first embedding vector of the bone image to be diagnosed, that is, the embedding vector of the image, and the second embedding vector of the target device information, that is, the embedding vector of the device, are determined by the target prediction model. The target prediction weights of multiple target prediction modules, that is, the expert model, in the target prediction model are calculated based on the first embedding vector and the second embedding vector. The target diagnostic result output by the target prediction model is calculated based on the diagnostic sub-results output by each expert model and its corresponding target prediction weights. The target diagnostic result includes diagnostic information such as bone density value, diagnostic classification, and anatomical position.

[0194] This specification provides an image processing method that, through a target prediction model, can fully extract texture features from bone images, ensuring the integrity of bone texture information. Attention weights are calculated based on a first and second embedding vectors, and the model's diagnostic results are output using the attention weights. This reduces diagnostic bias caused by differences in measurement data from different imaging devices, improves the model's generalization capabilities for diagnosing images from different devices, and enhances the accuracy of bone image diagnoses.

[0195] Corresponding to the above method embodiment, this specification also provides an image processing device embodiment, Figure 5 FIG1 shows a schematic diagram of the structure of an image processing device provided by an embodiment of this specification. Figure 5 As shown, the device includes:

[0196] The input module 502 is configured to input the bone image to be diagnosed and the target device information of the bone image to be diagnosed into the target prediction model.

[0197] The determination module 504 is configured to determine a first embedding vector of the bone image to be diagnosed and a second embedding vector of the target device information through the target prediction model.

[0198] The calculation module 506 is configured to calculate the target prediction weights of multiple target prediction modules in the target prediction model based on the first embedding vector and the second embedding vector, and predict the first embedding vector to determine multiple diagnostic sub-results of the bone image to be diagnosed.

[0199] The generating module 508 is configured to generate a target diagnostic result of the bone image to be diagnosed based on the target prediction weight and the multiple diagnostic sub-results.

[0200] This specification provides an image processing device that, through a target prediction model, can fully extract texture features from bone images, ensuring the integrity of bone texture information. Attention weights are calculated based on a first and second embedding vectors, and the model's diagnostic results are output using the attention weights. This reduces diagnostic bias caused by differences in measurement data from different image acquisition devices, improves the model's generalization capabilities for diagnosing images from different devices, and enhances the accuracy of bone image diagnoses.

[0201] See also Figure 6 , Figure 6 A flowchart of a computer-aided diagnosis method for osteoporosis provided according to one embodiment of this specification is shown, which specifically includes the following steps.

[0202] Step 602: Input the bone image to be diagnosed and the target device information of the bone image to be diagnosed into the target prediction model, wherein the target prediction model is obtained by training using the above-mentioned model training method.

[0203] Step 604: Determine a first embedding vector of the bone image to be diagnosed and a second embedding vector of the target device information through the target prediction model.

[0204] Step 606: Calculate target prediction weights of multiple target prediction modules in the target prediction model based on the first embedding vector and the second embedding vector, and predict the first embedding vector to determine multiple diagnostic sub-results of the bone image to be diagnosed.

[0205] Step 608: Generate an osteoporosis diagnosis result of the bone image to be diagnosed based on the target prediction weight and the multiple diagnosis sub-results.

[0206] In a specific embodiment of this specification, a target prediction model obtained by training the model training method provided in the above embodiment can realize computer-aided diagnosis of osteoporosis. Specifically, the target prediction model includes inputting the bone image to be diagnosed and the target device information of the bone image to be diagnosed into the target prediction model. The target prediction model can be deployed in a computer device, including but not limited to a personal computer, a smart phone, a workstation, a server, etc. After receiving the bone image to be diagnosed and the target device information, the target prediction model can determine a first embedding vector of the bone image to be diagnosed and a second embedding vector of the target device information, that is, a first embedding vector obtained by encoding the CT bone image to be diagnosed and a second embedding vector obtained by encoding the DXA device identification information associated with the CT bone image. Then, based on the first embedding vector and the second embedding vector, the target prediction weights of multiple target prediction modules in the target prediction model are calculated, and the first embedding vector is subjected to diagnostic prediction of osteoporosis according to the target prediction weights of the multiple target prediction modules to determine multiple diagnostic sub-results. Finally, the osteoporosis diagnostic result of the bone image to be diagnosed is generated based on the target prediction weights and the multiple diagnostic sub-results.

[0207] The computer-aided diagnosis method for osteoporosis provided in this specification achieves the complete extraction of texture features from bone images through a target prediction model, ensuring the integrity of bone texture information. Attention weights are calculated based on the first and second embedding vectors and used to output the model's diagnostic results. This reduces diagnostic bias caused by differences in measurement data from different imaging devices, improves the model's generalization ability for diagnosing images from different devices, and enhances the accuracy of osteoporosis diagnosis using bone images.

[0208] Corresponding to the above method embodiment, this specification also provides an embodiment of a computer-aided diagnosis device for osteoporosis. Figure 7 FIG1 shows a schematic diagram of the structure of a computer-aided diagnosis device for osteoporosis provided by an embodiment of this specification. Figure 7 As shown, the device includes:

[0209] The input module 702 is configured to input the bone image to be diagnosed and the target device information of the bone image to be diagnosed into the target prediction model, wherein the target prediction model is obtained by training through the above-mentioned model training method.

[0210] The determination module 704 is configured to determine a first embedding vector of the bone image to be diagnosed and a second embedding vector of the target device information through the target prediction model.

[0211] The calculation module 706 is configured to calculate the target prediction weights of multiple target prediction modules in the target prediction model based on the first embedding vector and the second embedding vector, and predict the first embedding vector to determine multiple diagnostic sub-results of the bone image to be diagnosed.

[0212] The generating module 708 is configured to generate an osteoporosis diagnosis result of the bone image to be diagnosed based on the target prediction weight and the multiple diagnosis sub-results.

[0213] This specification provides a computer-aided osteoporosis diagnosis device that fully extracts texture features from bone images through a target prediction model, ensuring the integrity of bone texture information. Attention weights are calculated based on a first and second embedding vectors, and the model's diagnostic results are output using the attention weights. This reduces diagnostic bias caused by differences in measurement data from different imaging devices, improves the model's generalization ability for diagnosing images from different devices, and enhances the accuracy of osteoporosis diagnosis using bone images.

[0214] See also Figure 8 , Figure 8 A flowchart of a computer-aided diagnosis method provided according to an embodiment of the present specification is shown. The method is applied to a medical system client and specifically includes the following steps.

[0215] Step 802: Receive the bone image to be diagnosed uploaded by the user, and input the bone image to be diagnosed and the target device information of the bone image to be diagnosed into the target prediction model, wherein the target prediction model is obtained by training through the above-mentioned model training method.

[0216] Step 804: Determine a first embedding vector of the bone image to be diagnosed and a second embedding vector of the target device information through the target prediction model.

[0217] Step 806: Calculate target prediction weights of multiple target prediction modules in the target prediction model based on the first embedding vector and the second embedding vector, and predict the first embedding vector to determine multiple diagnostic sub-results of the bone image to be diagnosed.

[0218] Step 808: Generate a target diagnosis result of the bone image to be diagnosed based on the target prediction weight and the multiple diagnosis sub-results, and return the target diagnosis result to the user.

[0219] The computer-aided diagnosis method provided in this specification can be applied to a medical system client, and the medical system client is deployed with a target prediction model obtained by training the model training method provided in the above embodiment. The medical system client can be an intelligent medical diagnostic device client for osteoporosis diagnosis. The user or patient can upload the bone image to be diagnosed obtained by CT scanning to the medical system client. The specific uploading method can be through electronic uploading or picture scanning uploading, etc. After the medical system client receives the bone image to be diagnosed and the corresponding target device information, the bone image to be diagnosed and the target device information are input into the target prediction model. The first embedding vector of the bone image to be diagnosed and the second embedding vector of the target device information are determined by the target prediction model, and the target prediction weights of multiple target prediction modules in the target prediction model are calculated based on the first embedding vector and the second embedding vector. The first embedding vector is used to perform osteoporosis diagnosis prediction according to the target prediction weights of the multiple target prediction modules to determine multiple diagnostic sub-results. Finally, the osteoporosis diagnosis result of the bone image to be diagnosed is generated according to the target prediction weights and the multiple diagnostic sub-results.

[0220] In a specific embodiment of this specification, a medical system client may be a device client deployed in a hospital for auxiliary diagnosis of osteoporosis. A doctor may upload a patient's skeletal image to be diagnosed to the medical system client, which will automatically analyze and diagnose the skeletal image and generate a corresponding osteoporosis diagnosis result. The osteoporosis diagnosis result will be returned to the doctor. The specific return method may include directly displaying the diagnosis result on the display screen of the medical system client. The doctor or patient may choose to download the diagnosis report to obtain a paper report. Implementing computer-aided diagnosis of osteoporosis through the medical system client can not only reduce the diagnostic pressure on doctors, but also improve the efficiency of patients' medical treatment.

[0221] The computer-aided diagnosis method provided in this specification is applied to the medical system client. Through the target prediction model, the texture features can be completely extracted from the bone image to ensure the integrity of the bone texture information. The attention weight is calculated based on the first embedding vector and the second embedding vector, and the diagnosis result of the model is output using the attention weight. This reduces the problem of diagnostic bias caused by the difference in measurement data caused by different image acquisition devices, improves the generalization ability of the model when diagnosing images from different devices, and improves the accuracy of osteoporosis diagnosis for bone images. The osteoporosis diagnosis result is automatically analyzed by the medical system client. While improving the diagnostic accuracy, it can not only reduce the diagnostic pressure of doctors, but also provide patients with a convenient and fast diagnostic process.

[0222] Corresponding to the above method embodiment, this specification also provides a computer-aided diagnosis device embodiment, Figure 9FIG1 shows a schematic diagram of the structure of a computer-aided diagnosis device provided by an embodiment of this specification. Figure 9 As shown, the device is applied to a medical system client, including:

[0223] The receiving module 902 is configured to receive the bone image to be diagnosed uploaded by the user, and input the bone image to be diagnosed and the target device information of the bone image to be diagnosed into the target prediction model, wherein the target prediction model is obtained by training through the above-mentioned model training method.

[0224] The determination module 904 is configured to determine a first embedding vector of the bone image to be diagnosed and a second embedding vector of the target device information through the target prediction model.

[0225] The calculation module 906 is configured to calculate the target prediction weights of multiple target prediction modules in the target prediction model based on the first embedding vector and the second embedding vector, and predict the first embedding vector to determine multiple diagnostic sub-results of the bone image to be diagnosed.

[0226] The return module 908 is configured to generate a target diagnosis result of the bone image to be diagnosed based on the target prediction weight and the multiple diagnosis sub-results, and return the target diagnosis result to the user.

[0227] This specification provides a computer-aided diagnosis device, which is applied to a medical system client. Through the target prediction model, it can completely extract texture features from bone images to ensure the integrity of bone texture information. The attention weight is calculated based on the first embedding vector and the second embedding vector, and the attention weight is used to output the diagnosis result of the model, which reduces the problem of diagnostic deviation caused by the difference in measurement data caused by different image acquisition devices, improves the generalization ability of the model when diagnosing images from different devices, and improves the accuracy of osteoporosis diagnosis of bone images. The osteoporosis diagnosis result is automatically analyzed by the medical system client, which not only reduces the diagnostic pressure of doctors while improving the diagnostic accuracy, but also provides patients with a convenient and fast diagnostic process.

[0228] See also Figure 10 , Figure 10 A flowchart of a computer-aided diagnosis method provided according to an embodiment of the present specification is shown. The method is applied to a medical system server and specifically includes the following steps.

[0229] Step 1002: Receive the diagnostic task sent by the terminal device, determine the bone image to be diagnosed corresponding to the diagnostic task, and input the bone image to be diagnosed and the target device information of the bone image to be diagnosed into the target prediction model, wherein the target prediction model is obtained by training using the above-mentioned model training method.

[0230] Step 1004: Obtain a target diagnosis result output by the target prediction model based on the bone image to be diagnosed, and send the target diagnosis result to the terminal-side device.

[0231] In a specific embodiment of this specification, the medical system server can be understood as a server that provides osteoporosis auxiliary diagnosis services. The medical system server can be a cloud computing device deployed in the cloud, or a computing device deployed locally. The end-side device can be understood as a terminal device for users to use osteoporosis auxiliary diagnosis services. After receiving the diagnostic task sent by the end-side device, the medical system server deployed by the service provider performs osteoporosis auxiliary diagnosis on the bone image to be diagnosed through the deployed target prediction model, and returns the obtained target diagnosis results to the end-side device. This enables users to use osteoporosis auxiliary diagnosis services through end-side devices, providing users with a convenient, fast and highly accurate diagnostic method.

[0232] This specification provides a computer-aided diagnosis method, which is applied to the medical system server. Through the target prediction model, it can completely extract texture features from bone images to ensure the integrity of bone texture information. The attention weight is calculated based on the first embedding vector and the second embedding vector, and the attention weight is used to output the diagnosis result of the model. This reduces the problem of diagnostic bias caused by differences in measurement data caused by different image acquisition devices, improves the generalization ability of the model when diagnosing images from different devices, and improves the accuracy of osteoporosis diagnosis in bone images. The osteoporosis diagnosis result is automatically analyzed by the medical system server, providing users with a convenient, fast and highly accurate diagnostic service.

[0233] Corresponding to the above method embodiment, this specification also provides a computer-aided diagnosis device embodiment, Figure 11 FIG1 shows a schematic diagram of the structure of a computer-aided diagnosis device provided by an embodiment of this specification. Figure 11 As shown, the device is applied to the medical system server, including:

[0234] The receiving module 1102 is configured to receive the diagnostic task sent by the terminal device, determine the bone image to be diagnosed corresponding to the diagnostic task, and input the bone image to be diagnosed and the target device information of the bone image to be diagnosed into the target prediction model, wherein the target prediction model is obtained by training through the above-mentioned model training method.

[0235] The sending module 1104 is configured to obtain a target diagnosis result output by the target prediction model based on the bone image to be diagnosed, and send the target diagnosis result to the terminal side device.

[0236] This specification provides a computer-aided diagnosis device, which is applied to the medical system server. Through the target prediction model, it can completely extract texture features from bone images to ensure the integrity of bone texture information. The attention weight is calculated based on the first embedding vector and the second embedding vector, and the attention weight is used to output the diagnosis result of the model. This reduces the problem of diagnostic bias caused by differences in measurement data caused by different image acquisition devices, improves the generalization ability of the model when diagnosing images from different devices, and improves the accuracy of osteoporosis diagnosis in bone images. The osteoporosis diagnosis result is automatically analyzed by the medical system server, providing users with a convenient, fast and highly accurate diagnostic service.

[0237] See also Figure 12 , Figure 12 A flowchart of a model training method provided according to an embodiment of this specification is shown. The method is applied to a cloud-side device and specifically includes the following steps.

[0238] Step 1202: Receive the model training task sent by the end-side device and determine the sample bone image corresponding to the model training task.

[0239] Step 1204: Input the sample bone image and the device information of the sample bone image into the initial prediction model for processing to obtain an image embedding vector and a device embedding vector.

[0240] Step 1206: Calculate prediction weights of multiple prediction modules in the initial prediction model based on the image embedding vector and the device embedding vector.

[0241] Step 1208: Generate predictive diagnosis information of the initial prediction model according to the prediction weights of the multiple prediction modules, and optimize the initial prediction model according to the predictive diagnosis information and the imaging diagnosis information to obtain a target prediction model.

[0242] Step 1210: Send the model parameters of the target prediction model to the end-side device, wherein the model parameters are used to deploy the target prediction model on the medical system client.

[0243] In a specific embodiment of this specification, the cloud-side device can be understood as a cloud computing device that provides model training services, and the end-side device can be understood as a terminal device for users to use the model training service. The service provider can receive the model training task sent by the end-side device. The model training task carries sample data for training, namely sample bone images. The sample bone images can be understood as sample data used for model training. The sample bone images correspond to the associated device information. The sample bone images and the device information are input into the initial prediction model to obtain the image embedding vector and the device embedding vector. The prediction weights of multiple prediction modules in the initial prediction model are calculated through the image embedding vector and the device embedding vector. Predictive diagnosis information is generated based on the multiple prediction weights, and the initial prediction model is optimized using the predictive diagnosis information and the image diagnosis information to obtain the target prediction model. The target prediction model can be understood as the prediction model that completes the model training task. After the model parameters of the target prediction model are returned to the end-side device, the end-side device can construct the corresponding target prediction model based on the returned model parameters. Specifically, the target prediction model can be deployed on the relevant medical system client according to the model parameters.

[0244] This specification provides a model training method that is applied to cloud-side devices, realizing the provision of model training services through cloud-side devices, and providing users with a convenient and fast model training method. The target prediction model obtained by training using this method can reduce the problem of diagnostic deviation caused by measurement data differences caused by different image acquisition devices, improve the model's generalization ability when diagnosing images from different devices, and improve the accuracy of bone image diagnosis.

[0245] Corresponding to the above method embodiment, this specification also provides a model training device embodiment, Figure 13 FIG1 shows a schematic diagram of the structure of a model training device provided by an embodiment of this specification. Figure 13 As shown, the device is applied to cloud-side equipment and includes:

[0246] The receiving module 1302 is configured to receive the model training task sent by the end-side device and determine the sample bone image corresponding to the model training task.

[0247] The input module 1304 is configured to input the sample bone image and the device information of the sample bone image into the initial prediction model for processing to obtain an image embedding vector and a device embedding vector.

[0248] The calculation module 1306 is configured to calculate the prediction weights of multiple prediction modules in the initial prediction model according to the image embedding vector and the device embedding vector.

[0249] The optimization module 1308 is configured to generate the prediction diagnosis information of the initial prediction model according to the prediction weights of the multiple prediction modules, and optimize the initial prediction model according to the prediction diagnosis information and the imaging diagnosis information to obtain a target prediction model.

[0250] The sending module 1310 is configured to send the model parameters of the target prediction model to the end-side device, wherein the model parameters are used to deploy the target prediction model on the medical system client.

[0251] The model training device provided in this specification is applied to cloud-side devices, realizing the provision of model training services through cloud-side devices, and providing users with a convenient and fast model training method. The target prediction model obtained by training using this method can reduce the problem of diagnostic deviation caused by measurement data differences caused by different image acquisition devices, improve the model's generalization ability when diagnosing images from different devices, and improve the accuracy of bone image diagnosis.

[0252] Figure 14 14 shows a block diagram of a computing device 1400 according to one embodiment of the present disclosure. Components of the computing device 1400 include, but are not limited to, a memory 1410 and a processor 1420. The processor 1420 is connected to the memory 1410 via a bus 1430, and a database 1450 is used to store data.

[0253] The computing device 1400 also includes an access device 1440 that enables the computing device 1400 to communicate via one or more networks 1460. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1440 may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, or a near field communication (NFC) interface.

[0254] In one embodiment of the present specification, the above components of the computing device 1400 and Figure 14 Other components not shown in the figure may also be connected to each other, for example, via a bus. Figure 14 The computing device structure block diagram shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art may add or replace other components as needed.

[0255] Computing device 1400 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, personal digital assistant, laptop computer, notebook computer, netbook computer, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or personal computer (PC). Computing device 1400 may also be a mobile or stationary server.

[0256] Among them, the processor 1420 is used to execute the following computer-executable instructions, which, when executed by the processor, implement the above-mentioned model training method, image processing method, osteoporosis computer-aided diagnosis method, and computer-aided diagnosis method steps.

[0257] The above is a schematic diagram of a computing device according to this embodiment. It should be noted that the technical solution of this computing device is based on the same concept as the technical solutions of the aforementioned model training method, image processing method, osteoporosis computer-aided diagnosis method, and computer-aided diagnosis method. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solutions of the aforementioned model training method, image processing method, osteoporosis computer-aided diagnosis method, and computer-aided diagnosis method.

[0258] An embodiment of the present specification also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-mentioned model training method, image processing method, osteoporosis computer-aided diagnosis method, and computer-aided diagnosis method steps.

[0259] The above is a schematic diagram of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium is based on the same concept as the technical solutions of the aforementioned model training method, image processing method, osteoporosis computer-aided diagnosis method, and computer-aided diagnosis method. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solutions of the aforementioned model training method, image processing method, osteoporosis computer-aided diagnosis method, and computer-aided diagnosis method.

[0260] An embodiment of the present specification also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the above-mentioned model training method, image processing method, osteoporosis computer-aided diagnosis method, and steps of the computer-aided diagnosis method.

[0261] The above is a schematic diagram of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product is based on the same concept as the technical solutions of the aforementioned model training method, image processing method, osteoporosis computer-aided diagnosis method, and computer-aided diagnosis method. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solutions of the aforementioned model training method, image processing method, osteoporosis computer-aided diagnosis method, and computer-aided diagnosis method.

[0262] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0263] The computer instructions include computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media does not include electric carrier signals and telecommunication signals.

[0264] It should be noted that for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0265] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0266] The preferred embodiments disclosed above are intended only to help illustrate this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations are possible based on the content of the embodiments described herein. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification.

Claims

1. A model training method, comprising: determining target texture features of the target bone image, and adjusting the pre-trained model to an initial prediction model based on the target texture features; Inputting the second bone image and the device information of the second bone image into the initial prediction model for processing to obtain an image embedding vector and a device embedding vector; Calculating prediction weights of multiple prediction modules in the initial prediction model according to the image embedding vector and the device embedding vector; The prediction diagnosis information of the initial prediction model is generated according to the prediction weights of the multiple prediction modules, and the initial prediction model is optimized according to the prediction diagnosis information and the imaging diagnosis information to obtain a target prediction model.

2. The method according to claim 1, before determining the target texture features of the target bone image, the method further comprises: determining an initial skeletal image and skeletal measurement data associated with the initial skeletal image; A target bone image is determined in the initial bone image based on the bone measurement data.

3. The method according to claim 1, wherein determining target texture features of the target bone image comprises: Perform window sliding processing on the target bone image to obtain multiple local bone images; Local texture features of the multiple local bone images are extracted, and target texture features of the target bone image are calculated based on the local texture features of the multiple local bone images.

4. The method according to claim 3, wherein the target texture features of the target bone image are calculated based on the local texture features of the multiple local bone images, comprising: generating an initial feature matrix according to the local texture features of the plurality of local bone images; The initial feature matrix is ​​adjusted according to the image attribute information of the target bone image to obtain a target feature matrix, and the target texture feature of the target bone image is determined according to the target feature matrix.

5. The method according to claim 1, adjusting the pre-trained model to the initial prediction model based on the target texture feature, comprising: Encoding the target texture features using a pre-trained model to obtain a texture embedding vector, and calculating prototype distribution information corresponding to the texture embedding vector; The model parameters of the pre-trained model are adjusted based on the prototype distribution information to obtain an initial prediction model.

6. The method according to claim 5, adjusting the model parameters of the pre-trained model based on the prototype distribution information to obtain an initial prediction model, comprising: Calculating exchange prediction loss and clustering regularization loss based on the prototype distribution information; The model parameters of the pre-trained model are adjusted based on the exchange prediction loss and the cluster regularization loss to obtain an initial prediction model.

7. The method according to claim 1, wherein the step of calculating prediction weights of multiple prediction modules in the initial prediction model based on the image embedding vector and the device embedding vector comprises: fusing the image embedding vector and the device embedding vector to obtain a fused embedding vector; An attention weight corresponding to the initial prediction model is calculated according to the fused embedding vector, and prediction weights of multiple prediction modules in the initial prediction model are determined based on the attention weight.

8. The method according to claim 1, generating the prediction diagnosis information of the initial prediction model according to the prediction weights of the plurality of prediction modules, comprising: Processing the image embedding vector by the multiple prediction modules to obtain prediction diagnosis sub-information of the multiple prediction modules; The predicted diagnosis information of the initial prediction model is calculated according to the prediction weights of the multiple prediction modules and the predicted diagnosis sub-information of the multiple prediction modules.

9. The method according to claim 1, wherein optimizing the initial prediction model based on the predicted diagnosis information and the imaging diagnosis information to obtain the target prediction model comprises: Determining image diagnostic information corresponding to the second skeletal image, and calculating a prediction task loss and a device regularization loss based on the predicted diagnostic information and the image diagnostic information; The model parameters of the initial prediction model are adjusted according to the prediction task loss and the device regularization loss to obtain a target prediction model.

10. An image processing method, the method being applied to a target prediction model obtained by training using the model training method according to any one of claims 1 to 9, comprising: Inputting the bone image to be diagnosed and the target device information of the bone image to be diagnosed into the target prediction model; Determining a first embedding vector of the bone image to be diagnosed and a second embedding vector of the target device information by using the target prediction model; Calculating target prediction weights of multiple target prediction modules in the target prediction model according to the first embedding vector and the second embedding vector, and predicting the first embedding vector to determine multiple diagnostic sub-results of the bone image to be diagnosed; A target diagnosis result of the bone image to be diagnosed is generated based on the target prediction weight and the multiple diagnosis sub-results.

11. A computer-aided diagnosis method for osteoporosis, comprising: Inputting the bone image to be diagnosed and the target device information of the bone image to be diagnosed into the target prediction model, wherein the target prediction model is obtained by training the model training method according to any one of claims 1 to 9; Determining a first embedding vector of the bone image to be diagnosed and a second embedding vector of the target device information by using the target prediction model; Calculating target prediction weights of multiple target prediction modules in the target prediction model according to the first embedding vector and the second embedding vector, and predicting the first embedding vector to determine multiple diagnostic sub-results of the bone image to be diagnosed; An osteoporosis diagnosis result of the bone image to be diagnosed is generated based on the target prediction weight and the multiple diagnosis sub-results.

12. A computer-aided diagnosis method, applied to a medical system client, comprising: Receiving a bone image to be diagnosed uploaded by a user, and inputting the bone image to be diagnosed and target device information of the bone image to be diagnosed into the target prediction model, wherein the target prediction model is obtained by training the model training method according to any one of claims 1 to 9; Determining a first embedding vector of the bone image to be diagnosed and a second embedding vector of the target device information by using the target prediction model; Calculating target prediction weights of multiple target prediction modules in the target prediction model according to the first embedding vector and the second embedding vector, and predicting the first embedding vector to determine multiple diagnostic sub-results of the bone image to be diagnosed; A target diagnosis result of the bone image to be diagnosed is generated based on the target prediction weight and the multiple diagnosis sub-results, and the target diagnosis result is returned to the user.

13. A computer-aided diagnosis method, applied to a medical system server, comprising: A diagnostic task sent by a receiving terminal device is determined, a skeletal image to be diagnosed corresponding to the diagnostic task is determined, and the skeletal image to be diagnosed and target device information of the skeletal image to be diagnosed are input into the target prediction model, wherein the target prediction model is obtained by training the model training method according to any one of claims 1 to 9; Obtain a target diagnosis result output by the target prediction model based on the bone image to be diagnosed, and send the target diagnosis result to the terminal side device.

14. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the steps of the method according to any one of claims 1 to 13.

15. A computer program product comprising a computer program or instructions, which implement the steps of the method according to any one of claims 1 to 13 when executed by a processor.

Images