Orthopedic disease auxiliary detection method, device, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence, and provides an orthopedic disease auxiliary detection method, device, equipment and medium, on one hand, according to the real lesion category of each X-ray image and a medical prompt generation module, text description of each X-ray image can be generated, and important information related to lesions is provided for subsequent model training; on the other hand, based on the X-ray images, the text description of each X-ray image and the training of the semantic learning module based on the matching degree guidance, an orthopedic lesion detection network is obtained, and under the guidance of medical prompts, the text-image alignment capability of the semantic learning module based on the matching degree guidance is used as a bridge from a text mode to an image mode; therefore, the orthopedic disease detection performance is improved, the medical staff is assisted in quickly reading the image, and the accuracy of orthopedic disease detection is indirectly improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an orthopedic disease auxiliary detection method, device, equipment and medium. Background Art

[0002] With the prevalence of electronic devices such as mobile phones and computers, people's work and lifestyles have changed. Long-term use of electronic devices with poor posture has led to an increase in the incidence of spinal diseases, which is becoming younger. Furthermore, the number of fractures caused by daily activities, sports injuries, traffic accidents, etc. is also increasing. Fractures not only seriously affect patients' quality of life, but if not diagnosed promptly and accurately, they may also lead to misaligned healing, functional impairment, and even disability. Therefore, early screening and accurate diagnosis and treatment of orthopedic diseases are crucial to slowing the progression of the disease and improving patients' quality of life.

[0003] In recent years, deep learning-based X-ray image-based orthopedic disease diagnosis methods have made significant progress. However, existing methods can only mine lesion information from images and have not explored the paradigm of using text descriptions of lesions to improve diagnostic accuracy. Summary of the Invention

[0004] In view of the above, it is necessary to provide an auxiliary detection method, device, equipment and medium for orthopedic diseases, aiming to solve the problem of low accuracy of existing auxiliary detection methods for orthopedic diseases.

[0005] An auxiliary detection method for orthopedic diseases, comprising:

[0006] Constructing the initial network, medical prompt generation module, and matching-guided semantic learning module;

[0007] Collecting X-ray images; wherein each X-ray image is labeled with a true lesion category and a true lesion location bounding box of an orthopedic disease;

[0008] generating a text description of each X-ray image based on the actual lesion category of each X-ray image and the medical prompt generation module;

[0009] training the initial network based on the X-ray images, the text description of each X-ray image, and the semantic learning module to obtain an orthopedic lesion detection network;

[0010] In response to an orthopedic disease auxiliary detection instruction for a target X-ray image, the target X-ray image is input into the orthopedic lesion detection network for processing, a detection result of the target X-ray image is obtained, and the detection result is sent to the trigger of the orthopedic disease auxiliary detection instruction.

[0011] According to a preferred embodiment of the present invention, the constructing of the initial network includes:

[0012] Build a backbone network based on the backbone of the YoLov8 network;

[0013] Construct a path aggregation feature pyramid network based on the neck part of the YoLov8 network;

[0014] Construct classification branch and positioning branch based on the detection head of YoLov8 network;

[0015] Construct semantic branches based on convolutional layers and activation function layers;

[0016] The backbone network, the path aggregation feature pyramid network, the classification branch, the positioning branch and the semantic branch are combined to obtain the initial network.

[0017] According to a preferred embodiment of the present invention, constructing semantic branches based on the convolution layer and the activation function layer includes:

[0018] Constructing a preset number of sub-branches according to the convolution layer and the activation function layer;

[0019] Connect each sub-branch end to end in sequence to obtain the semantic branch;

[0020] Among them, each sub-branch includes a convolution layer, a normalization function layer and an activation function layer connected end to end.

[0021] According to a preferred embodiment of the present invention, generating a text description of each X-ray image based on the actual lesion category of each X-ray image and the medical prompt generation module includes:

[0022] Get pre-configured medical prompt words;

[0023] Taking the real lesion category of each X-ray image as input, and using the medical prompt words to guide the medical prompt generation module to output the lesion pathological characteristics of each real lesion category and its typical manifestation in the corresponding X-ray image;

[0024] The lesion pathological features of each real lesion category and its typical manifestations in the corresponding X-ray image are determined as a text description of the corresponding X-ray image.

[0025] According to a preferred embodiment of the present invention, the semantic learning module includes a contrastive language-image pre-training model, an image encoder and a text encoder of the contrastive language-image pre-training model;

[0026] The step of training the initial network based on the X-ray images, the text description of each X-ray image, and the semantic learning module to obtain an orthopedic lesion detection network includes:

[0027] During the training process, freezing the contrastive language-image pre-trained model, inputting the X-ray image into the initial network, and obtaining the output data corresponding to each X-ray image in the semantic branch as the semantic feature of each X-ray image;

[0028] Inputting each X-ray image and the corresponding ground-truth lesion position bounding box into the image encoder to obtain a region of interest feature of each X-ray image, and inputting a text description of each X-ray image into the text encoder to obtain a text feature of each X-ray image;

[0029] Utilizing the semantic learning module, calculating a first similarity between a feature of a region of interest of each X-ray image and a corresponding text feature, and calculating a second similarity between a semantic feature of each X-ray image and a corresponding text feature;

[0030] Constructing a total loss function; wherein the total loss function includes a semantic consistency loss function and a detection loss function;

[0031] Using the first similarity as a supervisory signal, and using a semantic consistency loss function to constrain the second similarity, so as to guide the semantic branch to learn the fine-grained semantic information contained in the text description;

[0032] Taking the true lesion category and the true lesion location bounding box marked in each X-ray image as training targets, optimizing the classification branch and the localization branch using the detection loss function to guide the classification branch to learn lesion category information and the localization branch to learn lesion location information;

[0033] When the total loss function reaches convergence, the training is stopped, and the currently obtained network is determined as the orthopedic lesion detection network.

[0034] According to a preferred embodiment of the present invention, constructing the total loss function includes:

[0035] Constructing the semantic consistency loss function based on the Smooth L1 loss function;

[0036] Constructing the detection loss function based on the loss function of the YoLov8 network;

[0037] The sum of the semantic consistency loss function and the detection loss function is calculated to obtain the total loss function.

[0038] According to a preferred embodiment of the present invention, inputting the target X-ray image into the orthopedic lesion detection network for processing to obtain a detection result of the target X-ray image includes:

[0039] After removing the semantic branch from the orthopedic lesion detection network, using the backbone network to obtain multi-scale X-ray image features of the target X-ray image;

[0040] fusing the multi-scale X-ray image features using the path aggregation feature pyramid network to obtain enhanced multi-scale X-ray image features;

[0041] Inputting the enhanced multi-scale X-ray image features into the classification branch and the positioning branch respectively;

[0042] Obtaining the output of the classification branch as an orthopedic disease type prediction result, and obtaining the output of the positioning branch as an orthopedic lesion location prediction result;

[0043] The orthopedic disease type prediction result and the orthopedic lesion location prediction result are integrated to obtain the detection result.

[0044] An auxiliary detection device for orthopedic diseases, comprising:

[0045] Construction unit, used to construct the initial network, medical prompt generation module and matching-guided semantic learning module;

[0046] An acquisition unit, configured to acquire X-ray images; wherein each X-ray image is marked with a true lesion category and a true lesion location bounding box of an orthopedic disease;

[0047] a generating unit, configured to generate a text description of each X-ray image according to the actual lesion category of each X-ray image and the medical prompt generating module;

[0048] a training unit, configured to train the initial network based on the X-ray images, a text description of each X-ray image, and the semantic learning module to obtain an orthopedic lesion detection network;

[0049] A processing unit is used to respond to an orthopedic disease auxiliary detection instruction for a target X-ray image, input the target X-ray image into the orthopedic lesion detection network for processing, obtain a detection result of the target X-ray image, and send the detection result to the trigger of the orthopedic disease auxiliary detection instruction.

[0050] A computer device, comprising:

[0051] a memory storing at least one instruction; and

[0052] A processor executes instructions stored in the memory to implement the orthopedic disease auxiliary detection method.

[0053] A computer-readable storage medium stores at least one instruction, and the at least one instruction is executed by a processor in a computer device to implement the orthopedic disease auxiliary detection method.

[0054] It can be seen from the above technical solutions that, on the one hand, the present invention can generate a text description of each X-ray image based on the actual lesion category of each X-ray image and the medical prompt generation module, providing important information related to the lesion for subsequent model training; on the other hand, an orthopedic lesion detection network is obtained based on X-ray images, text descriptions of each X-ray image and a semantic learning module guided by matching degree. Under the guidance of medical prompts, the text-image alignment capability of the semantic learning module guided by matching degree can be used as a bridge from text modality to image modality to assist the orthopedic lesion detection network in learning the semantic information of lesions, thereby improving the detection performance of orthopedic diseases, assisting medical staff in quickly reading films, and indirectly improving the accuracy of orthopedic disease detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a flow chart of a preferred embodiment of the orthopedic disease auxiliary detection method of the present invention.

[0056] Figure 2 This is a functional module diagram of a preferred embodiment of the orthopedic disease auxiliary detection device of the present invention.

[0057] Figure 3 It is a structural diagram of a computer device according to a preferred embodiment of the present invention for implementing the auxiliary detection method for orthopedic diseases. DETAILED DESCRIPTION

[0058] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0059] like Figure 1 FIG. 1 is a flow chart of a preferred embodiment of the orthopedic disease auxiliary detection method of the present invention. The order of the steps in the flow chart can be changed and some steps can be omitted according to different requirements.

[0060] The orthopedic disease auxiliary detection method is applied to one or more computer devices, which are devices that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. Their hardware includes but is not limited to microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0061] The computer device can be any electronic product that can interact with a user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive network television (IPTV), a smart wearable device, etc.

[0062] The computer device may also include a network device and / or a user device, wherein the network device includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0063] The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0064] Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0065] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0066] The network where the computer device is located includes but is not limited to the Internet, wide area network, metropolitan area network, local area network, virtual private network (VPN), etc.

[0067] S10, construct the initial network, medical prompt generation module and matching-guided semantic learning module.

[0068] In this embodiment, the initial network construction includes:

[0069] Build a backbone network based on the backbone of the YoLov8 (You Only Look Once Version 8) network;

[0070] Construct a path aggregation feature pyramid network based on the neck part of the YoLov8 network;

[0071] Construct classification branch and positioning branch based on the detection head of YoLov8 network;

[0072] Construct semantic branches based on convolutional layers and activation function layers;

[0073] The backbone network, the path aggregation feature pyramid network, the classification branch, the positioning branch and the semantic branch are combined to obtain the initial network.

[0074] Specifically, constructing semantic branches based on the convolution layer and the activation function layer includes:

[0075] Constructing a preset number of sub-branches according to the convolution layer and the activation function layer;

[0076] Connect each sub-branch end to end in sequence to obtain the semantic branch;

[0077] Among them, each sub-branch includes a convolution layer, a normalization function layer and an activation function layer connected end to end.

[0078] For example, the preset number can be configured as 3. In this case, the semantic branch can be expressed as follows:

[0079] Input → first sub-branch (convolution layer → normalization function layer → activation function layer) → second sub-branch (convolution layer → normalization function layer → activation function layer) → third sub-branch (convolution layer → normalization function layer → activation function layer) → output.

[0080] The semantic branch is a self-developed network branch, which can support the orthopedic lesion detection network to learn the semantic information of lesions during subsequent model training.

[0081] In this embodiment, the medical prompt generation module is a large language model supported by professional orthopedic medical knowledge. Through the medical prompt generation module, a text description including the pathological characteristics of the lesion and the typical manifestations in the X-ray image can be obtained.

[0082] In this embodiment, the matching-guided semantic learning module utilizes the text-image matching capability of CLIP (Contrastive Language-Image Pre-training) to enable the orthopedic lesion detection network to better learn fine-grained lesion semantics.

[0083] S11, collecting X-ray images; wherein each X-ray image is marked with a true lesion category and a true lesion location boundary box of an orthopedic disease.

[0084] In this embodiment, the X-ray image may be an X-ray image of an orthopedic disease for which an accurate diagnosis result has been obtained.

[0085] In this embodiment, the marked true lesion category and the true lesion location bounding box can provide data support for subsequent model training.

[0086] S12, generating a text description of each X-ray image according to the actual lesion category of each X-ray image and the medical prompt generation module.

[0087] In this embodiment, generating a text description of each X-ray image based on the actual lesion category of each X-ray image and the medical prompt generation module includes:

[0088] Get pre-configured medical prompt words;

[0089] Taking the real lesion category of each X-ray image as input, and using the medical prompt words to guide the medical prompt generation module to output the lesion pathological characteristics of each real lesion category and its typical manifestation in the corresponding X-ray image;

[0090] The lesion pathological features of each real lesion category and its typical manifestations in the corresponding X-ray image are determined as a text description of the corresponding X-ray image.

[0091] For example, the medical prompt word may be: "In an X-ray image, what are the color, shape, and pathological properties of [cls]?".

[0092] The medical prompt generation module may be a large language model.

[0093] By rationally configuring the medical prompt words, the model can output a more accurate and comprehensive text description of the X-ray image.

[0094] Considering the complexity and diversity of orthopedic lesions, and the language-based model's strong learning capabilities and knowledge reserves, this embodiment can obtain professional text descriptions corresponding to each orthopedic lesion category. For example, osteophytes are bony protrusions formed along the edges of vertebrae, and intervertebral space narrowing refers to the reduction in space between adjacent vertebrae. Using this information helps the auxiliary model to more comprehensively understand the lesions.

[0095] Therefore, this embodiment uses a language-based model to professionally analyze the lesions and output a text description of the lesions, thereby improving the accuracy of orthopedic disease detection and having important clinical application value.

[0096] S13, training the initial network based on the X-ray images, the text description of each X-ray image, and the semantic learning module to obtain an orthopedic lesion detection network.

[0097] In this embodiment, the semantic learning module includes a contrastive language-image pre-training model, an image encoder and a text encoder of the contrastive language-image pre-training model.

[0098] In this embodiment, during the training phase, the image encoder and text encoder of the comparative language-image pre-training model remain frozen, and the initial network remains in a trainable state.

[0099] Specifically, the training of the initial network based on the X-ray images, the text description of each X-ray image, and the semantic learning module to obtain an orthopedic lesion detection network includes:

[0100] During the training process, freezing the contrastive language-image pre-trained model, inputting the X-ray image into the initial network, and obtaining the output data corresponding to each X-ray image in the semantic branch as the semantic feature of each X-ray image;

[0101] Inputting each X-ray image and the corresponding ground-truth lesion position bounding box into the image encoder to obtain a region of interest feature of each X-ray image, and inputting a text description of each X-ray image into the text encoder to obtain a text feature of each X-ray image;

[0102] Utilizing the semantic learning module, calculating a first similarity between a feature of a region of interest of each X-ray image and a corresponding text feature, and calculating a second similarity between a semantic feature of each X-ray image and a corresponding text feature;

[0103] Constructing a total loss function; wherein the total loss function includes a semantic consistency loss function and a detection loss function;

[0104] Using the first similarity as a supervisory signal, and using a semantic consistency loss function to constrain the second similarity, so as to guide the semantic branch to learn the fine-grained semantic information contained in the text description;

[0105] Taking the true lesion category and the true lesion location bounding box marked in each X-ray image as training targets, optimizing the classification branch and the localization branch using the detection loss function to guide the classification branch to learn lesion category information and the localization branch to learn lesion location information;

[0106] When the total loss function reaches convergence, the training is stopped, and the currently obtained network is determined as the orthopedic lesion detection network.

[0107] Wherein, constructing the total loss function includes:

[0108] Constructing the semantic consistency loss function based on the Smooth L1 loss function;

[0109] Constructing the detection loss function based on the loss function of the YoLov8 network;

[0110] The sum of the semantic consistency loss function and the detection loss function is calculated to obtain the total loss function.

[0111] The detection loss function is used to optimize the learning process of the classification branch and the positioning branch, and the semantic consistency loss function is used to optimize the learning process of the semantic branch.

[0112] In the above embodiment, the similarity between the features of the region of interest and the text features is used as the supervision signal, and the semantic consistency loss function is used to constrain the similarity between the semantic features output by the semantic branch in the lesion detection network and the text features, thereby guiding the semantic branch of the orthopedic lesion detection network to learn the fine-grained semantic information contained in the lesion text description, further improving the lesion detection performance.

[0113] S14, in response to the orthopedic disease auxiliary detection instruction for the target X-ray image, the target X-ray image is input into the orthopedic lesion detection network for processing, a detection result of the target X-ray image is obtained, and the detection result is sent to the trigger of the orthopedic disease auxiliary detection instruction.

[0114] In this embodiment, the orthopedic disease auxiliary detection instruction can be triggered by relevant staff according to actual detection needs; the orthopedic disease auxiliary detection instruction can also be automatically triggered when the target X-ray image is detected and uploaded to the designated platform, thereby reducing waiting time.

[0115] In this embodiment, inputting the target X-ray image into the orthopedic lesion detection network for processing to obtain a detection result of the target X-ray image includes:

[0116] After removing the semantic branch from the orthopedic lesion detection network, using the backbone network to obtain multi-scale X-ray image features of the target X-ray image;

[0117] fusing the multi-scale X-ray image features using the path aggregation feature pyramid network to obtain enhanced multi-scale X-ray image features;

[0118] Inputting the enhanced multi-scale X-ray image features into the classification branch and the positioning branch respectively;

[0119] Obtaining the output of the classification branch as an orthopedic disease type prediction result, and obtaining the output of the positioning branch as an orthopedic lesion location prediction result;

[0120] The orthopedic disease type prediction result and the orthopedic lesion location prediction result are integrated to obtain the detection result.

[0121] During the detection stage, it is equivalent to removing the medical prompt generation module, the matching-guided semantic learning module, and the semantic branch of the orthopedic lesion detection network, and only using the backbone network, path aggregation feature pyramid network, classification branch, and regression branch in the orthopedic lesion detection network to complete the auxiliary detection of orthopedic diseases and predict the type and location of orthopedic diseases.

[0122] Through the above embodiments, the detailed and professional orthopedic disease description information provided by the large language model and the text-image alignment capability of the comparative language-image pre-training model can be combined to enable the model to better understand the semantic information of the lesion, thereby improving the detection performance of orthopedic diseases.

[0123] In this embodiment, the trigger may be relevant medical staff.

[0124] Sending the test results to the trigger of the orthopedic disease auxiliary detection instruction in a timely manner can assist relevant medical staff in quickly reading the films. At the same time, the improvement of model performance also indirectly improves the accuracy of orthopedic disease detection.

[0125] It can be seen from the above technical solutions that, on the one hand, the present invention can generate a text description of each X-ray image based on the actual lesion category of each X-ray image and the medical prompt generation module, providing important information related to the lesion for subsequent model training; on the other hand, an orthopedic lesion detection network is obtained based on X-ray images, text descriptions of each X-ray image and a semantic learning module guided by matching degree. Under the guidance of medical prompts, the text-image alignment capability of the semantic learning module guided by matching degree can be used as a bridge from text modality to image modality to assist the orthopedic lesion detection network in learning the semantic information of lesions, thereby improving the detection performance of orthopedic diseases, assisting medical staff in quickly reading films, and indirectly improving the accuracy of orthopedic disease detection.

[0126] like Figure 2 , which is a functional module diagram of a preferred embodiment of the orthopedic disease auxiliary detection device of the present invention. The orthopedic disease auxiliary detection device 11 comprises a construction unit 110, an acquisition unit 111, a generation unit 112, a training unit 113, and a processing unit 114. As used herein, a module / unit refers to a series of computer program segments that can be executed by a processor and perform a fixed function, and are stored in a memory. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0127] The construction unit 110 is used to construct an initial network, a medical prompt generation module, and a semantic learning module guided by matching degree;

[0128] The acquisition unit 111 is used to acquire X-ray images; wherein each X-ray image is marked with the actual lesion category and the actual lesion location boundary box of the orthopedic disease;

[0129] The generating unit 112 is configured to generate a text description of each X-ray image based on the actual lesion category of each X-ray image and the medical prompt generating module;

[0130] The training unit 113 is configured to train the initial network based on the X-ray images, the text description of each X-ray image, and the semantic learning module to obtain an orthopedic lesion detection network;

[0131] The processing unit 114 is used to respond to the orthopedic disease auxiliary detection instruction for the target X-ray image, input the target X-ray image into the orthopedic lesion detection network for processing, obtain the detection result of the target X-ray image, and send the detection result to the trigger of the orthopedic disease auxiliary detection instruction.

[0132] It can be seen from the above technical solutions that, on the one hand, the present invention can generate a text description of each X-ray image based on the actual lesion category of each X-ray image and the medical prompt generation module, providing important information related to the lesion for subsequent model training; on the other hand, an orthopedic lesion detection network is obtained based on X-ray images, text descriptions of each X-ray image and a semantic learning module guided by matching degree. Under the guidance of medical prompts, the text-image alignment capability of the semantic learning module guided by matching degree can be used as a bridge from text modality to image modality to assist the orthopedic lesion detection network in learning the semantic information of lesions, thereby improving the detection performance of orthopedic diseases, assisting medical staff in quickly reading films, and indirectly improving the accuracy of orthopedic disease detection.

[0133] like Figure 3 FIG. 1 is a schematic diagram of the structure of a computer device for implementing a preferred embodiment of the orthopedic disease auxiliary detection method of the present invention.

[0134] The computer device 1 may include a memory 12, a processor 13 and a bus (the arrow in the figure represents the bus), and may also include a computer program stored in the memory 12 and executable on the processor 13, such as an auxiliary detection program for orthopedic diseases.

[0135] Those skilled in the art will understand that the schematic diagram is merely an example of the computer device 1 and does not constitute a limitation on the computer device 1. The computer device 1 may have either a bus structure or a star structure. The computer device 1 may also include more or less other hardware or software than shown in the figure, or a different arrangement of components. For example, the computer device 1 may also include input and output devices, network access devices, etc.

[0136] It should be noted that the computer device 1 is only an example. Other existing or future electronic products that are suitable for the present invention should also be included in the scope of protection of the present invention and included here by reference.

[0137] Among them, the memory 12 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example, SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 12 can be an internal storage unit of the computer device 1, such as a mobile hard disk of the computer device 1. In other embodiments, the memory 12 can also be an external storage device of the computer device 1, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device 1. Furthermore, the memory 12 can also include both an internal storage unit of the computer device 1 and an external storage device. The memory 12 can not only be used to store application software and various types of data installed in the computer device 1, such as the code of the orthopedic disease auxiliary detection program, but can also be used to temporarily store data that has been output or is to be output.

[0138] In some embodiments, the processor 13 may be composed of an integrated circuit, such as a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor 13 is the control core (Control Unit) of the computer device 1, connecting the various components of the entire computer device 1 using various interfaces and lines. It executes or executes programs or modules stored in the memory 12 (such as executing an orthopedic disease auxiliary detection program) and calls data stored in the memory 12 to perform various functions of the computer device 1 and process data.

[0139] The processor 13 executes the operating system of the computer device 1 and various installed applications. The processor 13 executes the applications to implement the steps in the above-mentioned embodiments of the auxiliary detection method for orthopedic diseases, for example Figure 1 Steps shown.

[0140] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to implement the present invention. The one or more modules / units may be a series of computer-readable instruction segments capable of performing specific functions, which are used to describe the execution process of the computer program in the computer device 1. For example, the computer program may be divided into a construction unit 110, an acquisition unit 111, a generation unit 112, a training unit 113, and a processing unit 114.

[0141] The above-mentioned integrated unit implemented in the form of a software functional module can be stored in a computer-readable storage medium. The above-mentioned software functional module is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, computer equipment, or network equipment, etc.) or a processor to execute the portion of the orthopedic disease auxiliary detection method described in various embodiments of the present invention.

[0142] If the modules / units integrated in the computer device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the present invention can also implement all or part of the processes in the above-mentioned method embodiments by instructing relevant hardware devices through a computer program. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of each of the above-mentioned method embodiments.

[0143] The computer program includes computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory, etc.

[0144] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0145] Blockchain, as used in this article, refers to a novel application model for computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Blockchain is essentially a decentralized database, a series of data blocks generated using cryptographic methods. Each block contains information about a batch of online transactions, used to verify the validity of this information (to prevent counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, the platform product service layer, and the application service layer.

[0146] The bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 The figure shows that only one straight line is used, but it does not mean that there is only one bus or one type of bus. The bus is configured to realize the connection and communication between the memory 12 and at least one processor 13.

[0147] Although not shown, the computer device 1 may also include a power supply (such as a battery) to power various components. Preferably, the power supply can be logically connected to the at least one processor 13 via a power management device, thereby implementing functions such as charging management, discharging management, and power consumption management through the power management device. The power supply may also include one or more DC or AC power supplies, a recharging device, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components. The computer device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be detailed here.

[0148] Furthermore, the computer device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is usually used to establish a communication connection between the computer device 1 and other computer devices.

[0149] Optionally, the computer device 1 may further include a user interface, which may be a display or an input unit (such as a keyboard). Optionally, the user interface may also be a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display may also be appropriately referred to as a display screen or a display unit, and is used to display information processed in the computer device 1 and to display a visual user interface.

[0150] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0151] It will be understood by those skilled in the art that Figure 3 The structure shown does not constitute a limitation on the computer device 1 , and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0152] Combine Figure 1 The memory 12 in the computer device 1 stores a plurality of instructions to implement an orthopedic disease auxiliary detection method, and the processor 13 can execute the plurality of instructions to implement:

[0153] Constructing the initial network, medical prompt generation module, and matching-guided semantic learning module;

[0154] Collecting X-ray images; wherein each X-ray image is labeled with a true lesion category and a true lesion location bounding box of an orthopedic disease;

[0155] generating a text description of each X-ray image based on the actual lesion category of each X-ray image and the medical prompt generation module;

[0156] training the initial network based on the X-ray images, the text description of each X-ray image, and the semantic learning module to obtain an orthopedic lesion detection network;

[0157] In response to an orthopedic disease auxiliary detection instruction for a target X-ray image, the target X-ray image is input into the orthopedic lesion detection network for processing, a detection result of the target X-ray image is obtained, and the detection result is sent to the trigger of the orthopedic disease auxiliary detection instruction.

[0158] Specifically, the specific implementation method of the processor 13 for the above instructions can refer to Figure 1 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0159] It should be noted that the data involved in this case were all obtained legally. The software tools or components not produced by our company that appear in the embodiments of this application are merely examples and do not represent actual use.

[0160] In the several embodiments provided herein, it should be understood that the disclosed systems, devices, and methods may be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the module division is merely a logical functional division, and actual implementation may employ other division methods.

[0161] The present invention can be used in a wide variety of general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0162] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0163] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0164] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0165] Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0166] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices described in the present invention may also be implemented by a single unit or device through software or hardware. Terms such as first and second are used to indicate names and do not imply any particular order.

[0167] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for auxiliary detection of orthopedic diseases, characterized in that: The orthopedic disease auxiliary detection method comprises: Constructing the initial network, medical prompt generation module, and matching-guided semantic learning module; Collecting X-ray images; wherein each X-ray image is labeled with a true lesion category and a true lesion location bounding box of an orthopedic disease; generating a text description of each X-ray image based on the actual lesion category of each X-ray image and the medical prompt generation module; training the initial network based on the X-ray images, the text description of each X-ray image, and the semantic learning module to obtain an orthopedic lesion detection network; In response to an orthopedic disease auxiliary detection instruction for a target X-ray image, the target X-ray image is input into the orthopedic lesion detection network for processing, a detection result of the target X-ray image is obtained, and the detection result is sent to the trigger of the orthopedic disease auxiliary detection instruction.

2. The orthopedic disease auxiliary detection method according to claim 1, characterized in that: The constructing of the initial network includes: Build a backbone network based on the backbone of the YoLov8 network; Construct a path aggregation feature pyramid network based on the neck part of the YoLov8 network; Construct classification branch and positioning branch based on the detection head of YoLov8 network; Construct semantic branches based on convolutional layers and activation function layers; The backbone network, the path aggregation feature pyramid network, the classification branch, the positioning branch and the semantic branch are combined to obtain the initial network.

3. The orthopedic disease auxiliary detection method according to claim 2, characterized in that: The constructing of semantic branches according to the convolutional layer and the activation function layer includes: Constructing a preset number of sub-branches according to the convolution layer and the activation function layer; Connect each sub-branch end to end in sequence to obtain the semantic branch; Among them, each sub-branch includes a convolution layer, a normalization function layer and an activation function layer connected end to end.

4. The orthopedic disease auxiliary detection method according to claim 1, wherein: Generating a text description of each X-ray image according to the actual lesion category of each X-ray image and the medical prompt generation module includes: Get pre-configured medical prompt words; Taking the real lesion category of each X-ray image as input, and using the medical prompt words to guide the medical prompt generation module to output the lesion pathological characteristics of each real lesion category and its typical manifestation in the corresponding X-ray image; The lesion pathological features of each real lesion category and its typical manifestations in the corresponding X-ray image are determined as a text description of the corresponding X-ray image.

5. The orthopedic disease auxiliary detection method according to claim 2, characterized in that: The semantic learning module includes a contrastive language-image pre-training model, an image encoder and a text encoder of the contrastive language-image pre-training model; The step of training the initial network based on the X-ray images, the text description of each X-ray image, and the semantic learning module to obtain an orthopedic lesion detection network includes: During the training process, freezing the contrastive language-image pre-trained model, inputting the X-ray image into the initial network, and obtaining the output data corresponding to each X-ray image in the semantic branch as the semantic feature of each X-ray image; Inputting each X-ray image and the corresponding ground-truth lesion position bounding box into the image encoder to obtain a region of interest feature of each X-ray image, and inputting a text description of each X-ray image into the text encoder to obtain a text feature of each X-ray image; Utilizing the semantic learning module, calculating a first similarity between a feature of a region of interest of each X-ray image and a corresponding text feature, and calculating a second similarity between a semantic feature of each X-ray image and a corresponding text feature; Constructing a total loss function; wherein the total loss function includes a semantic consistency loss function and a detection loss function; Using the first similarity as a supervisory signal, and using a semantic consistency loss function to constrain the second similarity, so as to guide the semantic branch to learn the fine-grained semantic information contained in the text description; Taking the true lesion category and the true lesion location bounding box marked in each X-ray image as training targets, optimizing the classification branch and the localization branch using the detection loss function to guide the classification branch to learn lesion category information and the localization branch to learn lesion location information; When the total loss function reaches convergence, the training is stopped, and the currently obtained network is determined as the orthopedic lesion detection network.

6. The orthopedic disease auxiliary detection method according to claim 5, characterized in that: The constructing of the total loss function includes: Constructing the semantic consistency loss function based on the Smooth L1 loss function; Constructing the detection loss function based on the loss function of the YoLov8 network; The sum of the semantic consistency loss function and the detection loss function is calculated to obtain the total loss function.

7. The orthopedic disease auxiliary detection method according to claim 2, characterized in that: Inputting the target X-ray image into the orthopedic lesion detection network for processing to obtain a detection result of the target X-ray image includes: After removing the semantic branch from the orthopedic lesion detection network, using the backbone network to obtain multi-scale X-ray image features of the target X-ray image; fusing the multi-scale X-ray image features using the path aggregation feature pyramid network to obtain enhanced multi-scale X-ray image features; Inputting the enhanced multi-scale X-ray image features into the classification branch and the positioning branch respectively; Obtaining the output of the classification branch as an orthopedic disease type prediction result, and obtaining the output of the positioning branch as an orthopedic lesion location prediction result; The orthopedic disease type prediction result and the orthopedic lesion location prediction result are integrated to obtain the detection result.

8. An auxiliary detection device for orthopedic diseases, characterized in that: The orthopedic disease auxiliary detection device comprises: Construction unit, used to construct the initial network, medical prompt generation module and matching-guided semantic learning module; An acquisition unit, configured to acquire X-ray images; wherein each X-ray image is marked with a true lesion category and a true lesion location bounding box of an orthopedic disease; a generating unit, configured to generate a text description of each X-ray image according to the actual lesion category of each X-ray image and the medical prompt generating module; a training unit, configured to train the initial network based on the X-ray images, a text description of each X-ray image, and the semantic learning module to obtain an orthopedic lesion detection network; A processing unit is used to respond to an orthopedic disease auxiliary detection instruction for a target X-ray image, input the target X-ray image into the orthopedic lesion detection network for processing, obtain a detection result of the target X-ray image, and send the detection result to the trigger of the orthopedic disease auxiliary detection instruction.

9. A computer device, characterized in that: The computer device comprises: a memory storing at least one instruction; and A processor executes instructions stored in the memory to implement the orthopedic disease auxiliary detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, and the at least one instruction is executed by a processor in a computer device to implement the orthopedic disease auxiliary detection method according to any one of claims 1 to 7.

Images