Orthopedic disease auxiliary detection method, device, equipment and medium

Abstract

The present application relates to the technical field of artificial intelligence, and provides an orthopedic disease auxiliary detection method, device, equipment and medium, on the one hand, the present application can generate the text description of each X-ray image according to the real lesion category of each X-ray image and the medical prompt generation module, and provide important information related to the lesion for subsequent model training, on the other hand, based on the X-ray image, the text description of each X-ray image and the semantic learning module training based on the matching degree guidance, the orthopedic lesion detection network can be obtained, under the guidance of the medical prompt, the text-image alignment ability of the semantic learning module based on the matching degree guidance is used as the bridge from the text mode to the image mode, so as to assist the orthopedic lesion detection network to learn the semantic information of the lesion, thereby improving the detection performance of the orthopedic disease, assisting medical staff to quickly read the film, and indirectly improving the accuracy of the orthopedic disease detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a method and device for assisting detection of orthopedic diseases, equipment and a medium. BACKGROUND

[0002] With the popularity of electronic devices such as mobile phones and computers, people's work and life style has changed. Long-term use of electronic devices in an improper posture has led to an increase in the incidence of spinal diseases and a younger trend. In addition, cases of bone fractures caused by daily activities, sports injuries, traffic accidents, etc. are also increasing, and bone fractures not only seriously affect the quality of life of patients, but also may lead to malunion, dysfunction and even disability if not diagnosed in time and accurately. Therefore, early screening and accurate diagnosis and treatment of orthopedic diseases are crucial for slowing down the progression of the disease and improving the quality of life of patients.

[0003] In recent years, the method of assisting diagnosis of orthopedic diseases based on X-ray images using deep learning has made certain research progress. However, the existing method can only mine lesion information from images, and has not explored the paradigm of using textual descriptions of lesions to improve the accuracy of disease diagnosis. SUMMARY

[0004] In view of the above, it is necessary to provide a method and device for assisting detection of orthopedic diseases, equipment and a medium, which aims to solve the problem of low accuracy of existing methods for assisting detection of orthopedic diseases.

[0005] A method for assisting detection of orthopedic diseases, the method comprising:

[0006] constructing an initial network, a medical prompt generation module and a semantic learning module based on matching degree guidance;

[0007] collecting X-ray images; wherein each X-ray image is labeled with a true lesion category and a true lesion position bounding box of an orthopedic disease;

[0008] generating a textual description of each X-ray image according to the true lesion category of each X-ray image and the medical prompt generation module;

[0009] training the initial network based on the X-ray images, the textual description of each X-ray image and the semantic learning module to obtain an orthopedic lesion detection network;

[0010] in response to an instruction for assisting detection of orthopedic diseases of a target X-ray image, inputting the target X-ray image to the orthopedic lesion detection network for processing to obtain a detection result of the target X-ray image, and sending the detection result to the trigger of the instruction for assisting detection of orthopedic diseases.

[0011] According to the preferred embodiment of the present application, the constructing the initial network comprises:

[0012] constructing a backbone network according to a backbone part of a YoLov8 network;

[0013] constructing a path aggregation feature pyramid network according to a neck part of the YoLov8 network;

[0014] constructing a classification branch and a positioning branch according to a detection head of the YoLov8 network;

[0015] constructing a semantic branch according to a convolutional layer and an activation function layer;

[0016] combining the backbone network, the path aggregation feature pyramid network, the classification branch, the positioning branch and the semantic branch to obtain the initial network.

[0017] According to the preferred embodiment of the present application, the constructing the semantic branch according to the convolutional layer and the activation function layer comprises:

[0018] constructing a preset number of sub-branches according to the convolutional layer and the activation function layer;

[0019] sequentially connecting each sub-branch end to end to obtain the semantic branch;

[0020] wherein each sub-branch comprises 1 convolutional layer, 1 normalization function layer and 1 activation function layer connected end to end.

[0021] According to the preferred embodiment of the present application, the generating the text description of each X-ray image according to the real lesion category of each X-ray image and the medical prompt generation module comprises:

[0022] obtaining a pre-configured medical prompt word;

[0023] taking the real lesion category of each X-ray image as input, and using the medical prompt word to guide the medical prompt generation module to output the lesion pathological feature of each real lesion category and the typical performance in the corresponding X-ray image;

[0024] determining the lesion pathological feature of each real lesion category and the typical performance in the corresponding X-ray image as the text description of the corresponding X-ray image.

[0025] According to the preferred embodiment of the present application, the semantic learning module comprises a contrastive language-image pre-training model, an image encoder and a text encoder of the contrastive language-image pre-training model;

[0026] the training the initial network based on the X-ray image, the text description of each X-ray image and the semantic learning module to obtain an orthopedic lesion detection network comprises:

[0027] In the training process, the contrastive language-image pre-training model is frozen, the X-ray image is input into the initial network, and the output data corresponding to each X-ray image in the semantic branch is obtained as the semantic feature of each X-ray image;

[0028] Each X-ray image and the corresponding real lesion position bounding box are input into the image encoder to obtain the region of interest feature of each X-ray image, and the text description of each X-ray image is input into the text encoder to obtain the text feature of each X-ray image;

[0029] The semantic learning module is used to calculate the first similarity between the region of interest feature of each X-ray image and the corresponding text feature, and the second similarity between the semantic feature of each X-ray image and the corresponding text feature;

[0030] A total loss function is constructed; wherein the total loss function includes a semantic consistency loss function and a detection loss function;

[0031] The first similarity is used as a supervision signal, the semantic consistency loss function is used to constrain the second similarity, so as to guide the semantic branch to learn the fine-grained semantic information contained in the text description;

[0032] The real lesion class and real lesion position bounding box labeled for each X-ray image are used as a training target, the detection loss function is used to optimize the classification branch and the positioning branch, so as to guide the classification branch to learn the lesion class information and the positioning branch to learn the lesion position information;

[0033] When the total loss function converges, the training is stopped, and the current obtained network is determined as the orthopedic lesion detection network.

[0034] According to the preferred embodiment of the present application, the total loss function is constructed, including:

[0035] The semantic consistency loss function is constructed based on a Smooth L1 loss function;

[0036] The detection loss function is constructed based on the loss function of a YoLov8 network;

[0037] The sum of the semantic consistency loss function and the detection loss function is calculated to obtain the total loss function.

[0038] According to the preferred embodiment of the present application, the target X-ray image is input into the orthopedic lesion detection network for processing to obtain the detection result of the target X-ray image, including:

[0039] After removing the semantic branch from the orthopedic lesion detection network, a plurality of multi-scale X-ray image features of the target X-ray image are obtained by using the backbone network;

[0040] The plurality of multi-scale X-ray image features are fused by using the path aggregation feature pyramid network to obtain enhanced multi-scale X-ray image features;

[0041] The enhanced multi-scale X-ray image features are respectively input into the classification branch and the positioning branch;

[0042] The output of the classification branch is obtained as an orthopedic disease type prediction result, and the output of the positioning branch is obtained as an orthopedic lesion position prediction result;

[0043] The orthopedic disease type prediction result and the orthopedic lesion position prediction result are integrated to obtain the detection result.

[0044] An orthopedic disease auxiliary detection device, the orthopedic disease auxiliary detection device comprises:

[0045] A construction unit is configured to construct an initial network, a medical prompt generation module, and a semantic learning module based on matching degree guidance;

[0046] An acquisition unit is configured to acquire X-ray images; each X-ray image is marked with a real lesion category and a real lesion position bounding box of an orthopedic disease;

[0047] A generation unit is configured to generate a text description of each X-ray image according to the real lesion category of each X-ray image and the medical prompt generation module;

[0048] A training unit is configured to train the initial network based on the X-ray images, the text description of each X-ray image, and the semantic learning module to obtain an orthopedic lesion detection network;

[0049] A processing unit is configured to, in response to an orthopedic disease auxiliary detection instruction for a target X-ray image, input the target X-ray image into the orthopedic lesion detection network for processing to obtain a detection result of the target X-ray image, and send the detection result to a trigger of the orthopedic disease auxiliary detection instruction.

[0050] A computer device, the computer device comprises:

[0051] A memory stores at least one instruction; and

[0052] A processor executes the instructions stored in the memory to implement the orthopedic disease auxiliary detection method.

[0053] A computer readable storage medium, the computer readable storage medium has at least one instruction stored therein, the at least one instruction is executed by a processor in a computer device to implement the auxiliary detection method for orthopedic diseases.

[0054] From the above technical solutions, on the one hand, the present application can generate a text description of each X-ray image according to the real lesion category of each X-ray image and the medical prompt generation module, providing important information related to the lesion for subsequent model training; on the other hand, based on the X-ray image, the text description of each X-ray image, and the semantic learning module guided by matching degree, the orthopedic lesion detection network is trained, which can guide the medical prompt, use the text-image alignment capability of the semantic learning module guided by matching degree as a bridge from the text mode to the image mode, to assist the orthopedic lesion detection network to learn the semantic information of the lesion, thereby improving the detection performance of orthopedic diseases, assisting medical staff in rapid reading, and indirectly improving the accuracy of orthopedic disease detection. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 is a flowchart of a preferred embodiment of the auxiliary detection method for orthopedic diseases of the present application.

[0056] Figure 2 is a functional module diagram of a preferred embodiment of the auxiliary detection device for orthopedic diseases of the present application.

[0057] Figure 3 is a structural schematic diagram of a computer device for implementing a preferred embodiment of the auxiliary detection method for orthopedic diseases of the present application. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be described in detail below in combination with the drawings and specific embodiments.

[0059] As Figure 1 shown, is a flowchart of a preferred embodiment of the auxiliary detection method for orthopedic diseases of the present application. The order of steps in the flowchart can be changed according to different needs, and some steps can be omitted.

[0060] The auxiliary detection method for orthopedic diseases is applied to one or more computer devices, which is a device capable of automatically performing numerical calculation and / or information processing according to pre-set or stored instructions. The hardware thereof includes but is not limited to microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0061] The computer device can be any electronic product that can interact with the user, such as a personal computer, a tablet computer, a smartphone, a personal digital assistant (PDA), a game console, an interactive Internet Protocol Television (IPTV), a smart wearable device, etc.

[0062] The computer device can also include network devices and / or user devices. The network devices include, but are not limited to, single network servers, server groups composed of multiple network servers, or clouds composed of a large number of hosts or network servers based on cloud computing.

[0063] The server can be a standalone server, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, etc. basic cloud computing services.

[0064] Artificial intelligence (AI) is the use of digital computers or computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0065] The basic technology of artificial intelligence generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. The software technology of artificial intelligence mainly includes computer vision technology, robot technology, biometric technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc. several major directions.

[0066] The network in which the computer device is located includes, but is not limited to, the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), and the like.

[0067] S10, constructing an initial network, a medical prompt generation module and a semantic learning module based on matching degree guidance.

[0068] In the embodiment, the constructing the initial network comprises:

[0069] constructing a backbone network according to a backbone part of a YoLov8 (You Only Look Once Version 8) network;

[0070] constructing a path aggregation feature pyramid network according to a neck part of the YoLov8 network;

[0071] constructing a classification branch and a positioning branch according to a detection head of the YoLov8 network;

[0072] constructing a semantic branch according to a convolutional layer and an activation function layer;

[0073] combining the backbone network, the path aggregation feature pyramid network, the classification branch, the positioning branch and the semantic branch to obtain the initial network.

[0074] Specifically, the constructing the semantic branch according to the convolutional layer and the activation function layer comprises:

[0075] constructing a preset number of sub-branches according to the convolutional layer and the activation function layer;

[0076] sequentially connecting each sub-branch end to end to obtain the semantic branch;

[0077] Each sub-branch comprises one convolutional layer, one normalization function layer and one activation function layer connected end to end.

[0078] For example, the preset number can be configured as 3. At this time, the semantic branch can be represented in the following form:

[0079] Input→first sub-branch (convolutional layer→normalization function layer→activation function layer)→second sub-branch (convolutional layer→normalization function layer→activation function layer)→third sub-branch (convolutional layer→normalization function layer→activation function layer)→output.

[0080] The semantic branch is a self-developed network branch, and through the semantic branch, the subsequent model training can assist the orthopedic lesion detection network to learn semantic information of the lesion.

[0081] In this embodiment, the medical prompt generation module is a large language model supported by professional orthopedic medical knowledge. Through the medical prompt generation module, text descriptions containing pathological characteristics of lesions and typical manifestations in X-ray images can be obtained.

[0082] In this embodiment, the semantic learning module based on matching degree guidance utilizes the text-image matching capability of CLIP (Contrastive Language-Image Pre-training) to enable the orthopedic lesion detection network to better learn fine-grained lesion semantics.

[0083] S11, collect X-ray images; wherein each X-ray image is marked with a real lesion category and a real lesion position bounding box of an orthopedic disease.

[0084] In this embodiment, the X-ray images can be orthopedic disease X-ray images that have accurate diagnosis results.

[0085] In this embodiment, the marked real lesion category and real lesion position bounding box can provide data support for subsequent model training.

[0086] S12, generate a text description of each X-ray image according to the real lesion category of each X-ray image and the medical prompt generation module.

[0087] In this embodiment, generating a text description of each X-ray image according to the real lesion category of each X-ray image and the medical prompt generation module includes:

[0088] Obtaining preconfigured medical prompt words;

[0089] Taking the real lesion category of each X-ray image as input, and using the medical prompt words to guide the medical prompt generation module to output the lesion pathological characteristics of each real lesion category and the typical manifestations in the corresponding X-ray image;

[0090] Determining the lesion pathological characteristics of each real lesion category and the typical manifestations in the corresponding X-ray image as the text description of the corresponding X-ray image.

[0091] For example, the medical prompt words can be: "In an X-ray image, what are the color, shape, and pathological properties of [cls]?".

[0092] The medical prompt generation module can be a large language model.

[0093] Through reasonable configuration of the medical prompt words, the model can output more accurate and comprehensive text descriptions of the X-ray images.

[0094] Considering the complexity and diversity of orthopedic lesions, and the strong learning ability and knowledge reserve of the language base model, therefore, the professional text descriptions corresponding to each orthopedic lesion category can be obtained by using the embodiment, such as that the bone hyperplasia is a bony protrusion formed along the edge of the vertebral body, and the intervertebral space narrowing refers to the reduction of the space between adjacent vertebral bodies. These information is helpful for the model to understand the lesions more comprehensively.

[0095] Therefore, the embodiment uses the language base model to analyze the lesions professionally and output the text descriptions of the lesions, so as to improve the accuracy of orthopedic disease detection, and has important clinical application value.

[0096] S13, training the initial network based on the X-ray images, the text descriptions of each X-ray image, and the semantic learning module to obtain an orthopedic lesion detection network.

[0097] In the embodiment, the semantic learning module includes a contrastive language-image pre-training model, an image encoder, and a text encoder of the contrastive language-image pre-training model.

[0098] In the training stage, the image encoder and the text encoder of the contrastive language-image pre-training model are kept frozen, and the initial network is kept in a trainable state.

[0099] Specifically, the training of the initial network based on the X-ray images, the text descriptions of each X-ray image, and the semantic learning module to obtain an orthopedic lesion detection network includes:

[0100] In the training process, the contrastive language-image pre-training model is frozen, the X-ray images are input into the initial network, and the output data corresponding to each X-ray image in the semantic branch is obtained as the semantic features of each X-ray image;

[0101] Each X-ray image and the corresponding real lesion position bounding box are input into the image encoder to obtain the region of interest features of each X-ray image, and the text description of each X-ray image is input into the text encoder to obtain the text features of each X-ray image;

[0102] The first similarity between the region of interest features and the corresponding text features of each X-ray image is calculated by using the semantic learning module, and the second similarity between the semantic features and the corresponding text features of each X-ray image is calculated;

[0103] A total loss function is constructed; wherein the total loss function includes a semantic consistency loss function and a detection loss function;

[0104] use the semantic consistency loss function to constrain the second similarity degree, so as to guide the semantic branch to learn the fine-grained semantic information contained in the text description;

[0105] use the detection loss function to optimize the classification branch and the positioning branch, so as to guide the classification branch to learn the lesion class information and the positioning branch to learn the lesion position information, by taking the real lesion class and the real lesion position bounding box marked in each X-ray image as training targets;

[0106] When the total loss function reaches convergence, stop training, and determine the current obtained network as the orthopedic lesion detection network.

[0107] The constructing total loss function comprises:

[0108] The semantic consistency loss function is constructed based on a Smooth L1 loss function;

[0109] The detection loss function is constructed based on a loss function of a YoLov8 network;

[0110] The sum of the semantic consistency loss function and the detection loss function is calculated to obtain the total loss function.

[0111] The detection loss function is used to optimize the learning process of the classification branch and the positioning branch, and the semantic consistency loss function is used to optimize the learning process of the semantic branch.

[0112] In the above embodiment, the similarity degree between the region of interest features and the text features is used as a supervision signal, and the semantic consistency loss function is used to constrain the similarity degree between the semantic features output by the semantic branch of the lesion detection network and the text features, so that the semantic branch of the orthopedic lesion detection network can learn the fine-grained semantic information contained in the text description of the lesion, and the lesion detection performance is further improved.

[0113] S14, in response to an orthopedic disease auxiliary detection instruction of a target X-ray image, inputting the target X-ray image into the orthopedic lesion detection network for processing to obtain a detection result of the target X-ray image, and sending the detection result to a trigger of the orthopedic disease auxiliary detection instruction.

[0114] In the embodiment, the orthopedic disease auxiliary detection instruction can be triggered by relevant staff according to actual detection needs; the orthopedic disease auxiliary detection instruction can also be automatically triggered when the target X-ray image is uploaded to a designated platform, so as to reduce the waiting time.

[0115] In the embodiment, the inputting the target X-ray image into the orthopedic lesion detection network for processing to obtain a detection result of the target X-ray image comprises:

[0116] After removing the semantic branch from the orthopedic lesion detection network, the backbone network is used to obtain multi-scale X-ray image features of the target X-ray image;

[0117] The multi-scale X-ray image features are fused by using the path aggregation feature pyramid network to obtain enhanced multi-scale X-ray image features;

[0118] The enhanced multi-scale X-ray image features are input into the classification branch and the positioning branch, respectively;

[0119] The output of the classification branch is obtained as an orthopedic disease type prediction result, and the output of the positioning branch is obtained as an orthopedic lesion position prediction result;

[0120] The orthopedic disease type prediction result and the orthopedic lesion position prediction result are integrated to obtain the detection result.

[0121] In the detection stage, the medical prompt generation module, the semantic learning module based on matching degree guidance, and the semantic branch of the orthopedic lesion detection network are removed, and only the backbone network, the path aggregation feature pyramid network, the classification branch, and the regression branch in the orthopedic lesion detection network are used to complete the auxiliary detection of orthopedic diseases, and the type and position of the orthopedic diseases are predicted.

[0122] Through the above embodiments, the detailed and professional orthopedic disease description information provided by the large language model and the text-image alignment ability of the contrast language-image pre-training model can be combined to enable the model to better understand the lesion semantic information, thereby improving the detection performance of orthopedic diseases.

[0123] In the embodiment, the triggerer can be a relevant medical staff.

[0124] The detection result is sent to the triggerer of the orthopedic disease auxiliary detection instruction in a timely manner, which can assist the relevant medical staff in quickly reading the film, and the improvement of the model performance also indirectly improves the accuracy of the orthopedic disease detection.

[0125] From the above technical scheme can be seen, on the one hand, the present application can generate the text description of each X-ray image according to the real lesion category of each X-ray image and the medical prompt generation module, provide important information related to the lesion for subsequent model training;On the other hand, based on the X-ray image, the text description of each X-ray image and the semantic learning module guided by the matching degree, the orthopedic lesion detection network is trained, which can guide the medical prompt, use the text-image alignment ability of the semantic learning module guided by the matching degree as the bridge from the text mode to the image mode, to assist the orthopedic lesion detection network to learn the semantic information of the lesion, so as to improve the detection performance of orthopedic diseases, assist medical staff to read film quickly, and indirectly improve the accuracy of orthopedic disease detection.

[0126] As Figure 2 shown, it is the function module diagram of the preferred embodiment of the orthopedic disease auxiliary detection device of the application. The orthopedic disease auxiliary detection device 11 includes a construction unit 110, an acquisition unit 111, a generation unit 112, a training unit 113, and a processing unit 114. The module / unit referred to in the application refers to a series of computer program segments that can be executed by the processor and can complete fixed functions, which are stored in the memory. In this embodiment, the functions of each module / unit will be described in detail in the subsequent embodiments.

[0127] The construction unit 110 is configured to construct an initial network, a medical prompt generation module and a semantic learning module guided by matching degree.

[0128] The acquisition unit 111 is configured to acquire X-ray images, wherein each X-ray image is marked with the real lesion category and the real lesion position bounding box of orthopedic diseases.

[0129] The generation unit 112 is configured to generate the text description of each X-ray image according to the real lesion category of each X-ray image and the medical prompt generation module.

[0130] The training unit 113 is configured to train the initial network based on the X-ray image, the text description of each X-ray image and the semantic learning module, to obtain an orthopedic lesion detection network.

[0131] The processing unit 114 is configured to respond to the orthopedic disease auxiliary detection instruction of the target X-ray image, input the target X-ray image into the orthopedic lesion detection network for processing, obtain the detection result of the target X-ray image, and send the detection result to the trigger of the orthopedic disease auxiliary detection instruction.

[0132] From the above technical scheme can be seen, on the one hand, the present application can generate the text description of each X-ray image according to the real lesion category of each X-ray image and the medical prompt generation module, and provide important information related to the lesion for subsequent model training;On the other hand, based on the X-ray image, the text description of each X-ray image and the semantic learning module guided by matching degree, the orthopedic lesion detection network is trained, which can guide under the medical prompt, use the text-image alignment ability of the semantic learning module guided by matching degree as the bridge from text mode to image mode, to assist the orthopedic lesion detection network to learn the semantic information of the lesion, so as to improve the detection performance of orthopedic diseases, assist medical staff to read film quickly, and indirectly improve the accuracy of orthopedic disease detection.

[0133] As Figure 3 shown, it is a structural schematic diagram of a computer device of a preferred embodiment of the present application for realizing the auxiliary detection method of orthopedic diseases.

[0134] The computer device 1 can include a memory 12, a processor 13 and a bus (the arrow in the figure is the bus), and can also include a computer program stored in the memory 12 and executable on the processor 13, such as an auxiliary detection program of orthopedic diseases.

[0135] Those skilled in the art can understand that the schematic diagram is only an example of the computer device 1, and does not constitute a limitation on the computer device 1, the computer device 1 can be a bus type structure, or a star type structure, the computer device 1 can also include more or less other hardware or software than the diagram, or different component arrangement, for example, the computer device 1 can also include an input and output device, a network access device, etc.

[0136] It should be noted that the computer device 1 is only an example, and other existing or future electronic products, such as the present application, can also be adapted to the present application, and should be included in the protection scope of the present application, and is included herein by reference.

[0137] The memory 12 includes at least one type of readable storage medium, such as a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 12 can be an internal storage unit of the computer device 1, such as a mobile hard disk of the computer device 1. In other embodiments, the memory 12 can also be an external storage device of the computer device 1, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 12 can include both an internal storage unit and an external storage device of the computer device 1. The memory 12 can be used to store application software and various data installed in the computer device 1, such as the code of the orthopedic disease auxiliary detection program, and can also be used to temporarily store data that has been output or will be output.

[0138] The processor 13 can be composed of an integrated circuit in some embodiments, such as a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more combinations of a central processing unit (CPU), a microprocessor, a digital processing chip, a graphics processor, and various control chips, etc. The processor 13 is a control unit of the computer device 1, which connects various components of the computer device 1 through various interfaces and lines, executes programs or modules stored in the memory 12 (such as the orthopedic disease auxiliary detection program, etc.), and calls data stored in the memory 12, to perform various functions and process data of the computer device 1.

[0139] The processor 13 executes an operating system and various application programs installed in the computer device 1. The processor 13 executes the application programs to implement the steps in each of the above orthopedic disease auxiliary detection method embodiments, such as Figure 1 the steps shown in the above embodiments.

[0140] The computer program can be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to complete the present application. The one or more modules / units can be a series of computer-readable instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the computer device 1. For example, the computer program can be divided into a construction unit 110, a collection unit 111, a generation unit 112, a training unit 113, and a processing unit 114.

[0141] The integrated units in the form of software function modules described above can be stored in a computer-readable storage medium. The software function modules described above are stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a computer device, or a network device, etc.) or a processor to execute part of the orthopedic disease auxiliary detection method described in various embodiments of the present application.

[0142] The modules / units integrated in the computer device 1, if implemented in the form of software function modules and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the present application can also be implemented by a computer program to instruct related hardware devices to complete all or part of the processes in the above-mentioned embodiments. The computer program can be stored in a computer-readable storage medium and can implement the steps of the above-mentioned various method embodiments when executed by a processor.

[0143] The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate forms, etc. The computer-readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory, etc.

[0144] Further, the computer-readable storage medium can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required by a function, etc.; and the data storage area can store data created according to the use of the blockchain node, etc.

[0145] The blockchain referred to in the present application is a new application mode of distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm and other computer technologies. Blockchain, in essence, is a decentralized database, which is a series of data blocks associated using cryptographic methods, each data block contains a batch of network transaction information, used to verify the validity of the information (anti-fake) and generate the next block. The blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer, etc.

[0146] The bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one straight line is used in Figure 3 , but it does not mean that there is only one bus or one type of bus. The bus is arranged to realize the connection and communication between the memory 12, the at least one processor 13, etc.

[0147] Although not shown, the computer device 1 can also include a power supply (such as a battery) for powering the various components. Preferably, the power supply can be logically connected to the at least one processor 13 through a power management device, so that the power management device can realize functions such as charge management, discharge management, and power consumption management. The power supply can also include one or more direct current or alternating current power supplies, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, etc. The computer device 1 can also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described here.

[0148] Further, the computer device 1 can also include a network interface, which can optionally include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is usually used to establish a communication connection between the computer device 1 and other computer devices.

[0149] Optionally, the computer device 1 can further comprise a user interface, which can be a display, an input unit such as a keyboard, and optionally a standard wired interface, a wireless interface. Optionally, in some embodiments, the display can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, etc. The display can also be appropriately referred to as a display screen or a display unit, for displaying information processed in the computer device 1 and for displaying a visualized user interface.

[0150] It should be understood that the embodiments are only for illustration and are not limited in the scope of the patent application by the structure.

[0151] Those skilled in the art can understand that, Figure 3 The structure shown does not constitute a limitation on the computer device 1, which can include fewer or more components than shown, or combine certain components, or different component arrangements.

[0152] In combination Figure 1 The memory 12 in the computer device 1 stores a plurality of instructions to implement an auxiliary detection method for orthopedic diseases, and the processor 13 can execute the plurality of instructions to implement:

[0153] constructing an initial network, a medical prompt generation module, and a semantic learning module based on matching degree guidance;

[0154] collecting X-ray images; wherein each X-ray image is marked with a real lesion category and a real lesion position bounding box of orthopedic diseases;

[0155] generating a text description of each X-ray image according to the real lesion category of each X-ray image and the medical prompt generation module;

[0156] training the initial network based on the X-ray images, the text description of each X-ray image, and the semantic learning module to obtain an orthopedic lesion detection network;

[0157] in response to an auxiliary detection instruction for orthopedic diseases of a target X-ray image, inputting the target X-ray image to the orthopedic lesion detection network for processing to obtain a detection result of the target X-ray image, and sending the detection result to a trigger of the auxiliary detection instruction for orthopedic diseases.

[0158] Specifically, the processor 13 can refer to the description of the specific implementation method of the above instructions Figure 1 for the corresponding embodiments, which will not be described here.

[0159] It should be noted that the data involved in the case are all legally obtained. The non-company software tools or components appearing in the embodiments of the application are only used for example introduction, and do not represent actual use.

[0160] In several embodiments provided by the present application, it should be understood that the disclosed system, device and method can be implemented by other manners. For example, the device embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and another division manner can be used in actual implementation.

[0161] The application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The application can also be practiced in a distributed computing environment in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0162] The modules described as separate components can or can not be physically separated, and the components shown as modules can or can not be physical units, i.e. they can be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment.

[0163] In addition, the functional modules in each embodiment of the application can be integrated in one processing unit, or each unit can exist physically independently, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of hardware plus software functional modules.

[0164] It is obvious for those skilled in the art that the application is not limited to the details of the above exemplary embodiments, and the application can be implemented in other specific forms without departing from the spirit or essential characteristics of the application.

[0165] Thus, the embodiments can be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are intended to be embraced therein. No reference signs in the claims shall be construed as limiting the scope of the claims.

[0166] Furthermore, the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural and vice-versa, unless the context clearly requires these exclusions. The composition of the application can be either hardware-only or software-only or a combination of hardware and software.

[0167] Finally, it should be noted that the above-mentioned embodiments illustrate rather than limit the application, since the scope of the application is determined by the appended claims.

Claims

1. An auxiliary detection method for orthopedic diseases, characterized in that, The auxiliary detection method for the orthopedic diseases comprises: An initial network, a medical prompt generation module, and a semantic learning module based on matching degree guidance are constructed; the semantic learning module comprises a comparative language-image pre-training model, which comprises an image encoder and a text encoder; X-ray images are collected; each X-ray image is marked with a real lesion category and a real lesion position bounding box of an orthopedic disease; A text description of each X-ray image is generated according to the real lesion category of each X-ray image and the medical prompt generation module; The initial network is trained based on the X-ray images, the text description of each X-ray image, and the semantic learning module to obtain an orthopedic lesion detection network, comprising: in the training process, the comparative language-image pre-training model is frozen, the X-ray images are input into the initial network, and the semantic features of each X-ray image in the semantic branch of the initial network are obtained as the output data of each X-ray image; each X-ray image and the corresponding real lesion position bounding box are input into the image encoder to obtain the region of interest features of each X-ray image, and the text description of each X-ray image is input into the text encoder to obtain the text features of each X-ray image; the first similarity between the region of interest features of each X-ray image and the corresponding text features is calculated using the semantic learning module, and the second similarity between the semantic features of each X-ray image and the corresponding text features is calculated; a total loss function is constructed; the total loss function comprises a semantic consistency loss function and a detection loss function; the first similarity is used as a supervision signal, the second similarity is constrained by the semantic consistency loss function to guide the semantic branch to learn the fine-grained semantic information contained in the text description; the real lesion category and the real lesion position bounding box marked for each X-ray image are used as a training target, and the classification branch and the positioning branch of the initial network are optimized by the detection loss function to guide the classification branch to learn the lesion category information and the positioning branch to learn the lesion position information; when the total loss function converges, the training is stopped, and the current obtained network is determined as the orthopedic lesion detection network; In response to an auxiliary detection instruction for an orthopedic disease of a target X-ray image, the target X-ray image is input into the orthopedic lesion detection network for processing to obtain a detection result of the target X-ray image, and the detection result is sent to the trigger of the auxiliary detection instruction for the orthopedic disease.

2. The method of claim 1, wherein the method is for detecting an orthopedic disease. The initial network is constructed by: A backbone network is constructed according to the backbone part of a YoLov8 network; A path aggregation feature pyramid network is constructed according to the neck part of the YoLov8 network; The classification branch and the positioning branch are constructed according to the detection head of the YoLov8 network; The semantic branch is constructed according to a convolution layer and an activation function layer; The initial network is obtained by combining the backbone network, the path aggregation feature pyramid network, the classification branch, the positioning branch, and the semantic branch.

3. The auxiliary detection method for orthopedic diseases as described in claim 2, characterized in that, The constructing the semantic branch according to the convolutional layer and the activation function layer comprises: constructing a preset number of sub-branches according to the convolutional layer and the activation function layer; sequentially connecting each sub-branch end to end to obtain the semantic branch; wherein each sub-branch comprises 1 convolutional layer, 1 normalization function layer and 1 activation function layer connected end to end.

4. The method of claim 1, wherein the method is used for detecting orthopedic diseases. The generating the text description of each X-ray image according to the real lesion category of each X-ray image and the medical prompt generation module comprises: obtaining preconfigured medical prompt words; taking the real lesion category of each X-ray image as input, and guiding the medical prompt generation module to output the lesion pathological characteristics of each real lesion category and the typical performance in the corresponding X-ray image by using the medical prompt words; determining the lesion pathological characteristics of each real lesion category and the typical performance in the corresponding X-ray image as the text description of the corresponding X-ray image.

5. The method of claim 1, wherein the method is used for detecting orthopedic diseases. The constructing the total loss function comprises: constructing the semantic consistency loss function based on a Smooth L1 loss function; constructing the detection loss function based on a loss function of a YoLov8 network; calculating the sum of the semantic consistency loss function and the detection loss function to obtain the total loss function.

6. The method of claim 2, wherein the method is used for detecting orthopedic diseases. The inputting the target X-ray image into the orthopedic lesion detection network for processing to obtain the detection result of the target X-ray image comprises: after removing the semantic branch from the orthopedic lesion detection network, obtaining multi-scale X-ray image features of the target X-ray image by using the backbone network; fusing the multi-scale X-ray image features by using the path aggregation feature pyramid network to obtain enhanced multi-scale X-ray image features; inputting the enhanced multi-scale X-ray image features into the classification branch and the positioning branch respectively; obtaining the output of the classification branch as an orthopedic disease type prediction result, and obtaining the output of the positioning branch as an orthopedic lesion location prediction result; integrating the orthopedic disease type prediction result and the orthopedic lesion location prediction result to obtain the detection result.

7. An auxiliary detection device for orthopedic diseases, characterized in that, The orthopedic disease auxiliary detection device comprises: a construction unit configured to construct an initial network, a medical prompt generation module and a semantic learning module guided based on a matching degree; wherein the semantic learning module comprises a comparative language-image pre-training model, and the comparative language-image pre-training model comprises an image encoder and a text encoder; an acquisition unit configured to acquire X-ray images; wherein each X-ray image is marked with a real lesion category and a real lesion location bounding box of an orthopedic disease; a generation unit configured to generate a text description of each X-ray image according to the real lesion category of each X-ray image and the medical prompt generation module; The training unit is configured to train the initial network based on the X-ray images, the text descriptions of each X-ray image, and the semantic learning module to obtain an orthopedic lesion detection network. In the training process, the contrast language-image pre-training model is frozen, the X-ray images are input into the initial network, and the output data corresponding to each X-ray image in the semantic branch of the initial network is obtained as the semantic features of each X-ray image. Each X-ray image and the real lesion position bounding box are input into the image encoder to obtain the region of interest features of each X-ray image, and the text description of each X-ray image is input into the text encoder to obtain the text features of each X-ray image. The semantic learning module is used to calculate the first similarity between the region of interest features and the corresponding text features of each X-ray image, and the second similarity between the semantic features and the corresponding text features of each X-ray image. A total loss function is constructed. The total loss function includes a semantic consistency loss function and a detection loss function. The first similarity is used as a supervision signal, and the semantic consistency loss function is used to constrain the second similarity to guide the semantic branch to learn the fine-grained semantic information contained in the text description. The real lesion class and the real lesion position bounding box labeled for each X-ray image are used as a training target, and the detection loss function is used to optimize the classification branch and the positioning branch of the initial network to guide the classification branch to learn the lesion class information and the positioning branch to learn the lesion position information. When the total loss function converges, the training is stopped, and the current obtained network is determined as the orthopedic lesion detection network. The processing unit is configured to, in response to an orthopedic disease auxiliary detection instruction for a target X-ray image, input the target X-ray image into the orthopedic lesion detection network for processing to obtain a detection result of the target X-ray image, and send the detection result to a trigger of the orthopedic disease auxiliary detection instruction.

8. A computer device, comprising: The computer device comprises: a memory configured to store at least one instruction; and a processor configured to execute the instruction stored in the memory to implement the orthopedic disease auxiliary detection method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, and the at least one instruction is executed by the processor in the computer device to implement the orthopedic disease auxiliary detection method according to any one of claims 1 to 6.

Images