Bone X-ray image processing method and device, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence, and provides a bone X-ray image processing method, device and equipment and a medium, which can process a bone X-ray image by utilizing a network trained based on a segmentation all-in-one model, and the segmentation all-in-one model is used as a visual basic model, has strong zero sample learning ability, and is high in robustness. The bone edge information related to the lesion in the X-ray image can be extracted under the prompt of the lesion bounding box, and the bone edge image is utilized to guide the network to learn the bone edge features, so that the detection performance of orthopedic diseases is improved, and important clues can be provided for doctors to diagnose the orthopedic diseases to assist the doctors in improving the film reading speed and the diagnosis accuracy.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device, equipment and medium for processing bone X-ray images. Background Art

[0002] As the population ages, the incidence of orthopedic diseases is also rising. Thanks to the low cost, low radiation, and easy operation of X-ray imaging technology, X-ray imaging technology has become the most commonly used imaging examination method for the initial diagnosis of orthopedic diseases. However, since X-ray imaging technology uses the penetrating ability of X-rays to project the three-dimensional human structure onto a two-dimensional plane, the bone X-ray image will inevitably have blurred bone edges due to the overlap of bones and organs. However, the edges of bones in X-ray images can provide important information for doctors to diagnose orthopedic diseases. By observing the edges of bones, it can be determined whether the bones are deformed, displaced, or in other abnormal conditions.

[0003] In the prior art, for bone X-ray images, Ryu et al. used detectors and classifiers to simultaneously detect vertebral compression fractures and fracture severity in X-ray images, and Lee et al. used width and height attention mechanisms to improve the detector to achieve vertebral compression fracture detection. However, the above methods can only detect a single category of orthopedic diseases and cannot fully mine the bone edge information in X-ray images.

[0004] Therefore, in response to the above problems, it has become an urgent issue to study a method that can fully mine the edge information of bones in X-ray images to assist doctors in diagnosing orthopedic diseases. Summary of the invention

[0005] In view of the above, it is necessary to provide a method, device, equipment and medium for processing bone X-ray images, aiming to solve the problem of not being able to fully mine bone edge information in bone X-ray images.

[0006] A method for processing a bone X-ray image, the method comprising:

[0007] An initial network is constructed based on the segmentation model, and a target loss function is constructed; wherein the initial network includes an image feature extraction module, a skeleton edge information mining module, a multi-scale feature fusion module, an edge branch, a classification branch and a regression branch; the target loss function includes an edge loss function and a detection loss function;

[0008] Acquire historical bone X-ray images to construct a training set, and train the initial network based on the training set and the target loss function to obtain a target network;

[0009] In response to a processing instruction for a target bone X-ray image, extracting features from the target bone X-ray image using the image feature extraction module to obtain target image features;

[0010] Using the multi-scale feature fusion module to fuse the target image features to obtain fused features;

[0011] Using the classification branch to classify the fused features to obtain a target type, and using the regression branch to predict the position of the fused features to obtain a target position;

[0012] A processing result of the target bone X-ray image is generated according to the target type and the target position.

[0013] According to a preferred embodiment of the present invention, the structure of the image feature extraction module is the same as the structure of the backbone of the YoLov8 network;

[0014] The skeleton edge information mining module includes the segmentation model;

[0015] The structure of the multi-scale feature fusion module is the same as the structure of the neck of the YoLov8 network;

[0016] The edge branch includes a preset number of upsampling layers, and the features output by each upsampling layer are cascaded with the corresponding features output by the multi-scale feature fusion module along the channel dimension, and the features obtained after the cascade are used as inputs of the next upsampling layer; wherein the preset number is the same as the number of levels of the target image features;

[0017] The structure of the classification branch and the structure of the regression branch are the same as the structure of the detection head of the YoLov8 network.

[0018] According to a preferred embodiment of the present invention, the method further comprises:

[0019] In the process of training the initial network, for each sample in the training set, using the image encoder of the segmentation model to extract the encoded image features of the sample;

[0020] obtaining a ground-truth bounding box of a lesion marked on the sample;

[0021] Using the ground-truth bounding box as a cue, obtaining a bounding box cue embedding of the sample using a cue encoder of the segmentation-everything model;

[0022] Embedding the encoded image features and the bounding box hints into a mask decoder of the segmentation model to obtain a skeleton mask of the sample;

[0023] An edge detector is used to perform edge detection on the bone mask to obtain an initial bone edge map related to the lesion.

[0024] According to a preferred embodiment of the present invention, after obtaining the initial bone edge map related to the lesion, the method further includes:

[0025] Obtaining a target fusion feature corresponding to the sample output by the multi-scale feature fusion module;

[0026] The target fusion feature is input into the edge branch to predict the bone edge using the following formula to obtain the target bone edge map:

[0027] E M =P M ;

[0028]

[0029] E 0 =I edg ;

[0030] Among them, E M represents the features of the output of the Mth upsampling layer; P M represents the feature output by the multi-scale feature fusion module corresponding to the feature output by the Mth upsampling layer; E m-1 Represents the features of the output of the (m-1)th upsampling layer; E 0 Represents the features of the output of the 0th upsampling layer; E m Represents the features of the output of the mth upsampling layer; represents the upsampling operation of the features output by the mth upsampling layer; P m Represents the feature output by the multi-scale feature fusion module corresponding to the feature output by the mth upsampling layer; Indicates P m and Cascade operation; Conv represents the convolution operation, and the size of the convolution kernel is 1×1; I edg represents the target bone edge map; m represents an integer between [1, M], and M is an integer greater than 1.

[0031] According to a preferred embodiment of the present invention, before constructing the target loss function, the method further includes:

[0032] The initial bone edge map output by the bone edge information mining module is used as a supervisory signal to guide the edge branch to learn the bone edge information related to the lesion, and the edge loss function is obtained:

[0033]

[0034] Among them, Ledg represents edge loss; N represents the number of image pixels corresponding to each sample; g i represents the i-th pixel value predicted by the edge branch; s i Represents the i-th pixel value of the initial bone edge map output by the bone edge information mining module; i is an integer greater than or equal to 0.

[0035] According to a preferred embodiment of the present invention, constructing the target loss function includes:

[0036] The objective loss function is constructed using the following formula:

[0037] L total =L det +L edg ;

[0038] Among them, L total represents the objective loss function; L det Represents the detection loss function, and the detection loss function is the same as the loss function of the YoLov8 network.

[0039] According to a preferred embodiment of the present invention, after generating the processing result of the target bone X-ray image according to the target type and the target position, the method further includes:

[0040] Marking the target bone X-ray image according to the processing result to obtain a marked image;

[0041] The marked image is sent to a designated terminal device.

[0042] A device for processing a bone X-ray image, the device comprising:

[0043] A construction unit is used to construct an initial network based on the segmentation model and a target loss function; wherein the initial network includes an image feature extraction module, a skeleton edge information mining module, a multi-scale feature fusion module, an edge branch, a classification branch and a regression branch; the target loss function includes an edge loss function and a detection loss function;

[0044] A training unit, used for acquiring historical bone X-ray images to construct a training set, and training the initial network based on the training set and the target loss function to obtain a target network;

[0045] An extraction unit, configured to extract features of the target bone X-ray image using the image feature extraction module in response to a processing instruction for the target bone X-ray image, to obtain target image features;

[0046] A fusion unit, used to fuse the target image features using the multi-scale feature fusion module to obtain fusion features;

[0047] A processing unit, configured to classify the fused features using the classification branch to obtain a target type, and to predict the position of the fused features using the regression branch to obtain a target position;

[0048] A generating unit is used to generate a processing result of the target bone X-ray image according to the target type and the target position.

[0049] A computer device, comprising:

[0050] a memory storing at least one instruction; and

[0051] A processor executes the instructions stored in the memory to implement the method for processing the bone X-ray image.

[0052] A computer-readable storage medium stores at least one instruction, and the at least one instruction is executed by a processor in a computer device to implement the method for processing bone X-ray images.

[0053] It can be seen from the above technical scheme that the present invention uses a network trained based on the segmentation everything model to process bone X-ray images. The segmentation everything model, as a visual basic model, has a strong zero-sample learning ability, and can extract bone edge information related to lesions in X-ray images under the prompt of the lesion boundary box, and use the bone edge map to guide the network to learn bone edge features, thereby improving the detection performance of orthopedic diseases, and can provide important clues for doctors to diagnose orthopedic diseases, so as to assist doctors in improving the reading speed and diagnostic accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a flow chart of a preferred embodiment of the method for processing bone X-ray images of the present invention.

[0055] Figure 2 It is a functional module diagram of a preferred embodiment of the bone X-ray image processing device of the present invention.

[0056] Figure 3 It is a schematic diagram of the structure of a computer device of a preferred embodiment of the method for processing bone X-ray images of the present invention. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0058] like Figure 1FIG. 1 is a flowchart of a preferred embodiment of the method for processing bone X-ray images of the present invention. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted.

[0059] The method for processing bone X-ray images is applied to one or more computer devices, which are devices that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and their hardware includes but is not limited to microprocessors, application specific integrated circuits (ASIC), programmable gate arrays (FPGA), digital signal processors (DSP), embedded devices, etc.

[0060] The computer device may be any electronic product that can perform human-computer interaction with a user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive network television (IPTV), a smart wearable device, etc.

[0061] The computer device may also include a network device and / or a user device, wherein the network device includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud consisting of a large number of hosts or network servers based on cloud computing.

[0062] The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms.

[0063] Among them, Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0064] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0065] The network where the computer device is located includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), etc.

[0066] S10, constructing an initial network based on the Segment Anything Model (SAM), and constructing a target loss function; wherein the initial network includes an image feature extraction module, a skeleton edge information mining module, a multi-scale feature fusion module, an edge branch, a classification branch and a regression branch; the target loss function includes an edge loss function and a detection loss function.

[0067] In this embodiment, the structure of the image feature extraction module is the same as the backbone of the YoLov8 network;

[0068] The skeleton edge information mining module includes the segmentation model;

[0069] The structure of the multi-scale feature fusion module is the same as the structure of the neck of the YoLov8 network;

[0070] The edge branch includes a preset number of upsampling layers, and the features output by each upsampling layer are cascaded with the corresponding features output by the multi-scale feature fusion module along the channel dimension, and the features obtained after the cascade are used as the input of the next upsampling layer; wherein the preset number is the same as the number of levels of the output features of the image feature extraction module (that is, the preset number is the same as the number of levels of the target image features mentioned later);

[0071] The structure of the classification branch and the structure of the regression branch are the same as the structure of the detection head of the YoLov8 network.

[0072] S11, acquiring historical bone X-ray images to construct a training set, and training the initial network based on the training set and the target loss function to obtain a target network.

[0073] In this embodiment, each sample in the training set has a real boundary box of a marked lesion, which may be marked manually or by other means.

[0074] In this embodiment, the initial network may be trained in the following manner to obtain the target network:

[0075] First, the image feature extraction module is used to extract features from each sample in the training set.

[0076] For example, for each sample in the training set, i.e., a bone X-ray image I, I is used as the input of the image feature extraction module to learn the image features. Specifically, the image feature extraction module can extract M levels of image features in total. F m Represents the learned image features of the mth level. The feature extraction process formula is as follows:

[0077]

[0078] Among them, E n (I) represents the image feature extraction operation on I.

[0079] Secondly, the lesions of orthopedic diseases usually manifest as bone deformation or dislocation of adjacent bones. By observing the bone edges in the X-ray image, it can be determined whether the bones are abnormal, and then the orthopedic disease can be diagnosed. Therefore, the edge information of the bones can be used as an important clue for doctors to analyze X-ray images and diagnose diseases. However, due to the projection characteristics of X-ray imaging, X-ray images usually have the problem of blurred edges, and manual annotation of bone edges is time-consuming, labor-intensive and costly. As a visual basic model, the segmentation model has a powerful zero-sample segmentation capability. The model can segment the area within the bounding box under the prompt of the bounding box, which makes it possible to fully mine the bone edge information related to the lesion from the X-ray image. Therefore, the present embodiment further utilizes the bone edge information mining module, that is, the segmentation model mines the bone edge information of each sample.

[0080] Specifically, in the process of training the initial network, for each sample in the training set, the image encoder of the segmentation model is used to extract the encoded image features of the sample; the true bounding box of the lesion marked on the sample is obtained; the bounding box hint embedding of the sample is obtained using the hint encoder of the segmentation model with the true bounding box as a hint; the encoded image features and the bounding box hint embedding are input into the mask decoder of the segmentation model to obtain the bone mask of the sample; the bone mask is subjected to edge detection using an edge detector to obtain an initial bone edge map related to the lesion.

[0081] For example, taking an X-ray image I as input, the image encoder of the segmentation model is used to extract the X-ray image feature S I , the formula is as follows:

[0082] S I =Φ(I);

[0083] Among them, Φ(I) represents the extraction operation of I by the image encoder that splits all models.

[0084] At the same time, the real bounding box B of the lesion in the X-ray image g As a hint, we use the hint encoder of the split-everything model to get the bounding box hint embedding R g , the formula is as follows:

[0085] R g =ξ(B g );

[0086] Among them, ξ(B g ) represents the hint encoder pair B of the segmentation model g operation.

[0087] Then, the bounding box hints are embedded in R g With image feature S I The mask decoder of all segmentation models is sent together to obtain the bone mask F in the lesion area. mask , the formula is as follows:

[0088] F mask =Ψ(R g , S I );

[0089] Among them, Ψ(R g , S I ) represents the mask decoder pair R of the segmentation model g and S I operation.

[0090] Finally, the bone mask F mask Edge detection is performed to capture the initial bone edge map associated with the lesion. e , the formula is as follows:

[0091] I e =D e (F mask );

[0092] Among them, D e (F mask ) represents the edge detector pair F mask The edge detection operation can be implemented by edge detection operators such as Canny.

[0093] Thirdly, the multi-scale feature fusion module is used to perform multi-scale fusion on the features extracted by the image feature extraction module.

[0094] For example: the image features output by the image feature extraction module With different resolutions, the feature map with a larger resolution contains richer disease details, and the feature map with a smaller resolution focuses more on the semantic information of the disease. Since there are many types of orthopedic diseases and their sizes vary, in order to enable the network to simultaneously detect orthopedic diseases of different categories and sizes, this embodiment constructs the multi-scale feature fusion module to fully fuse details and semantic information. The feature fusion process formula is as follows:

[0095]

[0096] in, Represents the multi-scale feature fusion module Fusion operation; Represents the corresponding fusion features.

[0097] Again, the edge branch is used to further guide the model to learn edge information.

[0098] Specifically, in order to enable the orthopedic disease auxiliary diagnosis network to have the ability to learn bone edge information, the edge branch is self-constructed, and the initial bone edge map output by the bone edge information mining module is used as a supervisory signal to guide the edge branch to learn the bone edge information related to the lesion. Specifically, after obtaining the initial bone edge map related to the lesion, the method also includes:

[0099] Obtaining a target fusion feature corresponding to the sample output by the multi-scale feature fusion module;

[0100] The target fusion feature is input into the edge branch to predict the bone edge using the following formula to obtain the target bone edge map:

[0101] E M =P M ;

[0102]

[0103] E 0 =I edg ;

[0104] Among them, E M represents the features of the output of the Mth upsampling layer; P M represents the feature output by the multi-scale feature fusion module corresponding to the feature output by the Mth upsampling layer; E m-1 Represents the features of the output of the (m-1)th upsampling layer; E 0 Represents the features of the output of the 0th upsampling layer; E mRepresents the features of the output of the mth upsampling layer; represents the upsampling operation of the features output by the mth upsampling layer; P m Represents the feature output by the multi-scale feature fusion module corresponding to the feature output by the mth upsampling layer; Indicates the relationship between Pm and Cascade operation; Conv represents the convolution operation, and the size of the convolution kernel is 1×1; I edg represents the target bone edge map; m represents an integer between [1, M], and M is an integer greater than 1.

[0105] For example, when M is 4, the features output by each upsampling layer are concatenated with the corresponding features output by the multi-scale feature fusion module along the channel dimension and used as the input of the next upsampling layer, and so on, gradually restoring the spatial resolution of the image and generating the final target bone edge map.

[0106] Finally, the initial network is trained based on the training set and the target loss function to obtain the target network.

[0107] Specifically, before constructing the target loss function, the method further includes:

[0108] The initial bone edge map output by the bone edge information mining module is used as a supervisory signal to guide the edge branch to learn the bone edge information related to the lesion, and the edge loss function is obtained:

[0109]

[0110] Among them, L edg represents edge loss; N represents the number of image pixels corresponding to each sample; g i represents the i-th pixel value predicted by the edge branch; s i Represents the i-th pixel value of the initial bone edge map output by the bone edge information mining module; i is an integer greater than or equal to 0.

[0111] During the training phase of the target network, the initial bone edge map output by the bone edge information mining module is used to guide the edge branch to learn the edge information of the bone, and the learning process of the edge information is optimized by constraining the prediction results of the edge branch to be consistent with the initial bone edge map, so that the prediction results of the edge branch are consistent with the initial bone edge map.

[0112] Furthermore, the constructing the target loss function includes:

[0113] The objective loss function is constructed using the following formula:

[0114] L total=L det +L edg ;

[0115] Among them, L total represents the objective loss function; L det Represents the detection loss function, and the detection loss function is the same as the loss function of the YoLov8 network.

[0116] It can be seen that during the training process, the loss function L total It consists of two parts, among which the detection loss function L det For optimizing the learning process of the classification branch and the regression branch, the marginal loss function L edg The learning process for optimizing the edge branch, while optimizing the edge branch, the classification branch and the regression branch, can make the features learned by the network suitable for predicting the features of the category and location of orthopedic diseases in X-ray images, as well as for predicting bone edges.

[0117] In the above embodiment, the zero-sample segmentation capability of the visual basic model is used to segment all models to improve the existing target detection network, and an edge branch auxiliary network is designed to fully mine the bone edge information, so that the trained target network can process bone X-ray images more accurately.

[0118] S12, in response to a processing instruction for a target bone X-ray image, extracting features from the target bone X-ray image using the image feature extraction module to obtain target image features.

[0119] In this embodiment, the processing instruction can be triggered by relevant medical personnel, or can be automatically triggered when a bone X-ray image is detected to be uploaded.

[0120] S13, using the multi-scale feature fusion module to fuse the target image features to obtain fused features.

[0121] In this embodiment, detail and semantic information can be fully integrated through feature fusion to improve the accuracy of detection.

[0122] S14, using the classification branch to classify the fused features to obtain a target type, and using the regression branch to predict the position of the fused features to obtain a target position.

[0123] In this embodiment, after the target network is trained, when the target network is actually used to process the bone X-ray image, only the image feature extraction module, the multi-scale feature fusion module, the classification branch and the regression branch are used to complete the image processing, and finally the predicted orthopedic disease type and location are obtained. The bone edge information mining module and the edge branch are only used to optimize the modules of the entire network during the training stage.

[0124] S15, generating a processing result of the target bone X-ray image according to the target type and the target position.

[0125] In this embodiment, after generating the processing result of the target bone X-ray image according to the target type and the target position, the method further includes:

[0126] Marking the target bone X-ray image according to the processing result to obtain a marked image;

[0127] The marked image is sent to a designated terminal device.

[0128] The designated terminal device may be a terminal device of relevant medical staff. For example, the type and location of orthopedic diseases predicted by the target network may be marked in the target bone X-ray image with a special color or a box, and the marked image may be sent to the doctor's office equipment to assist the doctor in quickly reading and diagnosing the image, which can not only improve the doctor's reading speed, but also improve the doctor's diagnostic efficiency.

[0129] It can be seen from the above technical scheme that the present invention can process bone X-ray images using a network trained based on the segmentation model. The segmentation model, as a visual basic model, has a strong zero-sample learning ability, and can extract bone edge information related to lesions in X-ray images under the prompt of the lesion boundary box, and use the bone edge map to guide the network to learn bone edge features, thereby improving the detection performance of orthopedic diseases, and can provide important clues for doctors to diagnose orthopedic diseases, so as to assist doctors in improving the reading speed and diagnostic accuracy.

[0130] like Figure 2 As shown, it is a functional module diagram of a preferred embodiment of the processing device for bone X-ray images of the present invention. The processing device 11 for bone X-ray images includes a construction unit 110, a training unit 111, an extraction unit 112, a fusion unit 113, a processing unit 114, and a generation unit 115. The module / unit referred to in the present invention refers to a series of computer program segments that can be executed by a processor and can complete fixed functions, which are stored in a memory. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0131] The construction unit 110 is used to construct an initial network based on the segmentation model and to construct a target loss function; wherein the initial network includes an image feature extraction module, a skeleton edge information mining module, a multi-scale feature fusion module, an edge branch, a classification branch and a regression branch; the target loss function includes an edge loss function and a detection loss function;

[0132] The training unit 111 is used to acquire historical bone X-ray images to construct a training set, and train the initial network based on the training set and the target loss function to obtain a target network;

[0133] The extraction unit 112 is used to extract features of the target bone X-ray image using the image feature extraction module in response to a processing instruction for the target bone X-ray image to obtain target image features;

[0134] The fusion unit 113 is used to fuse the target image features using the multi-scale feature fusion module to obtain fused features;

[0135] The processing unit 114 is used to classify the fused features using the classification branch to obtain a target type, and to predict the position of the fused features using the regression branch to obtain a target position;

[0136] The generating unit 115 is used to generate a processing result of the target bone X-ray image according to the target type and the target position.

[0137] It can be seen from the above technical scheme that the present invention can process bone X-ray images using a network trained based on the segmentation model. The segmentation model, as a visual basic model, has a strong zero-sample learning ability, and can extract bone edge information related to lesions in X-ray images under the prompt of the lesion boundary box, and use the bone edge map to guide the network to learn bone edge features, thereby improving the detection performance of orthopedic diseases, and can provide important clues for doctors to diagnose orthopedic diseases, so as to assist doctors in improving the reading speed and diagnostic accuracy.

[0138] like Figure 3 FIG. 1 is a schematic diagram of the structure of a computer device of a preferred embodiment of the method for processing bone X-ray images of the present invention.

[0139] The computer device 1 may include a memory 12, a processor 13 and a bus, and may also include a computer program stored in the memory 12 and executable on the processor 13, such as a processing program for bone X-ray images.

[0140] Those skilled in the art will appreciate that the schematic diagram is merely an example of the computer device 1 and does not constitute a limitation on the computer device 1. The computer device 1 may be a bus-type structure or a star-type structure. The computer device 1 may also include more or less other hardware or software than shown in the diagram, or a different arrangement of components. For example, the computer device 1 may also include input and output devices, network access devices, etc.

[0141] It should be noted that the computer device 1 is only an example, and other existing or future electronic products that are suitable for the present invention should also be included in the protection scope of the present invention and included here by reference.

[0142] The memory 12 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 12 may be an internal storage unit of the computer device 1, such as a mobile hard disk of the computer device 1. In other embodiments, the memory 12 may also be an external storage device of the computer device 1, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the computer device 1. Further, the memory 12 may also include both an internal storage unit of the computer device 1 and an external storage device. The memory 12 may not only be used to store application software and various types of data installed in the computer device 1, such as the code of the processing program of the bone X-ray image, etc., but also be used to temporarily store data that has been output or is to be output.

[0143] In some embodiments, the processor 13 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The processor 13 is the control core (Control Unit) of the computer device 1, and uses various interfaces and lines to connect various components of the entire computer device 1, and executes or executes programs or modules stored in the memory 12 (for example, executing a processing program for bone X-ray images, etc.), and calls data stored in the memory 12 to execute various functions of the computer device 1 and process data.

[0144] The processor 13 executes the operating system of the computer device 1 and various installed applications. The processor 13 executes the applications to implement the steps in the above-mentioned embodiments of the method for processing bone X-ray images, for example Figure 1 Steps shown.

[0145] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to complete the present invention. The one or more modules / units may be a series of computer-readable instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the computer device 1. For example, the computer program may be divided into a construction unit 110, a training unit 111, an extraction unit 112, a fusion unit 113, a processing unit 114, and a generation unit 115.

[0146] The above-mentioned integrated unit implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, and includes a number of instructions for enabling a computer device (which can be a personal computer, a computer device, or a network device, etc.) or a processor to execute the part of the method for processing bone X-ray images described in various embodiments of the present invention.

[0147] If the module / unit integrated in the computer device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware devices through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of each of the above-mentioned method embodiments can be implemented.

[0148] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory, etc.

[0149] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0150] The blockchain referred to in this invention is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0151] The bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 The bus is represented by only one straight line, but it does not mean that there is only one bus or one type of bus. The bus is configured to realize the connection and communication between the memory 12 and at least one processor 13, etc.

[0152] Although not shown, the computer device 1 may also include a power source (such as a battery) for supplying power to each component. Preferably, the power source may be logically connected to the at least one processor 13 through a power management device, so that the power management device can realize functions such as charging management, discharging management, and power consumption management. The power source may also include any components such as one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, etc. The computer device 1 may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0153] Furthermore, the computer device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is usually used to establish a communication connection between the computer device 1 and other computer devices.

[0154] Optionally, the computer device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touch device. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the computer device 1 and to display a visual user interface.

[0155] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0156] It can be understood by those skilled in the art that Figure 3 The structure shown does not constitute a limitation on the computer device 1, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0157] Combination Figure 1 The memory 12 in the computer device 1 stores a plurality of instructions to implement a method for processing a bone X-ray image, and the processor 13 can execute the plurality of instructions to implement:

[0158] An initial network is constructed based on the segmentation model, and a target loss function is constructed; wherein the initial network includes an image feature extraction module, a skeleton edge information mining module, a multi-scale feature fusion module, an edge branch, a classification branch and a regression branch; the target loss function includes an edge loss function and a detection loss function;

[0159] Acquire historical bone X-ray images to construct a training set, and train the initial network based on the training set and the target loss function to obtain a target network;

[0160] In response to a processing instruction for a target bone X-ray image, extracting features from the target bone X-ray image using the image feature extraction module to obtain target image features;

[0161] Using the multi-scale feature fusion module to fuse the target image features to obtain fused features;

[0162] Using the classification branch to classify the fused features to obtain a target type, and using the regression branch to predict the position of the fused features to obtain a target position;

[0163] A processing result of the target bone X-ray image is generated according to the target type and the target position.

[0164] Specifically, the specific implementation method of the processor 13 for the above instructions can refer to Figure 1 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0165] It should be noted that the data involved in this case were all obtained legally.

[0166] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0167] The present invention can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0168] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0169] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0170] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0171] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0172] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the present invention can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

[0173] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for processing a bone X-ray image, characterized in that: The method for processing the bone X-ray image comprises: An initial network is constructed based on the segmentation model, and a target loss function is constructed; wherein the initial network includes an image feature extraction module, a skeleton edge information mining module, a multi-scale feature fusion module, an edge branch, a classification branch and a regression branch; the target loss function includes an edge loss function and a detection loss function; Acquire historical bone X-ray images to construct a training set, and train the initial network based on the training set and the target loss function to obtain a target network; In response to a processing instruction for a target bone X-ray image, extracting features from the target bone X-ray image using the image feature extraction module to obtain target image features; Using the multi-scale feature fusion module to fuse the target image features to obtain fused features; Using the classification branch to classify the fused features to obtain a target type, and using the regression branch to predict the position of the fused features to obtain a target position; A processing result of the target bone X-ray image is generated according to the target type and the target position.

2. The method for processing bone X-ray images according to claim 1, characterized in that: The structure of the image feature extraction module is the same as the backbone of the YoLov8 network; The skeleton edge information mining module includes the segmentation model; The structure of the multi-scale feature fusion module is the same as the structure of the neck of the YoLov8 network; The edge branch includes a preset number of upsampling layers, and the features output by each upsampling layer are cascaded with the corresponding features output by the multi-scale feature fusion module along the channel dimension, and the features obtained after the cascade are used as inputs of the next upsampling layer; wherein the preset number is the same as the number of levels of the target image features; The structure of the classification branch and the structure of the regression branch are the same as the structure of the detection head of the YoLov8 network.

3. The method for processing bone X-ray images according to claim 2, characterized in that: The method further comprises: In the process of training the initial network, for each sample in the training set, using the image encoder of the segmentation model to extract the encoded image features of the sample; obtaining a ground-truth bounding box of a lesion marked on the sample; Using the ground-truth bounding box as a cue, obtaining a bounding box cue embedding of the sample using a cue encoder of the segmentation-everything model; Embedding the encoded image features and the bounding box hints into a mask decoder of the segmentation model to obtain a skeleton mask of the sample; An edge detector is used to perform edge detection on the bone mask to obtain an initial bone edge map related to the lesion.

4. The method for processing a bone X-ray image according to claim 3, characterized in that: After obtaining the initial bone edge map related to the lesion, the method further includes: Obtaining a target fusion feature corresponding to the sample output by the multi-scale feature fusion module; The target fusion feature is input into the edge branch to predict the bone edge using the following formula to obtain the target bone edge map: AND M =P M ; E 0 =I edg ; Among them, E M represents the features of the output of the Mth upsampling layer; P M represents the feature output by the multi-scale feature fusion module corresponding to the feature output by the Mth upsampling layer; E m-1 Represents the features of the output of the (m-1)th upsampling layer; E 0 Represents the features of the output of the 0th upsampling layer; E m Represents the features of the output of the mth upsampling layer; represents the upsampling operation of the features output by the mth upsampling layer; P m Represents the feature output by the multi-scale feature fusion module corresponding to the feature output by the mth upsampling layer; Indicates P m and Cascade operation; Conv represents the convolution operation, and the size of the convolution kernel is 1×1; I edg represents the target bone edge map; m represents an integer between [1, M], and M is an integer greater than 1.

5. The method for processing bone X-ray images according to claim 3, characterized in that: Before constructing the target loss function, the method further includes: The initial bone edge map output by the bone edge information mining module is used as a supervisory signal to guide the edge branch to learn the bone edge information related to the lesion, and the edge loss function is obtained: Among them, L edg represents edge loss; N represents the number of image pixels corresponding to each sample; g i represents the i-th pixel value predicted by the edge branch; s i Represents the i-th pixel value of the initial bone edge map output by the bone edge information mining module; i is an integer greater than or equal to 0.

6. The method for processing a bone X-ray image according to claim 5, characterized in that: The constructing target loss function comprises: The objective loss function is constructed using the following formula: L total =L det +L edg ; Among them, L total represents the objective loss function; L det Represents the detection loss function, and the detection loss function is the same as the loss function of the YoLov8 network.

7. The method for processing bone X-ray images according to claim 1, characterized in that: After generating the processing result of the target bone X-ray image according to the target type and the target position, the method further includes: Marking the target bone X-ray image according to the processing result to obtain a marked image; The marked image is sent to a designated terminal device.

8. A device for processing bone X-ray images, characterized in that: The bone X-ray image processing device comprises: A construction unit is used to construct an initial network based on the segmentation model and a target loss function; wherein the initial network includes an image feature extraction module, a skeleton edge information mining module, a multi-scale feature fusion module, an edge branch, a classification branch and a regression branch; the target loss function includes an edge loss function and a detection loss function; A training unit, used for acquiring historical bone X-ray images to construct a training set, and training the initial network based on the training set and the target loss function to obtain a target network; An extraction unit, configured to extract features of the target bone X-ray image using the image feature extraction module in response to a processing instruction for the target bone X-ray image, to obtain target image features; A fusion unit, used to fuse the target image features using the multi-scale feature fusion module to obtain fusion features; A processing unit, configured to classify the fused features using the classification branch to obtain a target type, and to predict the position of the fused features using the regression branch to obtain a target position; A generating unit is used to generate a processing result of the target bone X-ray image according to the target type and the target position.

9. A computer device, characterized in that: The computer device comprises: a memory storing at least one instruction; and A processor executes the instructions stored in the memory to implement the method for processing bone X-ray images as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, and the at least one instruction is executed by a processor in a computer device to implement the method for processing bone X-ray images as described in any one of claims 1 to 7.

Images