Hip joint lesion identification method, device and equipment based on deep learning algorithm

Abstract

The present application provides a method, device, equipment and computer-readable storage medium for hip joint lesion recognition based on a deep learning algorithm. The method for hip joint lesion recognition based on a deep learning algorithm includes: obtaining a hip joint X-ray image; inputting the hip joint X-ray image into a preset hip joint lesion recognition model, and outputting a detection frame position, lesion recognition results and a heat map; wherein the hip joint lesion recognition model includes a network structure for simultaneously focusing on global and local information when processing image tasks. According to the embodiment of the present application, the efficiency of hip joint lesion recognition can be improved.

Description

Technical Field

[0001] The present application belongs to the field of deep learning intelligent recognition technology, and in particular, relates to a hip joint lesion recognition method, device, equipment and computer-readable storage medium based on a deep learning algorithm. Background Art

[0002] At present, the related art of hip joint lesion recognition is that one model can only recognize one specific lesion type, resulting in low efficiency of lesion recognition.

[0003] Therefore, how to improve the efficiency of hip joint lesion identification is a technical problem that technical personnel in this field urgently need to solve. Summary of the invention

[0004] The embodiments of the present application provide a method, device, equipment and computer-readable storage medium for identifying hip joint lesions based on a deep learning algorithm, which can improve the efficiency of identifying hip joint lesions.

[0005] In a first aspect, an embodiment of the present application provides a method for identifying hip joint lesions based on a deep learning algorithm, comprising:

[0006] Obtain hip x-rays;

[0007] Input the hip joint X-ray image into the preset hip joint lesion recognition model, and output the detection frame position, lesion recognition result and heat map;

[0008] Among them, the hip joint lesion recognition model includes a network structure for simultaneously focusing on global and local information when processing image tasks.

[0009] Optionally, the network structure used to focus on both global and local information when processing image tasks is the Hourglass network structure, including:

[0010] Input layer, used to input hip joint X-ray images;

[0011] The first convolutional layer is used to extract the feature map of the hip joint X-ray image;

[0012] Downsampling layer, used to reduce the size of feature maps;

[0013] Residual blocks with residual connections for network training and information transfer;

[0014] Upsampling layer, used to increase the size of the feature map;

[0015] The second convolutional layer is used to extract the upsampled feature map;

[0016] The output layer is used to output the results.

[0017] Optionally, output the detection box position, including:

[0018] A regression method is used to predict the center point of each target in the hip joint X-ray image;

[0019] For each spatial location, the head outputs a score indicating whether the location contains the target center;

[0020] The head is used to predict the size information of the target, including the width and height of the target box; these predictions are relative offsets from the center point.

[0021] Optionally, output lesion identification results, including:

[0022] The possible target categories at each center point are predicted through the head output;

[0023] Probability distribution calculated by sonmax function;

[0024] Based on the probability distribution, the lesion recognition results are output.

[0025] Optionally, output heatmap, including:

[0026] Extract the feature map through the detection frame;

[0027] The sigmoid activation function is used to generate a visual heat map.

[0028] Optionally, the network structure used to focus on both global and local information when processing image tasks is the UNet network structure, including:

[0029] One downsampling and one upsampling enable the network to capture the global and local features of the image while retaining high-resolution information.

[0030] Optionally, also include:

[0031] Determine the loss functions of the detection box position, lesion recognition results, and heat map, as well as the corresponding weights;

[0032] Based on the three loss functions and their corresponding weights, the overall loss function is calculated.

[0033] In a second aspect, an embodiment of the present application provides a hip joint lesion recognition device based on a deep learning algorithm, the device comprising:

[0034] An image acquisition module, used for acquiring an X-ray image of a hip joint;

[0035] The lesion recognition module is used to input the hip joint X-ray image into the preset hip joint lesion recognition model, and output the detection frame position, lesion recognition result and thermal map;

[0036] Among them, the hip joint lesion recognition model includes a network structure for simultaneously focusing on global and local information when processing image tasks.

[0037] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising: a processor and a memory storing computer program instructions;

[0038] When the processor executes the computer program instructions, it implements the hip joint lesion identification method based on the deep learning algorithm as shown in the first aspect.

[0039] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the method for identifying hip joint lesions based on a deep learning algorithm as shown in the first aspect is implemented.

[0040] The hip joint lesion identification method, device, equipment and computer-readable storage medium based on deep learning algorithm in the embodiments of the present application can improve the efficiency of hip joint lesion identification.

[0041] The embodiments of the present application provide a method, device, equipment and computer-readable storage medium for identifying hip joint lesions based on a deep learning algorithm, which can improve the efficiency of identifying hip joint lesions.

[0042] The hip joint lesion recognition method based on deep learning algorithm includes: obtaining hip joint X-ray images; inputting the hip joint X-ray images into a preset hip joint lesion recognition model, and outputting the detection frame position, lesion recognition results and heat map; wherein the hip joint lesion recognition model includes a network structure for simultaneously focusing on global and local information when processing image tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 It is a flowchart of a hip joint lesion identification method based on a deep learning algorithm provided by an embodiment of the present application;

[0045] Figure 2 It is a flowchart of a hip joint lesion identification method based on a deep learning algorithm provided by an embodiment of the present application;

[0046] Figure 3This is a schematic diagram of the Hourglass network structure provided by an embodiment of the present application;

[0047] Figure 4 This is a schematic diagram of the UNet network structure provided by an embodiment of the present application;

[0048] Figure 5 It is a structural schematic diagram of a hip joint lesion identification device based on a deep learning algorithm provided by an embodiment of the present application;

[0049] Figure 6 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0050] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0051] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0052] In order to solve the problems of the prior art, the embodiments of the present application provide a method, device, equipment and computer-readable storage medium for identifying hip joint lesions based on a deep learning algorithm. The following first introduces the method for identifying hip joint lesions based on a deep learning algorithm provided in the embodiments of the present application.

[0053] Figure 1 FIG. 1 is a flow chart of a method for identifying hip joint lesions based on a deep learning algorithm provided by an embodiment of the present application. Figure 1 As shown, the hip joint lesion recognition method based on deep learning algorithm includes:

[0054] S101, obtaining an X-ray image of the hip joint;

[0055] S102, inputting the hip joint X-ray image into a preset hip joint lesion recognition model, and outputting the detection frame position, lesion recognition result and heat map;

[0056] Among them, the hip joint lesion recognition model includes a network structure for simultaneously focusing on global and local information when processing image tasks.

[0057] Specifically, Figure 2 As shown in the figure, this algorithm first extracts the feature map of the X-ray image through several hourglass blocks. Then it gives three output heads. The first output head imitates the head output method of the center Ne model, outputting the center point of the hip joint position to be detected as well as the width and height (Bounding Box RegressionHead). The second branch is the multi-label classification label (Classification Head), which performs multi-label lesion classification through the classification head, and outputs a total of 9 labels:

[0058] Incomplete femoral head, femoral head collapse, gap changes, cystic changes, femoral head dislocation, osteophytes, cartilage sclerosis, sacroiliac joint fusion, and insect bite changes.

[0059] The descriptions of the above 9 tags are as follows:

[0060] Incomplete femoral head:

[0061] Description: The contour of the femoral head is incomplete or missing.

[0062] Femoral head collapse:

[0063] Description: Subsidence or collapse of the surface of the femoral head.

[0064] Gap Change:

[0065] Description: Changes in the joint space in the hip joint, which may become narrower or wider.

[0066] Cystic changes:

[0067] Description: Fluid-filled cysts form around joints.

[0068] Femoral head dislocation:

[0069] Description: The femoral head is not in its normal position and is out of the acetabulum.

[0070] Osteophytes:

[0071] Description: Extra bone tissue growth on the surface of a bone.

[0072] Cartilage sclerosis:

[0073] Description: The cartilage in the joints becomes hardened and loses its elasticity.

[0074] Sacroiliac joint fusion:

[0075] Description: Fusion of the joints between the sacrum and ilium, resulting in loss of normal joint motion.

[0076] Insect bite changes:

[0077] Description: Insect-like changes in bone tissue, usually small bone defects.

[0078] The third branch is the heat map label of the hip joint (Segmentation Head). Since the feature map extracted by hourglass itself has the property of a heat map, this property can be reused in the model. After extracting the feature map of the hip joint, the heat map of the hip joint is generated through the sigmoid operation, which is more conducive to the visualization effect.

[0079] In one embodiment, the network structure for simultaneously focusing on global and local information when processing image tasks is an Hourglass network structure, including:

[0080] Input layer, used to input hip joint X-ray images;

[0081] The first convolutional layer is used to extract the feature map of the hip joint X-ray image;

[0082] Downsampling layer, used to reduce the size of feature maps;

[0083] Residual blocks with residual connections for network training and information transfer;

[0084] Upsampling layer, used to increase the size of the feature map;

[0085] The second convolutional layer is used to extract the upsampled feature map;

[0086] The output layer is used to output the results.

[0087] Specifically, Figure 3 As shown in the figure, "Hourglass" is usually used to describe a type of network structure with symmetry, layer-by-layer reduction and increase, which is usually used for tasks in image processing, such as human posture estimation. The network diagram of this structure looks like an hourglass, hence the name.

[0088] illustrate:

[0089] Input: Input image or feature map.

[0090] Convolutional Layers: A series of convolutional layers to extract features from images.

[0091] Downsampling: Downsampling is achieved through pooling or convolution operations, which reduces the size of the feature map.

[0092] Residual Blocks: Residual blocks containing residual connections help with network training and information transfer.

[0093] Upsampling: Increase the size of the feature map through upsampling operations (such as nearest neighbor interpolation).

[0094] Convolutional Layers: Another set of convolutional layers to further process the upsampled features.

[0095] Output: The output of the network, which may be predicted pose key points, segmentation results, etc., depending on the task of the network.

[0096] The key feature of the Hourglass network structure is to retain high-level semantic information while reducing the size, and then gradually restore details through upsampling. This structure enables the network to focus on both global and local information when processing image tasks, so it has achieved good performance in many computer vision tasks, especially in areas such as human pose estimation.

[0097] In one embodiment, outputting the detection box position includes:

[0098] A regression method is used to predict the center point of each target in the hip joint X-ray image;

[0099] For each spatial location, the head outputs a score indicating whether the location contains the target center;

[0100] The head is used to predict the size information of the target, including the width and height of the target box; these predictions are relative offsets from the center point.

[0101] Specifically, center point prediction (Center Heatmap):

[0102] This patent uses regression to predict the center point of each object in the image. For each spatial position, the head outputs a score indicating whether the position contains the center of the object.

[0103] Target size prediction (Size Prediction):

[0104] The head predicts the size information of the target, usually the width and height of the target box. These predictions are relative offsets from the center point.

[0105] In one embodiment, outputting the lesion identification result includes:

[0106] The possible target categories at each center point are predicted through the head output;

[0107] Probability distribution calculated by the softmax function;

[0108] Based on the probability distribution, the lesion recognition results are output.

[0109] Specifically, class prediction (Class Prediction):

[0110] This patent also predicts the possible target categories at each center point through the head output. The probability distribution is calculated by the softmax function.

[0111] In one embodiment, outputting a heat map includes:

[0112] Extract the feature map through the detection frame;

[0113] The sigmoid activation function is used to generate a visual heat map.

[0114] Specifically, heat map Mask Generation:

[0115] The patent head outputs a heat map of the center point, where the value at each position indicates whether the position contains the target center. This heat map is usually processed by a sigmoid activation function to make its value between 0 and 1.

[0116] Specific method of generating heat map:

[0117] In CNN networks, by adding a sigmoid activation function after the feature map, it can be converted into a value in the range of [0, 1] to generate a heat map. This technique is often used to visualize the network's attention to the input, that is, the area that the network focuses on when processing the input image.

[0118] Head output mode:

[0119] There are three heads in this patent, one for target object segmentation (Segmentation Head), one for target object classification (Classification Head), and one for target object coordinate positioning (BoundingBox Regression Head).

[0120] The details of the three Heads are as follows:

[0121] Each head has its specific output, which can be jointly trained by comparing these outputs with the corresponding targets through appropriately designed loss functions. This multi-head structure allows the network to handle segmentation, regression, and classification tasks simultaneously, improving the versatility of the model.

[0122] In one embodiment, the network structure for simultaneously focusing on global and local information when processing image tasks is a UNet network structure, including:

[0123] One downsampling and one upsampling enable the network to capture the global and local features of the image while retaining high-resolution information.

[0124] Specifically, Figure 4 As shown in the figure, UNet (full name U-shaped Network) is a convolutional neural network (CNN) structure for image segmentation, which was originally used for biomedical image segmentation. The uniqueness of the UNet structure is that it adopts a U-shaped architecture, consisting of a downsampling (encoder) and an upsampling (decoder) part, which enables the network to capture the global and local features of the image while retaining high-resolution information.

[0125] In one embodiment, it further includes:

[0126] Determine the loss functions of the detection box position, lesion recognition results, and heat map, as well as the corresponding weights;

[0127] Based on the three loss functions and their corresponding weights, the overall loss function is calculated.

[0128] Specifically, the following is the general form of the total loss function of CenterNet:

[0129] L total =λ center ·L center +λ size ·L size +λ class ·L class +λ reg ·L reg

[0130] in:

[0131] L center is the center point prediction loss.

[0132] L size is the target box size loss.

[0133] L class is the class loss.

[0134] L reg is the mask prediction loss.

[0135] λ center , size , class , reg is the weight hyperparameter of the corresponding loss term, which is used to balance the contribution of each loss.

[0136]

[0137]

[0138]

[0139]

[0140] Here, N is the total number of prediction boxes, y ij is a binary label of whether the actual center point contains the target, is the model's prediction score for the center point, b ij are the coordinates of the actual target box, is the model's prediction of the target box. C is the number of categories.

[0141] In deep learning, "end-to-end" training refers to optimizing the entire system or model as a single, end-to-end trainable whole, rather than splitting the system into multiple stages and training each stage separately. This means that the model can generate the final output directly from the input without manual design or manual feature extraction.

[0142] This application uses a model to achieve end-to-end training and reasoning of hip joint lesions in X-ray images, which reduces the complexity of the model and the reasoning time of the model.

[0143] Figure 5 : is a schematic diagram of the structure of a hip joint lesion recognition device based on a deep learning algorithm provided by an embodiment of the present application. The hip joint lesion recognition device based on a deep learning algorithm comprises:

[0144] An image acquisition module 501 is used to acquire an X-ray image of a hip joint;

[0145] The lesion recognition module 502 is used to input the hip joint X-ray image into a preset hip joint lesion recognition model, and output the detection frame position, lesion recognition result and heat map;

[0146] Among them, the hip joint lesion recognition model includes a network structure for simultaneously focusing on global and local information when processing image tasks.

[0147] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown.

[0148] The electronic device may include a processor 601 and a memory 602 storing computer program instructions.

[0149] Specifically, the processor 601 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0150] The memory 602 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 602 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 602 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 602 may be inside or outside the electronic device. In a particular embodiment, the memory 602 may be a non-volatile solid-state memory.

[0151] In one embodiment, the memory 602 may be a read-only memory (ROM). In one embodiment, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these.

[0152] The processor 601 reads and executes computer program instructions stored in the memory 602 to implement any one of the hip joint lesion identification methods based on deep learning algorithms in the above embodiments.

[0153] In one example, the electronic device may further include a communication interface 603 and a bus 610. Figure 6As shown, the processor 601, the memory 602, and the communication interface 603 are connected via a bus 610 and communicate with each other.

[0154] The communication interface 603 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0155] Bus 610 includes hardware, software or both, and the parts of electronic equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 610 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.

[0156] In addition, in combination with the hip joint lesion identification method based on deep learning algorithm in the above embodiment, the embodiment of the present application can provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any one of the hip joint lesion identification methods based on deep learning algorithm in the above embodiment is implemented.

[0157] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0158] The functional modules shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. Programs or code segments can be stored in machine-readable media, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable media" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0159] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0160] The above reference is according to the method of the embodiment of the present application, the flow chart of the device (system) and the computer program product and / or the block diagram described various aspects of the present application.It should be understood that each square box in the flow chart and / or the block diagram and the combination of each square box in the flow chart and / or the block diagram can be realized by computer program instructions.These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the realization of the function / action specified in one or more square boxes of the flow chart and / or the block diagram.Such a processor can be but is not limited to a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit.It can also be understood that each square box in the block diagram and / or the flow chart and the combination of the square boxes in the block diagram and / or the flow chart can also be realized by the dedicated hardware that performs the specified function or action, or can be realized by the combination of dedicated hardware and computer instructions.

[0161] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A hip joint lesion recognition method based on deep learning algorithm, characterized in that: include: Obtain hip x-rays; Input the hip joint X-ray image into the preset hip joint lesion recognition model, and output the detection frame position, lesion recognition result and heat map; Among them, the hip joint lesion recognition model includes a network structure for simultaneously focusing on global and local information when processing image tasks; First, the feature map of the X-ray image is extracted through several hourglass blocks; then three output heads are given; the first output head, imitating the head output method of the CenterNet model, outputs the center point, width and height of the hip joint position to be detected; the second branch is the multi-label classification label, and the multi-label lesion classification is performed through the classification head, and a total of 9 labels are output: Incomplete femoral head, femoral head collapse, gap changes, cystic changes, femoral head dislocation, osteophytes, cartilage sclerosis, sacroiliac joint fusion, insect bite changes; The descriptions of the above 9 tags are as follows: Incomplete femoral head: Description: The contour of the femoral head is incomplete or missing; Femoral head collapse: Description: Subsidence or collapse of the femoral head surface; Gap Change: Description: Changes in the joint space in the hip joint, which may narrow or widen; Cystic changes: Description: Fluid cysts form around joints; Femoral head dislocation: Description: The femoral head is not in its normal position and is out of the acetabulum; Osteophytes: Description: Extra bone tissue growth on the surface of bones; Cartilage sclerosis: Description: The cartilage in the joints becomes hardened and loses its elasticity; Sacroiliac joint fusion: Description: Fusion of the joints between the sacrum and ilium, loss of normal joint movement; Insect bite changes: Description: Bone tissue shows insect-like changes, usually small bone defects; The third branch is the heat map label of the hip joint. Since the feature map extracted by Hourglass has the property of a heat map, this property is reused in the model. After extracting the feature map of the hip joint, a heat map of the hip joint is generated through a sigmoid operation, which is more conducive to visualization. The network structure used to focus on both global and local information when processing image tasks is the Hourglass network structure, which includes: Input layer, used to input hip joint X-ray images; The first convolutional layer is used to extract the feature map of the hip joint X-ray image; Downsampling layer, used to reduce the size of feature maps; Residual blocks with residual connections for network training and information transfer; Upsampling layer, used to increase the size of the feature map; The second convolutional layer is used to extract the upsampled feature map; Output layer, used to output results; Specifically, Hourglass is usually used to describe a type of network structure that is symmetrical, decreasing and increasing layer by layer, and is usually used for tasks in image processing; Hourglass network structure description: Input: input image or feature map; Convolutional Layers: A series of convolutional layers used to extract features of the image; Downsampling: Downsampling achieved through pooling or convolution operations, which reduces the size of the feature map; Residual Blocks: Residual blocks containing residual connections, which help network training and information transfer; Upsampling: Increase the size of the feature map through upsampling operation; Convolutional Layers: Another set of convolutional layers to further process the upsampled features; Output: The output of the network is the predicted pose key points and segmentation results, which depends on the task of the network; The key feature of the Hourglass network structure is to retain high-level semantic information while reducing the size, and then gradually restore details through upsampling; this structure enables the network to focus on both global and local information when processing image tasks; Output the detection box position, including: A regression method is used to predict the center point of each target in the hip joint X-ray image; For each spatial location, the head outputs a score indicating whether the location contains the target center; Predict the size of the target through the head, including the width and height of the target box; these predictions are relative offsets from the center point; Specifically, center point prediction: regression is used to predict the center point of each object in the image; for each spatial position, the head outputs a score indicating whether the position contains the center of the object; Target size prediction: The size information of the target is predicted by the head, which is the width and height of the target box; these predictions are relative offsets from the center point; Output lesion identification results, including: The possible target categories at each center point are predicted through the head output; Probability distribution calculated by the softmax function; Based on the probability distribution, output the lesion recognition results; Specifically, category prediction: The target category at each center point is predicted through the head output; the probability distribution is calculated by the softmax function; Output heatmap, including: Extract the feature map through the detection frame; Generate a visual heat map through the sigmoid activation function; Specifically, heat map Mask generation: The head outputs a heat map of the center point, where the value at each position indicates whether the position contains the target center; this heat map is processed by the sigmoid activation function so that its value is between 0 and 1; Specific method of generating heat map: In CNN networks, a sigmoid activation function is added after the feature map to convert it into a value in the range [0, 1], thereby generating a heat map. This technique is used to visualize the network's attention to the input, that is, the area that the network focuses on when processing the input image. Head output mode: There are three heads in total, one for target object segmentation, one for target object classification, and one for target object coordinate positioning; The details of the three heads are as follows: Each head has its specific output, which is compared with the corresponding target by properly designing the loss function, thus being jointly trained. This multi-head structure allows the network to handle segmentation, regression, and classification tasks simultaneously, improving the versatility of the model. The network structure used to focus on both global and local information when processing image tasks is the UNet network structure, which includes: One downsampling and one upsampling enable the network to capture the global and local features of the image while retaining high-resolution information; Determine the loss functions of the detection box position, lesion recognition results, and heat map, as well as the corresponding weights; Based on the three loss functions and their corresponding weights, calculate the overall loss function; Specifically, the following is the general form of the total loss function of CenterNet: in: is the center point prediction loss; is the target box size loss; is the class loss; is the mask prediction loss; is the weight hyperparameter of the corresponding loss term, which is used to balance the contribution of each loss; Among them, here, is the total number of prediction boxes, is a binary label of whether the actual center point contains the target, is the model's prediction score for the center point, are the coordinates of the actual target box, is the model's prediction of the target box, w is the width of the prediction box, h is the height of the prediction box; C is the number of categories.

2. A hip joint lesion recognition device based on deep learning algorithm, characterized in that: The device comprises: An image acquisition module, used for acquiring an X-ray image of a hip joint; The lesion recognition module is used to input the hip joint X-ray image into the preset hip joint lesion recognition model, and output the detection frame position, lesion recognition result and thermal map; Among them, the hip joint lesion recognition model includes a network structure for simultaneously focusing on global and local information when processing image tasks; First, the feature map of the X-ray image is extracted through several hourglass blocks; then three output heads are given; the first output head, imitating the head output method of the CenterNet model, outputs the center point, width and height of the hip joint position to be detected; the second branch is the multi-label classification label, and the multi-label lesion classification is performed through the classification head, and a total of 9 labels are output: Incomplete femoral head, femoral head collapse, gap changes, cystic changes, femoral head dislocation, osteophytes, cartilage sclerosis, sacroiliac joint fusion, insect bite changes; The descriptions of the above 9 tags are as follows: Incomplete femoral head: Description: The contour of the femoral head is incomplete or missing; Femoral head collapse: Description: Subsidence or collapse of the femoral head surface; Gap Change: Description: Changes in the joint space in the hip joint, which may narrow or widen; Cystic changes: Description: Fluid cysts form around joints; Femoral head dislocation: Description: The femoral head is not in its normal position and is out of the acetabulum; Osteophytes: Description: Extra bone tissue growth on the surface of bones; Cartilage sclerosis: Description: The cartilage in the joints becomes hardened and loses its elasticity; Sacroiliac joint fusion: Description: Fusion of the joints between the sacrum and ilium, loss of normal joint movement; Insect bite changes: Description: Bone tissue shows insect-like changes, usually small bone defects; The third branch is the heat map label of the hip joint. Since the feature map extracted by Hourglass has the property of a heat map, this property is reused in the model. After extracting the feature map of the hip joint, a heat map of the hip joint is generated through a sigmoid operation, which is more conducive to visualization. The network structure used to focus on both global and local information when processing image tasks is the Hourglass network structure, which includes: Input layer, used to input hip joint X-ray images; The first convolutional layer is used to extract the feature map of the hip joint X-ray image; Downsampling layer, used to reduce the size of feature maps; Residual blocks with residual connections for network training and information transfer; Upsampling layer, used to increase the size of the feature map; The second convolutional layer is used to extract the upsampled feature map; Output layer, used to output results; Specifically, Hourglass is usually used to describe a type of network structure that is symmetrical, decreasing and increasing layer by layer, and is usually used for tasks in image processing; Hourglass network structure description: Input: input image or feature map; Convolutional Layers: A series of convolutional layers used to extract features of the image; Downsampling: Downsampling achieved through pooling or convolution operations, which reduces the size of the feature map; Residual Blocks: Residual blocks containing residual connections, which help network training and information transfer; Upsampling: Increase the size of the feature map through upsampling operation; Convolutional Layers: Another set of convolutional layers to further process the upsampled features; Output: The output of the network is the predicted pose key points and segmentation results, which depends on the task of the network; The key feature of the Hourglass network structure is to retain high-level semantic information while reducing the size, and then gradually restore details through upsampling; this structure enables the network to focus on both global and local information when processing image tasks; Output the detection box position, including: A regression method is used to predict the center point of each target in the hip joint X-ray image; For each spatial location, the head outputs a score indicating whether the location contains the target center; Predict the size of the target through the head, including the width and height of the target box; these predictions are relative offsets from the center point; Specifically, center point prediction: regression is used to predict the center point of each object in the image; for each spatial position, the head outputs a score indicating whether the position contains the center of the object; Target size prediction: The size information of the target is predicted by the head, which is the width and height of the target box; these predictions are relative offsets from the center point; Output lesion identification results, including: The possible target categories at each center point are predicted through the head output; Probability distribution calculated by the softmax function; Based on the probability distribution, output the lesion recognition results; Specifically, category prediction: The target category at each center point is predicted through the head output; the probability distribution is calculated by the softmax function; Output heatmap, including: Extract the feature map through the detection frame; Generate a visual heat map through the sigmoid activation function; Specifically, heat map Mask generation: The head outputs a heat map of the center point, where the value at each position indicates whether the position contains the target center; this heat map is processed by the sigmoid activation function so that its value is between 0 and 1; Specific method of generating heat map: In CNN networks, a sigmoid activation function is added after the feature map to convert it into a value in the range [0, 1], thereby generating a heat map. This technique is used to visualize the network's attention to the input, that is, the area that the network focuses on when processing the input image. Head output mode: There are three heads in total, one for target object segmentation, one for target object classification, and one for target object coordinate positioning; The details of the three heads are as follows: Each head has its specific output, which is compared with the corresponding target by properly designing the loss function, thus being jointly trained. This multi-head structure allows the network to handle segmentation, regression, and classification tasks simultaneously, improving the versatility of the model. The network structure used to focus on both global and local information when processing image tasks is the UNet network structure, which includes: One downsampling and one upsampling enable the network to capture the global and local features of the image while retaining high-resolution information; Determine the loss functions of the detection box position, lesion recognition results, and heat map, as well as the corresponding weights; Based on the three loss functions and their corresponding weights, calculate the overall loss function; Specifically, the following is the general form of the total loss function of CenterNet: in: is the center point prediction loss; is the target box size loss; is the class loss; is the mask prediction loss; is the weight hyperparameter of the corresponding loss term, which is used to balance the contribution of each loss; Among them, here, is the total number of prediction boxes, is a binary label of whether the actual center point contains the target, is the model's prediction score for the center point, are the coordinates of the actual target box, is the model's prediction of the target box, w is the width of the prediction box, h is the height of the prediction box; C is the number of categories.

3. An electronic device, characterized in that: The electronic device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the hip joint lesion identification method based on deep learning algorithm as described in claim 1.

4. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the hip joint lesion identification method based on a deep learning algorithm as described in claim 1.

Images