A method and system for counting orthopedic lesions based on detection guidance

Abstract

The present invention belongs to the field of artificial intelligence technology and discloses a detection-guided orthopedic lesion counting method and system. The method extracts edge features of an image as the fourth channel of network input, performing lesion detection and counting tasks in parallel. The detection task generates a suspected target area, and the counting task generates a characteristic value heat map. The heat map values ​​are summed within the detection area, and the count value of the detected category targets is increased based on the product of the sum of the heat maps within the area and the detection score. The present invention extracts the bounding box of the target through the detection network, counts the number of targets within a more precise range, and significantly improves the effective utilization of image features in specific scenarios, making counting more accurate.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and in particular relates to an orthopedic lesion counting method and system based on detection guidance. Background Art

[0002] Currently, medical imaging technologies such as CT, MRI, PET, ultrasound, and X-rays play a vital role in the early detection, diagnosis, and treatment of diseases. In recent years, with the continuous development of deep learning technology, computer-assisted diagnosis technology has also been continuously upgraded. However, due to the large errors in current counting networks, the task of accurately counting lesions is still performed by professional doctors.

[0003] Currently, conventional counting networks regress the number of targets by outputting target density maps, which lack categorical information and lead to confusion in lesion counts. Commonly used detection networks generate bounding boxes around targets with categorical information. However, lesions are small and dense, making detection difficult and leading to a high rate of missed detections.

[0004] Through the above analysis, the problems and defects of the existing technology are as follows:

[0005] (1) Conventional counting networks have no category information, which leads to confusion in counting between lesions and large errors.

[0006] (2) The commonly used detection network has small lesion images, high density, high detection difficulty, and high frequency of missed detection.

[0007] The difficulty of solving the above problems and defects is:

[0008] Counting networks typically use heatmap regression to regress the number of targets. Commonly used Gaussian function-based labeling methods lack categorical information. Furthermore, lesions are prone to mutual occlusion, leading to confusion in lesion counting.

[0009] The detection network uses a unified label matching scheme to match target labels. Samples with an intersection-over-union ratio (the ratio of the intersection and union of the recommended area and the enclosing area of ​​the true value) greater than a fixed threshold (usually set to 0.7) are considered positive samples, while samples with an intersection-over-union ratio less than a fixed threshold (usually set to 0.3) are considered negative samples. Based on the above mechanism, lesions with different imaging areas will have different impacts. For lesions with small imaging and high density, there are fewer matching positive samples, resulting in an imbalance between positive and negative samples, which damages network performance and causes a high frequency of missed detections.

[0010] The significance of solving the above problems and defects is:

[0011] The detection algorithm detects rough areas and counts within precise areas, suppressing interference from irrelevant information. This improves counting accuracy while also identifying the correct category. This improves lesion counting accuracy, assisting doctors in providing precise counting and identification results.

[0012] For junior doctors, resolving these issues can help improve their accuracy in reading images. A doctor's reading speed is 10-20 seconds per image, and after 10 hours of continuous work, their reading efficiency decreases by 30%. For senior doctors, resolving these issues can improve their reading efficiency and alleviate pressure on medical resources. Summary of the Invention

[0013] In response to the problems existing in the prior art, the present invention provides an orthopedic lesion counting method and system based on detection guidance.

[0014] A method for counting orthopedic lesions based on detection guidance includes:

[0015] Step 1: Extract the edge features of the image as the fourth channel of the network input, and perform lesion detection and counting tasks in parallel;

[0016] Step 2: The detection task generates suspected target areas, and the counting task generates feature value heat maps;

[0017] Step 3: Calculate the sum of the eigenvalues ​​in the detection area, and increase the detection category target count value according to the product of the sum of the eigenvalues ​​in the area and the detection score.

[0018] Furthermore, the step 1 specifically includes:

[0019] Use the Laplace function to process the original image to obtain the edge imaging of the image, which is spliced ​​together with the original image (RGB image) as the fourth channel and recorded as M_1;

[0020] Send M_1 as the initial input of the network to the feature extraction network f_1;

[0021] The extracted features are fed into the detection network f_2 and the counting network f_3 in parallel.

[0022] Furthermore, the step 2 specifically includes:

[0023] The detection network f_2 regresses the target bounding box d and the target category score s, and the counting network f_3 regresses the image heat map m_f;

[0024] During training, the counting network f_3 and the detection network f_2 jointly optimize the feature extraction network f_1;

[0025] During inference, the image is input into the feature extraction network f_1 and passes through the object detection network f_2 and the counting network f_3 in parallel;

[0026] Extract the target detection results greater than a given threshold and the heat map m_f generated by the counting module.

[0027] Furthermore, the step three specifically includes:

[0028] In the heat map m_f generated by the counting module, calculate the sum of the feature values ​​s_d in the target detection bounding box d;

[0029] The product of the sum of the feature values ​​s_d and the detection category score s is taken as the counting result, where the counting category is the detection category.

[0030] Another object of the present invention is to provide an orthopedic lesion counting system based on detection guidance, the orthopedic lesion counting system based on detection guidance comprising:

[0031] The edge extraction module is used to extract image edge features through the Laplace operator; the original image is input, processed by the Laplace algorithm, and the edge information image is output.

[0032] The feature extraction module is used to extract the edge features of the image through the feature extraction network; splice the RGB image with the edge information image, and output the Robin features based on the swin-transformer as the backbone network.

[0033] The detection module is used to detect the extracted edge features through the detection network and output the detection results based on the Retinenet detector.

[0034] The counting module is used to count the extracted edge features through the counting network; the heat map of the counts is regressed through the fully connected layer for the Robin features.

[0035] The fusion module counts the values ​​of the heat map output by the counting module in the suspected area output by the detection module to obtain the counting result and detection category.

[0036] Combining all the above technical solutions, the advantages and positive effects of the present invention are as follows:

[0037] The present invention extracts the bounding box of the target through the detection network, counts the number of targets in a more accurate range, greatly improves the effective utilization of image features in specific scenarios, and makes the counting more accurate. Figure 6 As shown in the lesion detection comparison chart, the box is the detection result, and the Gaussian distribution of the regression heat map is inside the box. By counting in the detection box area, it can be clearly counted that there are 9 lesions in the picture, and the category is enchondroma, such as Figure 6 (b) shows that when the conventional counting algorithm is applied to 6(a), the regressed count value is 7, and there is no clear type and location. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0039] Figure 1 This is a flow chart of a method for counting orthopedic lesions based on detection guidance provided by an embodiment of the present invention.

[0040] Figure 2 This is a flowchart of step S101 of the orthopedic lesion counting method based on detection guidance provided by an embodiment of the present invention.

[0041] Figure 3 This is a flowchart of step S102 of the orthopedic lesion counting method based on detection guidance provided by an embodiment of the present invention.

[0042] Figure 4 This is a flowchart of step S103 of the orthopedic lesion counting method based on detection guidance provided by an embodiment of the present invention.

[0043] Figure 5 Schematic diagram of a neural network structure provided by an embodiment of the present invention.

[0044] In the figure: 1. Original image; 2. Edge image extracted from the original image; 3. Detection box and target category score output by the detection network; 4. Heat map output by the counting network; 5. Statistical heat map feature values ​​within the detection box; M1, network input of the combination of original image and edge imaging; f1, feature extraction network; f2, detection network; f3, counting network.

[0045] Figure 6 This is a comparative diagram of lesion detection provided by an embodiment of the present invention. Figure 6 (a) Using the conventional counting algorithm, the regressed count value is 7, and there is no clear type and location; Figure 6 (b) shows the Gaussian distribution of the regression heat map within the frame. By counting within the detection frame area, it can be clearly counted that there are 9 lesions in the image, which are classified as enchondroma. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0047] In view of the problems existing in the prior art, the present invention provides an orthopedic lesion counting method and system based on detection guidance. The present invention is described in detail below with reference to the accompanying drawings.

[0048] like Figure 1 As shown, the orthopedic lesion counting method based on detection guidance provided by the embodiment of the present invention includes:

[0049] S101, extracting edge features of the image as the fourth channel of the network input, and performing lesion detection and counting tasks in parallel;

[0050] S102, the detection task generates a suspected target area, and the counting task generates a feature value heat map;

[0051] S103, calculating the sum of the eigenvalues ​​in the detection area, and increasing the detection category target count value according to the product of the sum of the eigenvalues ​​in the area and the detection score.

[0052] like Figure 2 As shown, step S101 in the embodiment of the present invention specifically includes:

[0053] S201, using the Laplace function to process the original image to obtain image edge imaging, which is spliced ​​together with the original image as the fourth channel, denoted as M_1;

[0054] S202, sending M_1 as the initial input of the network to the feature extraction network f_1;

[0055] S203, the extracted features are sent to the detection network f_2 and the counting network f_3 in parallel.

[0056] like Figure 3 As shown, step S102 in the embodiment of the present invention specifically includes:

[0057] S301, the detection network f_2 regresses the target bounding box d and the target category score s, and the counting network f_3 regresses the image heat map m_f;

[0058] S302, during training, the counting network f_3 and the detection network f_2 jointly optimize the feature extraction network f_1;

[0059] S303, during inference, the image is input into the feature extraction network f_1 and passes through the object detection network f_2 and the counting network f_3 in parallel;

[0060] S304: Extract target detection results greater than a given threshold and the heat map m_f generated by the counting module.

[0061] like Figure 4 As shown, step S103 in the embodiment of the present invention specifically includes:

[0062] S401, in the heat map m_f generated by the counting module, calculate the sum s_d of the feature values ​​in the target detection bounding box d;

[0063] S402 : The product of the sum of the eigenvalues ​​s_d and the detection category score s is taken as a counting result, where the counting category is the detection category.

[0064] The present invention will be further described below with reference to specific embodiments.

[0065] like Figure 5 As shown in the figure, first, the input image of size (W_0, H_0) is processed using the Laplace function to obtain the edge image 2, which is spliced ​​with the original image as the fourth channel and recorded as M_1. M_1 is sent as the initial input to the feature extraction network f_1. The extracted features are then fed into the detection network f_2 and the counting network f_3 in parallel.

[0066] The detection network f_2 regresses 3, the target bounding box d and the target category score s; the counting network f_3 regresses 4, the image heat map m_f.

[0067] During training, the counting network f_3 and the detection network f_2 jointly optimize the feature extraction network f_1. During inference, the image is fed into the feature extraction network f_1 and then passes through the object detection network f_2 and the counting network f_3 in parallel. Object detection results above a given threshold are extracted, along with the heatmap m_f generated by the counting module.

[0068] In the heatmap m_f generated by the counting module, the sum of the eigenvalues ​​s_d within the object detection bounding box d is calculated. As shown in Figure 5, the product of the sum of the eigenvalues ​​s_d (1.4 in this example) and the detection category score s (0.80 in this example) is used as the counting result (1.12 in this example). The counting category is the detection category. The counting result is rounded to an integer in the final output.

[0069] The advantages of the present invention are that the designed counting network process improves the authenticity of the counting network results, and the method is simple, easy to operate and has strong applicability.

[0070] like Figure 6 As shown in the lesion detection comparison chart, the box is the detection result, and the Gaussian distribution of the regression heat map is inside the box. By counting in the detection box area, it can be clearly counted that there are 9 lesions in the picture, and the category is enchondroma, such as Figure 6 (b) shows that when the conventional counting algorithm is applied to 6(a), the regressed count value is 7, and there is no clear type and location.

[0071] According to practical statistics, the accuracy rate of junior doctors in reading films is 85%. Based on the auxiliary reading of the invention, the counting accuracy rate is improved by 11.5%.

[0072] Doctors can read films in 10 to 20 seconds per film. With the aid of the present invention, doctors' work efficiency is increased by 33%.

[0073] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware portion can be implemented using dedicated logic; the software portion can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. Those skilled in the art will appreciate that the above-mentioned devices and methods can be implemented using computer-executable instructions and / or contained in processor control code, for example, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, or can be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0074] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with this technical field within the technical scope disclosed by the present invention and within the spirit and principles of the present invention should be covered by the scope of protection of the present invention.

Claims

1. A method for counting orthopedic lesions based on detection guidance, comprising: The orthopedic lesion counting method based on detection guidance includes: Step 1: Extract the edge features of the image as the fourth channel of the network input, and perform lesion detection and counting tasks in parallel; Step 2: The detection task generates suspected target areas, and the counting task generates feature value heat maps; Step 3: Calculate the sum of the eigenvalues ​​in the detection area, and obtain the counting result based on the product of the sum of the eigenvalues ​​in the area and the detection score; The second step specifically includes: The detection network f_2 regresses the target bounding box d and the target category score s, and the counting network f_3 regresses the image heat map m_f; During training, the detection network f_2 and the counting network f_3 jointly optimize the feature extraction network f_1; During inference, the image is input into the feature extraction network f_1 and passes through the detection network f_2 and the counting network f_3 in parallel; Extract the target detection results greater than a given threshold and the heat map m_f generated by the counting module; The step three specifically includes: In the heat map m_f generated by the counting module, calculate the sum of the feature values ​​s_d in the target detection bounding box d; The product of the sum of the feature values ​​s_d and the detection category score s is taken as the counting result.

2. The orthopedic lesion counting method based on detection guidance according to claim 1, characterized in that: The step 1 specifically includes: Use the Laplace function to process the original image to obtain the edge imaging of the image, which is spliced ​​together with the original image as the fourth channel and recorded as M_1; Send M_1 as the initial input of the network to the feature extraction network f_1; The extracted features are fed into the detection network f_2 and the counting network f_3 in parallel.

3. A detection-guided orthopedic lesion counting system for implementing the detection-guided orthopedic lesion counting method according to any one of claims 1 to 2, characterized in that: The orthopedic lesion counting system based on detection guidance includes: Edge extraction module, used to extract image edge features through Laplacian operator; A feature extraction module is used to extract edge features of an image through a feature extraction network; The detection module is used to detect the extracted features through the detection network; The counting module is used to generate a counting heat map for the extracted features through the counting network; Fusion module for counting in the detection area.

4. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the orthopedic lesion counting method based on detection guidance as described in any one of claims 1 to 2.

5. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the orthopedic lesion counting method based on detection guidance according to any one of claims 1 to 2.

6. An information data processing terminal, characterized in that: The information data processing terminal is used to implement the orthopedic lesion counting method based on detection guidance as described in any one of claims 1 to 2.

Images