A key point marking method based on bone age detection and related device

Abstract

The embodiment of the present application provides a key point labeling method based on bone age detection and a related device. Among them, a key point labeling method based on bone age detection includes: obtaining a target DR image; labeling each bone of the m bones in the target DR image, and obtaining the epiphyseal point corresponding to each bone in the m bones; connecting the center point of the line between the epiphyseal point corresponding to the ulna bone and the epiphyseal point corresponding to the radius bone with the epiphyseal points corresponding to the n bones respectively, and determining the vertical line perpendicular to the line between the center points corresponding to each epiphyseal point in the n epiphyseal points; in the finger area corresponding to the i-th bone, according to the vertical line corresponding to the i-th bone, calculate the center of mass corresponding to the i-th bone; if the center of mass coincides with the epiphyseal point corresponding to the i-th bone, then update the epiphyseal point corresponding to the i-th bone to the key point of the i-th bone. The embodiment of the present application can improve the accuracy of key point labeling when detecting bone age.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular, to a key point annotation method and related device based on bone age detection. Background Art

[0002] Key point detection is a hot research issue in computer vision and is widely used in various fields of computer vision, such as object detection methods based on key point detection, object orientation positioning, human action classification, abnormal behavior detection, autonomous driving, medical images, etc.

[0003] In the existing deep learning-based key point detection methods, the annotation of key points requires a large amount of manual annotation. The accuracy of manual annotation directly affects the accuracy of the detection method. However, due to the different feelings and understandings of each person for images, it is very time-consuming and laborious to obtain a dataset with high annotation quality. In addition, with the development of technology, the models for key point detection are becoming more and more complex, with more and more parameters, and the requirements for hardware are also increasing accordingly, thus increasing the application cost of the system.

[0004] Therefore, how to improve the accuracy of key point annotation is an urgent problem to be solved. Summary of the Invention

[0005] In view of the above problems, this application is proposed to provide a key point annotation method and related device based on bone age detection that can overcome the above problems or at least partially solve the above problems.

[0006] In a first aspect, an embodiment of this application provides a key point annotation method based on bone age detection, which is applied to a first device and may include:

[0007] Obtain a target digital X-ray DR image, where the target DR image includes a bone image of a hand;

[0008] Annotate each of the m bones in the target DR image to obtain an epiphyseal point corresponding to each of the m bones. The m bones include n bones, as well as the ulna bone and the radius bone. m is a positive integer greater than 2, and n is a positive integer less than m or greater than 1;

[0009] Connect the center point between the epiphyseal points corresponding to the ulna bone and the epiphyseal points corresponding to the radius bone with the epiphyseal points corresponding to the n bones respectively, and determine a perpendicular line perpendicular to the connection line between each epiphyseal point corresponding to the n epiphyseal points and the center point;

[0010] In the finger region corresponding to the i-th bone, calculate the centroid corresponding to the i-th bone according to the perpendicular line corresponding to the i-th bone, where i = 1, 2... n;

[0011] If the centroid coincides with the epiphyseal point corresponding to the ith bone, update the epiphyseal point corresponding to the ith bone to the key point of the ith bone.

[0012] Through the method of the first aspect, the embodiments of the present application describe a method for improving the detection accuracy of epiphyseal key points - centroid iteration method based on the background of bone age prediction. Through this method, the embodiments of the present application can determine whether the centroid corresponding to the bone coincides with the epiphyseal point corresponding to the bone. If the centroid and the epiphyseal point coincide, it proves that the epiphyseal point can be marked as the key point of the bone; if they do not coincide, continue to screen out the key points until they coincide with the centroid. This method can improve the quality of manual marking and improve the detection accuracy of the model key points according to prior knowledge (the images of the epiphyseal part of the fingers are density-related and symmetric left and right) during prediction. A simple and lightweight model can achieve good results, improve the speed of the overall bone age prediction process, reduce the requirements of the key point detection method for hardware, and save the application costs of various key point detection systems.

[0013] In a possible implementation manner, the method further includes: if the centroid corresponding to the ith bone does not coincide with the epiphyseal point corresponding to the ith bone, update the average value of the epiphyseal point and the centroid to the epiphyseal point of the ith bone; according to the updated epiphyseal point of the ith bone, re-determine the key point corresponding to the ith bone until the updated epiphyseal point of the ith bone coincides with the centroid corresponding to the ith bone.

[0014] In a possible implementation manner, the obtaining of the digital X-ray DR image of the hand includes: obtaining a first DR image; processing the first DR image to remove the non-hand area part of the first DR image to obtain the target DR image.

[0015] In a possible implementation manner, the method further includes: obtaining the bone information of the ith bone, where the bone information includes the length and / or width of the ith bone; based on the bone information of the ith bone, extracting the epiphyseal region image of the ith bone from the target DR image according to the key point of the ith bone and the perpendicular line corresponding to the ith bone; predicting the bone age of the hand bone of the target DR image according to the extracted multiple ROI images of the epiphyseal region of interest.

[0016] In a possible implementation manner, obtaining the bone information of the \(i\)th bone includes: obtaining the minimum bounding rectangle in the target DR image according to the perpendicular line corresponding to the \(i\)th bone; calculating the width-to-height ratio of the target DR image according to the minimum bounding rectangle in the target DR image and the minimum bounding rectangle of the pre-stored standard hand; and calculating the bone information corresponding to each of the \(m\) bones according to the width-to-height ratio of the target DR image and the pre-stored bone information of the standard hand.

[0017] In a possible implementation manner, the epiphyseal point corresponding to the ulna bone is located at the center point of the epiphysis corresponding to the ulna bone, and the epiphyseal point corresponding to the radius bone is located at the center point of the epiphysis corresponding to the radius bone.

[0018] In a second aspect, an embodiment of the present application provides a key point annotation device based on bone age detection, which may include:

[0019] A first acquisition unit, configured to acquire a target digital X-ray DR image, where the target DR image includes a bone image of a hand;

[0020] A marking unit, configured to mark each of the \(m\) bones in the target DR image to obtain the epiphyseal points corresponding to each of the \(m\) bones, where the \(m\) bones include \(n\) bones, an ulna bone, and a radius bone, \(m\) is a positive integer greater than 2, and \(n\) is a positive integer less than \(m\) or greater than 1;

[0021] A perpendicular line unit, configured to respectively connect the center point between the epiphyseal points corresponding to the ulna bone and the epiphyseal points corresponding to the radius bone with the epiphyseal points corresponding to the \(n\) bones, and determine a perpendicular line passing through each of the epiphyseal points corresponding to the \(n\) epiphyseal points and perpendicular to the connection line with the center point;

[0022] A centroid unit, configured to calculate the centroid corresponding to the \(i\)th bone in the finger area corresponding to the \(i\)th bone according to the perpendicular line corresponding to the \(i\)th bone, where \(i = 1, 2, \cdots, n\);

[0023] A first key point unit, configured to update the epiphyseal point corresponding to the \(i\)th bone to the key point of the \(i\)th bone if the centroid coincides with the epiphyseal point corresponding to the \(i\)th bone.

[0024] In a possible implementation manner, the device further includes: a second key point unit, configured to: if the centroid corresponding to the i-th bone does not coincide with the epiphyseal point corresponding to the i-th bone, update the average value of the epiphyseal point and the centroid as the epiphyseal point of the i-th bone; and re-determine the key points corresponding to the i-th bone according to the updated epiphyseal point of the i-th bone until the updated epiphyseal point of the i-th bone coincides with the centroid corresponding to the i-th bone.

[0025] In a possible implementation manner, the first acquisition unit is specifically configured to: acquire a first DR image in which the fingers of the hand point upward; and process the first DR image to remove the non-hand region part in the first DR image to obtain the target DR image.

[0026] In a possible implementation manner, the device further includes: a second acquisition unit, configured to acquire the bone information of the i-th bone, where the bone information includes the length and / or width of the i-th bone; an extraction unit, configured to extract the epiphyseal region image of the i-th bone from the target DR image based on the bone information of the i-th bone according to the key points of the i-th bone and the perpendicular line corresponding to the i-th bone; and a prediction unit, configured to predict the bone age of the hand bone in the target DR image according to the extracted multiple epiphyseal ROI images.

[0027] In a possible implementation manner, the second acquisition unit is specifically configured to: obtain the minimum circumscribed rectangle in the target DR image according to the perpendicular line corresponding to the i-th bone; calculate the width-to-height ratio of the target DR image according to the minimum circumscribed rectangle in the target DR image and the minimum circumscribed rectangle of the pre-stored standard hand; and calculate the bone information corresponding to each of the m bones according to the width-to-height ratio of the target DR image and the pre-stored bone information of the standard hand.

[0028] In a possible implementation manner, the epiphyseal point corresponding to the ulna bone is located at the center point of the epiphysis corresponding to the ulna bone, and the epiphyseal point corresponding to the radius bone is located at the center point of the epiphysis corresponding to the radius bone.

[0029] In a third aspect, an embodiment of the present application provides a key point annotation device for bone age detection, including a storage component, a processing component, and a communication component, where the storage component, the processing component, and the communication component are interconnected. The storage component is configured to store a computer program, and the communication component is configured to interact with an external device for information. The processing component is configured to call the computer program to execute the method described in the first aspect, which will not be elaborated here.

[0030] Fourthly, an embodiment of the present application provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method of the first aspect above. Description of the Drawings

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background art, the following will describe the drawings required to be used in the embodiments of the present application or the background art.

[0032] Figure 1 It is a schematic diagram of the architecture of a key point annotation system based on bone age detection provided by an embodiment of the present application.

[0033] Figure 2 It is a schematic diagram of the flow of a key point annotation method based on bone age detection provided by an embodiment of the present application.

[0034] Figure 3 It is a schematic diagram of a target DR image provided by an embodiment of the present application.

[0035] Figure 4 It is a schematic diagram of a DR image of the connection line and perpendicular line between the epiphyseal point and the center point provided by an embodiment of the present application.

[0036] Figure 5 It is a schematic diagram of the bone age detection process provided by an embodiment of the present application.

[0037] Figure 6 It is a schematic diagram of the structure of a key point annotation device based on bone age detection provided by an embodiment of the present application.

[0038] Figure 7 It is a schematic diagram of the structure of another key point annotation device based on bone age detection provided by an embodiment of the present application. Detailed Embodiments

[0039] The following will describe the embodiments of the present application in conjunction with the drawings in the embodiments of the present application.

[0040] The terms "first", "second", "third", etc. in the specification, claims and drawings of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products or devices.

[0041] References to "embodiments" in this application mean that the specific features, structures, or characteristics described in connection with the embodiments can be included in at least one embodiment of the present application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment each time, nor are they independent or alternative embodiments mutually exclusive of other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0042] As used in this application, the terms "second device", "unit", "system", etc. are used to denote computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, the second device can be, but is not limited to, a processor, a data processing platform, a computing device, a computer, two or more computers, etc.

[0043] First, some terms used in this application are explained to facilitate understanding by those skilled in the art.

[0044] (1) ROI (region of interest), the region of interest. In machine vision and image processing, the area to be processed is outlined in the form of a square, circle, ellipse, irregular polygon, etc. from the processed image, which is called the region of interest, ROI. Various operators and functions are commonly used in machine vision software such as Halcon, OpenCV, and Matlab to obtain the region of interest ROI and perform the next step of image processing.

[0045] (2) Cisco's Internetwork Operating System (IOS) is an operating system optimized for internetworking, a software architecture separated from hardware. With the continuous development of network technology, it can be dynamically upgraded to adapt to the ever-changing technology (hardware and software), and has modularity, flexibility, scalability, and controllability.

[0046] (3) Windows Phone (abbreviated as WP) is a mobile operating system officially released by Microsoft on October 21, 2010. Windows Phone has a series of avant-garde operating experiences such as desktop customization, icon dragging, and sliding control. Its home screen displays new emails, text messages, missed calls, calendar appointments, etc. by providing a dashboard-like experience. It also includes an enhanced touchscreen interface for more convenient finger operation.

[0047] (3) The DR system, namely the direct digital radiography system, is composed of an electronic cassette, a scanning controller, a system controller, an image monitor, etc. It directly converts X-ray photons into digital images through an electronic cassette, which is a direct digital radiography in a broad sense. The direct digital radiography in a narrow sense, namely DDR (Direct Digit Radiography), usually refers to digital radiography using the image direct conversion technology of flat panel detectors, which is a direct digital radiography system in the true sense. It is mainly divided into amorphous silicon flat panel DR (mainstream), amorphous selenium flat panel DR, and CCD DR (mainstream) according to the detector type; and into suspended DR and column (UC arm) DR according to the gantry structure.

[0048] Secondly, a description is given to one of the key point annotation system architectures based on bone age detection on which the embodiments of the present application are based. Please refer to the appendix Figure 1 , Figure 1 FIG. is a schematic diagram of a key point annotation system architecture based on bone age detection provided by an embodiment of the present application, including: a key point annotation device 001 based on bone age detection and a terminal device 002. Among them:

[0049] The key point annotation device 001 based on bone age detection may include but is not limited to a background server, a component server, a data processing server, etc. When the key point annotation device 001 based on bone age detection is a server, the server can communicate with multiple terminals through the Internet, and a corresponding server-side program also needs to run on the server to provide corresponding automatic document generation services, such as database services, data calculation, decision execution, etc. For example, the server can obtain a target digital X-ray DR image, where the target DR image includes a hand bone image; annotate each of the m bones in the target DR image to obtain the epiphyseal points corresponding to each of the m bones. The m bones include n bones, the ulna bone, and the radius bone, m is a positive integer greater than 2, and n is a positive integer less than m or greater than 1; connect the center point between the epiphyseal points corresponding to the ulna bone and the epiphyseal points corresponding to the radius bone with the epiphyseal points corresponding to the n bones respectively, and determine the perpendicular lines perpendicular to the connection lines between the center point and each of the n epiphyseal points; in the finger area corresponding to the i-th bone, calculate the centroid corresponding to the i-th bone according to the perpendicular line corresponding to the i-th bone, i = 1, 2... n; if the centroid coincides with the epiphyseal point corresponding to the i-th bone, update the epiphyseal point corresponding to the i-th bone to the key point of the i-th bone.

[0050] Optionally, when the centroid corresponding to the \(i\)th bone does not coincide with the epiphyseal point corresponding to the \(i\)th bone, the key point annotation device 001 based on bone age detection may also update the average value of the epiphyseal point and the centroid as the epiphyseal point of the \(i\)th bone; according to the updated epiphyseal point of the \(i\)th bone, re-determine the key points corresponding to the \(i\)th bone until the updated epiphyseal point of the \(i\)th bone coincides with the centroid corresponding to the \(i\)th bone.

[0051] The terminal device 002 can install and run relevant applications. An application refers to a program that corresponds to the server and provides local services for customers. Here, the local services may include but are not limited to: displaying the target DR image and sending the target DR image information to the server. The terminal in the embodiments of this solution may include but are not limited to any electronic product based on an intelligent operating system, which can interact with users through input devices such as keyboards, virtual keyboards, touch pads, touch screens, and voice control devices, such as smart phones, tablet computers, personal computers, etc. Among them, the intelligent operating system includes but is not limited to any operating system that enriches the functions of the device by providing various mobile applications, such as: Android TM )、iOS TM 、Windows Phone TM and so on.

[0052] It can also be understood that Figure 1 the key point annotation system architecture based on bone age detection is only a partial exemplary implementation manner in the embodiments of this application. The key point annotation system architecture based on bone age detection in the embodiments of this application includes but is not limited to the above key point annotation system architecture based on bone age detection.

[0053] Refer to the appendix Figure 2 , Figure 2 which is a schematic diagram of the key point annotation method flow provided by the embodiments of this application. It can be applied to the system in the above Figure 1 . The following will describe the interaction between the key point annotation device 001 based on bone age detection and the terminal device 002 in combination with Figure 2 . Among them, the method may include the following steps S201 - step S208.

[0054] Step S201, obtain the first DR image.

[0055] Specifically, obtain the first DR image. Among them, the shape of the hand in the first DR image can be with fingers spread upward or downward. For example: The key point annotation device for bone age detection can use a trained model to detect the hand area in the bone age DR image and select a DR image with a better hand orientation. Or directly receive a DR image of a hand with fingers spread and upward. Among them, the DR image is a hand bone photo taken by a DR system or obtained through a terminal device 002.

[0056] Step S202: Process the first DR image to remove the non-hand area part in the first DR image to obtain a target DR image.

[0057] Specifically, the key point annotation device for bone age detection can obtain a target digital X-ray DR image, where the target DR image includes the bone image of the hand. The key point annotation device for bone age detection can, according to the first DR image, remove the non-hand area part in the bone age DR image to obtain a target DR image.

[0058] Step S203: Annotate each of the m bones in the target DR image to obtain the epiphyseal points corresponding to each of the m bones.

[0059] Specifically, the key point annotation device for bone age detection can perform key point annotation of the epiphyseal position on the bone age DR image, annotate each of the m bones in the target DR image to obtain the epiphyseal points corresponding to each of the m bones. The m bones include n bones, the ulna bone, and the radius bone. m is a positive integer greater than 2, and n is a positive integer less than m or greater than 1. Among them, the n bones in the target DR image can include one or more of multiple phalanges, multiple metacarpal bones, and multiple carpal bones. It can be understood that, please refer to the appendix Figure 3 , Figure 3 is a schematic diagram of a target DR image provided by an embodiment of the present application. As Figure 3 shown, when annotating each of the m bones in the target DR image, the epiphyseal points can be annotated at the center point of the epiphysis corresponding to each bone, the top center point, etc.

[0060] In a possible implementation manner, the epiphyseal point corresponding to the ulna bone is located at the center point of the epiphysis corresponding to the ulna bone, and the epiphyseal point corresponding to the radius bone is located at the center point of the epiphysis corresponding to the radius bone. The key point annotation device for bone age detection can annotate the epiphyseal point corresponding to the ulna bone at the center point of the epiphysis corresponding to the ulna bone; and annotate the epiphyseal point corresponding to the radius bone at the center point of the epiphysis corresponding to the radius bone.

[0061] Step S204: Connect the center point of the line between the epiphyseal points corresponding to the ulna bone and the epiphyseal points corresponding to the radius bone to the epiphyseal points corresponding to the n bones respectively, and determine the perpendicular lines passing through each of the n epiphyseal points and perpendicular to the lines connecting them to the center point.

[0062] Specifically, the key point annotation device for bone age detection can connect the center point of the line between the epiphyseal points corresponding to the ulna bone and the epiphyseal points corresponding to the radius bone to the epiphyseal points corresponding to the n bones respectively, and determine the perpendicular lines passing through each of the n epiphyseal points and perpendicular to the lines connecting them to the center point. Please refer to the appendix Figure 4 , Figure 4 FIG. is a schematic DR image of the lines and perpendicular lines between the epiphyseal points and the center point provided by an embodiment of the present application. As Figure 4 shown, connect the epiphyseal points corresponding to the ulna bone and the epiphyseal points corresponding to the radius bone to obtain a first connecting line; take the midpoint of the first connecting line and connect it to the epiphyseal point of one of the metacarpal bones among the n epiphyseal points to obtain a second connecting line; then draw a line perpendicular to the second connecting line through the epiphyseal point of this metacarpal bone, and the line segment of this line on this metacarpal bone is the perpendicular line corresponding to this metacarpal bone.

[0063] Step S205: In the finger region corresponding to the i-th bone, calculate the centroid corresponding to the i-th bone according to the perpendicular line corresponding to the i-th bone.

[0064] Specifically, the key point annotation device for bone age detection can calculate the centroid corresponding to the i-th bone in the finger region corresponding to the i-th bone according to the perpendicular line corresponding to the i-th bone, where i = 1, 2... n. It can be understood that in the bone age DR image, at each epiphysis, the density of the bone is greater than that of the meat. Therefore, the center point of the finger epiphysis should be the centroid in the perpendicular line direction, and in this bone region, calculate the centroid of all points on the perpendicular line along the perpendicular line direction. It can also be understood that the i-th bone is any one of the n bones.

[0065] Step S206: If the centroid coincides with the epiphyseal point corresponding to the i-th bone, update the epiphyseal point corresponding to the i-th bone to the key point of the i-th bone.

[0066] Specifically, if the centroid coincides with the epiphyseal point corresponding to the i-th bone, the key point annotation device for bone age detection can update the epiphyseal point corresponding to the i-th bone to the key point of the i-th bone. It can be understood that the key point annotation device for bone age detection can sequentially determine the key points corresponding to the n bones. If the centroid coincides with the epiphyseal point annotation, it is considered that the epiphyseal point is accurately marked as the key point.

[0067] Step S207, if the centroid corresponding to the i-th bone does not coincide with the epiphyseal point corresponding to the i-th bone, update the average value of the epiphyseal point and the centroid as the epiphyseal point of the i-th bone.

[0068] Specifically, if the centroid corresponding to the i-th bone does not coincide with the epiphyseal point corresponding to the i-th bone, the key point annotation device for bone age detection can update the average value of the epiphyseal point and the centroid as the epiphyseal point of the i-th bone. It can be understood that if the centroid does not coincide with the epiphyseal point annotation, it is considered that the epiphyseal point is inaccurately marked as a key point, and the average value of the epiphyseal point and the centroid needs to be calculated as the new epiphyseal point until the updated epiphyseal point of the i-th bone coincides with the centroid corresponding to the i-th bone.

[0069] Step S208, re-determine the key points corresponding to the i-th bone according to the updated epiphyseal point of the i-th bone.

[0070] Specifically, the key point annotation device for bone age detection can re-determine the key points corresponding to the i-th bone according to the updated epiphyseal point of the i-th bone until the updated epiphyseal point of the i-th bone coincides with the centroid corresponding to the i-th bone. It can be understood that after updating the epiphyseal point, reconnect the updated epiphyseal point to the midpoint of the above first connection line to obtain an updated second connection line; then draw a line perpendicular to the updated second connection line through the updated epiphyseal point to obtain an updated perpendicular line. Finally, determine the new centroid according to the updated perpendicular line until the epiphyseal point coincides with the centroid, so as to determine the key points.

[0071] In a possible implementation manner, the method further includes: obtaining the bone information of the i-th bone, where the bone information includes the length and / or width of the i-th bone; based on the bone information of the i-th bone, extracting the epiphyseal region image of the i-th bone from the target DR image according to the key points of the i-th bone and the perpendicular line corresponding to the i-th bone; predicting the bone age of the hand bones in the target DR image according to the extracted multiple ROI images of the epiphysis of interest. The key point annotation device for bone age detection can send the extracted ROI into a trained epiphyseal grading network to perform prediction of each epiphyseal grade, so as to calculate the bone age of the current target DR image.

[0072] In a possible implementation manner, obtaining the bone information of the \(i\)th bone includes: obtaining the minimum bounding rectangle in the target DR image according to the perpendicular line corresponding to the \(i\)th bone; calculating the width-to-height ratio of the target DR image according to the minimum bounding rectangle in the target DR image and the minimum bounding rectangle of the pre-stored standard hand; calculating the bone information corresponding to each of the \(m\) bones according to the width-to-height ratio of the target DR image and the pre-stored bone information of the standard hand. The key point annotation device for bone age detection can select the minimum bounding rectangle of the hand region according to the determined perpendicular line direction and the obtained hand region, calculate the width-to-height ratio using the width and height of the minimum bounding rectangle and the minimum bounding rectangle of the standard hand, and calculate the actual width and height of each epiphysis according to the width-to-height ratio and the width and height of the epiphysis of the standard hand.

[0073] Please refer to the appendix Figure 5 , Figure 5 which is a schematic diagram of a bone age detection process provided by an embodiment of the present application. As Figure 5 shown, after inputting the bone age DR image dataset, the key point annotation device for bone age detection can obtain the annotated hand region, the epiphysis points of the bones, the bone ROI images, the bone grades, etc. Then, according to the center point of the connection line of the annotated ulna and radius bones, connect lines to each epiphysis annotation point, and use the perpendicular line direction of the connection line at each epiphysis point as the calculation direction. Along the perpendicular line direction, calculate the centroid of all points on the perpendicular line in the current finger region until the centroid coincides with the epiphysis point, indicating that the current annotation is accurate. Among them, when obtaining the annotated hand region, the epiphysis points of the bones, the bone ROI images, the bone grades, etc., it is necessary to obtain them through the hand region segmentation model, the key point detection model, and the bone grade classification model according to the selected standard hand. In addition, before passing through the key point detection model and the bone grade classification model, the bone age DR image to be predicted is segmented for the hand region to remove the non-hand region part, and then the bone key points are detected. If the detected key points are inaccurate, the bone key points are corrected. Finally, according to the corrected key points and the bone ROI images determined according to the width-to-height ratio of the circumscribed rectangle of the hand region and the circumscribed rectangle of the standard hand, the bone grade is predicted.

[0074] An embodiment of the present application describes a centroid iteration method for improving the detection accuracy of epiphyseal key points based on the background of bone age prediction. Through this method, the embodiment of the present application can determine whether the centroid corresponding to the bone coincides with the epiphyseal point corresponding to the bone. If the centroid and the epiphyseal point coincide, it proves that the epiphyseal point can be marked as the key point of the bone; if they do not coincide, the key points are continuously screened until they coincide with the centroid. This method can improve the quality of manual annotation and improve the detection accuracy of the model key points according to prior knowledge (the images of the epiphyseal part of the fingers are density-related and symmetric left and right) during prediction. A simple and lightweight model can achieve good results, improve the speed of the overall bone age prediction process, reduce the requirements of the key point detection method for hardware, and save the application costs of various key point detection systems.

[0075] The method of the embodiment of the present application is elaborated in detail above. Below, a key point annotation device based on bone age detection related to the embodiment of the present application is provided. The key point annotation device 10 based on bone age detection can be a service device that quickly acquires, processes, analyzes, and extracts valuable data, and brings various conveniences to third-party use based on interactive data. Please refer to the appendix Figure 6 , Figure 6 FIG. is a schematic structural diagram of a key point annotation device based on bone age detection provided by an embodiment of the present application. The key point annotation device 10 based on bone age detection may include a first acquisition unit 101, an annotation unit 102, a perpendicular line unit 103, a centroid unit 104, and a first key point unit 105, and may further include a second key point unit 106, a second acquisition unit 107, an extraction unit 108, and a prediction unit 109.

[0076] The first acquisition unit 101 is configured to acquire a target digital X-ray DR image, and the target DR image includes a bone image of the hand;

[0077] The annotation unit 102 is configured to annotate each of the m bones in the target DR image to obtain the epiphyseal points corresponding to each of the m bones. The m bones include n bones, as well as the ulna bone and the radius bone. m is a positive integer greater than 2, and n is a positive integer less than m or greater than 1;

[0078] The perpendicular line unit 103 is configured to connect the center point between the epiphyseal points corresponding to the ulna bone and the epiphyseal points corresponding to the radius bone to the epiphyseal points corresponding to the n bones respectively, and determine the perpendicular lines passing through each of the n epiphyseal points and perpendicular to the connection line between each epiphyseal point and the center point;

[0079] The centroid unit 104 is configured to calculate the centroid corresponding to the i-th bone within the finger region corresponding to the i-th bone according to the perpendicular line corresponding to the i-th bone, where i = 1, 2... n;

[0080] The first key point unit 105 is configured to update the epiphyseal point corresponding to the i-th bone to the key point of the i-th bone if the centroid coincides with the epiphyseal point corresponding to the i-th bone.

[0081] In a possible implementation manner, the apparatus further includes: a second key point unit 106, configured to: if the centroid corresponding to the i-th bone does not coincide with the epiphyseal point corresponding to the i-th bone, update the average value of the epiphyseal point and the centroid to the epiphyseal point of the i-th bone; re-determine the key point corresponding to the i-th bone according to the updated epiphyseal point of the i-th bone until the updated epiphyseal point of the i-th bone coincides with the centroid corresponding to the i-th bone.

[0082] In a possible implementation manner, the first acquisition unit 101 is specifically configured to: acquire a first DR image; process the first DR image to remove the non-hand region part of the first DR image to obtain the target DR image.

[0083] In a possible implementation manner, the apparatus further includes: a second acquisition unit 107, configured to acquire the bone information of the i-th bone, where the bone information includes the length and / or width of the i-th bone; an extraction unit 108, configured to extract the epiphyseal region image of the i-th bone from the target DR image based on the key point of the i-th bone and the perpendicular line corresponding to the i-th bone according to the bone information of the i-th bone; a prediction unit 109, configured to predict the bone age of the hand bone in the target DR image according to the extracted multiple epiphyseal region of interest (ROI) images.

[0084] In a possible implementation manner, the second acquisition unit 107 is specifically configured to: obtain the minimum circumscribed rectangle in the target DR image according to the perpendicular line corresponding to the i-th bone; calculate the width-to-height ratio of the target DR image according to the minimum circumscribed rectangle in the target DR image and the minimum circumscribed rectangle of the pre-stored standard hand; calculate the bone information corresponding to each of the m bones according to the width-to-height ratio of the target DR image and the pre-stored bone information of the standard hand.

[0085] In a possible implementation manner, the epiphyseal point corresponding to the ulna bone is located at the center point of the epiphysis corresponding to the ulna bone, and the epiphyseal point corresponding to the radius bone is located at the center point of the epiphysis corresponding to the radius bone.

[0086] It should be noted that the implementation of each operation can also refer to the corresponding description in the method embodiment shown in Figures 2 - 5 which will not be elaborated here.

[0087] As shown Figure 7 in Figure 7 Figure 5, it is a schematic structural diagram of another key point annotation device based on bone age detection provided by an embodiment of the present application. The device 20 is applied to a second device and includes at least one processor 201, at least one memory 202, and at least one communication interface 203. In addition, the device may also include general components such as an antenna, which will not be elaborated here.

[0088] The processor 201 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the above programs.

[0089] The communication interface 203 is used to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), core network, wireless local area network (WLAN), etc.

[0090] The memory 202 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), or other optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic storage media such as magnetic disks, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may exist independently and be connected to the processor through a bus. The memory may also be integrated with the processor.

[0091] Among them, the memory 202 is used to store the application program code for executing the above solution, and is controlled by the processor 201 for execution. The processor 201 is used to execute the application program code stored in the memory 202.

[0092] The code stored in the memory 202 can execute the above Figure 2The provided key point annotation method based on bone age detection. For example, when device 20 is a key point annotation device based on bone age detection, a target digital radiography (DR) image can be obtained, and the target DR image includes the bone image of the hand. Each of the m bones in the target DR image is annotated to obtain the epiphyseal points corresponding to each of the m bones. The m bones include n bones, the ulna bone, and the radius bone. m is a positive integer greater than 2, and n is a positive integer less than m or greater than 1. The central point between the epiphyseal points corresponding to the ulna bone and the epiphyseal points corresponding to the radius bone is respectively connected to the epiphyseal points corresponding to the n bones to determine the perpendicular lines perpendicular to the connections between each of the n epiphyseal points and the central point. In the finger region corresponding to the i-th bone, according to the perpendicular line corresponding to the i-th bone, the centroid corresponding to the i-th bone is calculated, where i = 1, 2... n. If the centroid coincides with the epiphyseal point corresponding to the i-th bone, the epiphyseal point corresponding to the i-th bone is updated to the key point of the i-th bone.

[0093] It should be noted that the functions of the functional units in the key point annotation device based on bone age detection described in the embodiments of the present application can be referred to Figures 2 - 5 the corresponding description of the method embodiment shown, and will not be elaborated here.

[0094] In the present application, the units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiments of the present application.

[0095] In addition, the functional components in each embodiment of the present application can be integrated into one component, or each component can exist physically alone, or two or more components can be integrated into one component. The above integrated components can be implemented in the form of hardware or in the form of software functional units.

[0096] When the integrated component is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a second device, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs that can store program codes.

[0097] As described above, the above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed by this application can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

[0098] It should be understood that in various embodiments of this application, the magnitude of the serial numbers of the above processes does not mean the order of execution is prior or subsequent. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of this application. Although this application has been described in combination with various embodiments herein, however, in the process of this application required to be protected by the embodiments, those skilled in the art can understand and implement other variations of the disclosed embodiments.

Claims

1. A key point labeling method based on bone age detection, characterized in that: include: Acquire a target digital X-ray DR image, wherein the target DR image includes a bone image of a hand; Label each bone of the m bones in the target DR image to obtain an epiphyseal point corresponding to each bone of the m bones, wherein the m bones include n bones and an ulna bone and a radius bone, where m is a positive integer greater than 2, and n is a positive integer less than m or greater than 1; Connect the center point of the line between the epiphyseal point corresponding to the ulna bone and the epiphyseal point corresponding to the radius bone to the epiphyseal points corresponding to the n bones respectively, and determine a vertical line perpendicular to the line between the center points and corresponding to each epiphyseal point in the n epiphyseal points; In the finger region corresponding to the ith bone, the center of mass corresponding to the ith bone is calculated according to the vertical line corresponding to the ith bone, i=1, 2…n; If the centroid coincides with the epiphyseal point corresponding to the i-th bone, the epiphyseal point corresponding to the i-th bone is updated to the key point of the i-th bone.

2. The method according to claim 1, characterized in that: The method further comprises: If the centroid corresponding to the i-th bone does not coincide with the epiphyseal point corresponding to the i-th bone, updating the average value of the epiphyseal point and the centroid to the epiphyseal point of the i-th bone; According to the updated epiphyseal point of the i-th bone, the key point corresponding to the i-th bone is re-determined until the updated epiphyseal point of the i-th bone coincides with the center of mass corresponding to the i-th bone.

3. The method according to claim 1, characterized in that: The step of acquiring a target digital X-ray DR image comprises: Acquire a first DR image; The first DR image is processed, and the non-hand area portion of the first DR image is removed to obtain the target DR image.

4. The method according to claim 1, characterized in that: The method further comprises: Obtaining bone information of the i-th bone, wherein the bone information includes the length and / or width of the i-th bone; Extracting an epiphyseal region image of the i-th bone from the target DR image according to the key points of the i-th bone and the vertical line corresponding to the i-th bone and based on the bone information of the i-th bone; The bone age of the hand bones in the target DR image is predicted based on the extracted multiple epiphyseal region of interest ROI images.

5. The method according to claim 4, characterized in that: The obtaining of the skeleton information of the i-th skeleton comprises: Obtaining a minimum circumscribed rectangle in the target DR image according to the vertical line corresponding to the i-th bone; Calculating the aspect ratio of the target DR image according to the minimum bounding rectangle in the target DR image and the pre-stored minimum bounding rectangle of a standard hand; According to the aspect ratio of the target DR image and the pre-stored skeleton information of the standard hand, the skeleton information corresponding to each of the m skeletons is calculated.

6. The method according to claim 1, characterized in that: The epiphysis point corresponding to the ulna bone is located at the center point of the epiphysis corresponding to the ulna bone, and the epiphysis point corresponding to the radius bone is located at the center point of the epiphysis corresponding to the radius bone.

7. A key point marking device based on bone age detection, characterized in that: include: A first acquisition unit is used to acquire a target digital X-ray DR image, wherein the target DR image includes a bone image of a hand; A labeling unit, used for labeling each bone of the m bones in the target DR image to obtain an epiphyseal point corresponding to each bone of the m bones, wherein the m bones include n bones, ulna bones and radius bones, m is a positive integer greater than 2, and n is a positive integer less than m or greater than 1; A vertical line unit, used to connect the center point of the line between the epiphyseal point corresponding to the ulna bone and the epiphyseal point corresponding to the radius bone with the epiphyseal points corresponding to the n bones, and determine a vertical line perpendicular to the line between the center points and corresponding to each epiphyseal point in the n epiphyseal points; A centroid unit is used to calculate the centroid corresponding to the ith bone in the finger region corresponding to the ith bone according to the vertical line corresponding to the ith bone, i=1, 2…n; The first key point unit is used to update the epiphyseal point corresponding to the ith bone to the key point of the ith bone if the center of mass coincides with the epiphyseal point corresponding to the ith bone.

8. The device according to claim 7, characterized in that: The device further includes: a second key point unit, wherein the second key point unit is configured to: If the centroid corresponding to the i-th bone does not coincide with the epiphyseal point corresponding to the i-th bone, updating the average value of the epiphyseal point and the centroid to the epiphyseal point of the i-th bone; According to the updated epiphyseal point of the i-th bone, the key point corresponding to the i-th bone is re-determined until the updated epiphyseal point of the i-th bone coincides with the center of mass corresponding to the i-th bone.

9. A computer device, characterized in that: The method comprises a processing component, a storage component and a communication module component, wherein the processing component, the storage component and the communication component are interconnected, wherein the storage component is used to store computer programs, and the communication component is used to exchange information with external devices; the processing component is configured to call the computer program to execute the method as described in any one of claims 1-6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 6.

Images