A bone image recognition method, device, equipment and storage medium
By constructing and optimizing candidate objects in skeletal images, the problems of low reliability and accuracy in skeletal image recognition technology are solved. The mutual verification and fusion of recognition results are realized, improving the reliability and accuracy of recognition and providing reliable data support for disease diagnosis.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- PEKING UNION MEDICAL COLLEGE HOSPITAL
- Filing Date
- 2026-01-06
- Publication Date
- 2026-06-05
Smart Images
Figure CN122156041A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical information processing technology, and in particular to a method, apparatus, device, and storage medium for skeletal image recognition. Background Technology
[0002] The health of the human skeleton is directly related to overall health, and skeletal imaging technology is the most basic and commonly used basis for diagnosing such diseases. For example, the spine, as the central axial skeleton, is the source of many common and frequently occurring diseases, such as scoliosis, herniated discs, and spondylolisthesis. In clinical practice, doctors typically assess the spine's sequence, morphology, and biomechanical parameters, such as the Cobb angle, based on X-ray images. Traditionally, the analysis of skeletal images in clinical practice relies heavily on the doctor's medical knowledge and experience, which suffers from high subjectivity, low efficiency, and poor repeatability.
[0003] With the development of deep learning technology, CNN (Convolutional Neural Network)-based models are widely used in medical image analysis, especially for identifying specific objects in skeletal images for diagnosis. However, using a single-task model for skeletal image recognition makes it difficult to guarantee the reliability and accuracy of the model. For example, a single segmentation model is prone to producing blurred or incorrect edges at vertebral adhesions in spinal images, while a single keypoint detection model is prone to localization bias or missed detections / false detections due to unclear local features. On the other hand, models using integrated multi-task learning have high coupling in their structure, complex training and optimization processes, and the replacement and upgrading of sub-task architectures are very difficult, resulting in high technology iteration costs. Summary of the Invention
[0004] This invention provides a method, apparatus, device, and storage medium for skeletal image recognition, aiming to achieve mutual verification and fusion of different types of recognition results, significantly improve the reliability and accuracy of skeletal image recognition, and provide a rich and reliable data foundation for downstream data analysis and disease diagnosis.
[0005] In a first aspect, embodiments of the present invention provide a skeletal image recognition method, comprising: Obtain bone recognition points and bone recognition regions from the original skeletal image; Candidate bone objects are constructed based on the spatial relationship between the bone recognition points and the bone recognition areas; Based on the candidate skeleton objects, the skeleton recognition points and the skeleton recognition regions are optimized to obtain the skeleton key points and the skeleton object regions. The object image in the original skeletal image is determined based on the skeletal key points and the skeletal object region.
[0006] Optionally, constructing candidate skeleton objects based on the spatial relationship between the skeleton recognition points and the skeleton recognition regions includes: Obtain the spatial range of the skeleton recognition area; The bone recognition points within the spatial range and the corresponding bone recognition regions within the spatial range are constructed as candidate bone objects.
[0007] Optionally, the skeleton recognition region is a skeleton segmentation mask; The acquisition of the spatial range of the skeleton recognition region includes: Calculate the target bounding rectangle range of the skeleton segmentation mask, and use it as the spatial range of the region.
[0008] Optionally, the step of performing candidate optimization processing on the bone recognition points and the bone recognition regions based on the candidate bone objects to obtain bone key points and bone object regions includes: The bone recognition points constituting the candidate bone object are identified as candidate key points; Based on the candidate key points, the bone recognition region constituting the candidate bone object is subjected to region optimization processing to obtain the bone object region; Based on the skeletal object region, the skeletal key points are determined from the candidate key points.
[0009] Optionally, the step of performing region optimization processing on the bone recognition region constituting the candidate bone object based on the candidate key points to obtain the bone object region includes: Based on the candidate key points corresponding to the candidate skeleton object, obtain the number of key points and the key point category of the candidate skeleton object; The candidate skeleton objects whose number of key points and key point categories meet the point association conditions are determined as the target skeleton objects; The bone recognition region corresponding to the target bone object is determined as the bone object region.
[0010] Optionally, after determining the candidate skeleton objects whose number of key points and key point categories meet the point association conditions as target skeleton objects, the method further includes: Obtain the key point positions of each bone key point corresponding to the target bone object; Based on the location of the key points, the region boundary of the bone object corresponding to the target bone object is optimized.
[0011] Optionally, the step of optimizing the region boundary of the skeleton object corresponding to the target skeleton object based on the key point location includes: Generate a target polygon range with the positions of each of the aforementioned key points as vertices; The skeletal object region is optimized based on the target polygon range to optimize the skeletal object region into the target polygon region corresponding to the target polygon range.
[0012] In a second aspect, embodiments of the present invention provide a skeletal image recognition device, comprising: The image recognition module is used to acquire bone recognition points and bone recognition regions in the original bone image; The object construction module is used to construct candidate bone objects based on the spatial relationship between the bone recognition points and the bone recognition regions; The object optimization module is used to perform candidate optimization processing on the bone recognition points and the bone recognition regions based on the candidate bone objects, so as to obtain the bone key points and the bone object regions. The object determination module is used to determine the image of the object to be identified in the original skeletal image based on the skeletal key points and the skeletal object region.
[0013] Thirdly, embodiments of the present invention provide a skeletal image recognition device, comprising: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the skeletal image recognition method provided in any embodiment of the present invention.
[0014] Fourthly, embodiments of the present invention provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the skeletal image recognition method as provided in any embodiment of the present invention.
[0015] This invention provides a method, apparatus, device, and storage medium for skeletal image recognition. By acquiring skeletal recognition points and regions from an original skeletal image, and constructing candidate skeletal objects based on their spatial relationships, the invention further optimizes the skeletal recognition points and regions to obtain key skeletal points and skeletal object regions. These key skeletal points and skeletal object regions are then used to determine the recognition object image in the original skeletal image. This invention solves the problem of low reliability and accuracy in existing skeletal image recognition technologies, enabling mutual verification and fusion of different types of recognition results. It significantly improves the reliability and accuracy of skeletal image recognition, providing a rich and reliable data foundation for downstream data analysis and disease diagnosis. Attached Figure Description
[0016] Figure 1 This is a flowchart of a skeletal image recognition method provided in Embodiment 1 of the present invention; Figure 2 This is a flowchart of a skeletal image recognition method provided in Embodiment 2 of the present invention; Figure 3 This is a flowchart of a skeletal image recognition method provided in Embodiment 2 of the present invention; Figure 4 This is a schematic diagram of the structure of a skeletal image recognition device provided in Embodiment 3 of the present invention; Figure 5 This is a schematic diagram of the structure of a skeletal image recognition device provided in Embodiment 4 of the present invention. Detailed Implementation
[0017] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.
[0018] Example 1 Figure 1 This is a flowchart of a skeletal image recognition method provided in Embodiment 1 of the present invention. This embodiment is applicable to the situation of recognizing a specific skeletal object image in a skeletal image. The method can be executed by a skeletal image recognition device, which can be implemented by hardware and / or software and is generally integrated into an electronic device, such as a computer device. The method specifically includes: Step 110: Obtain the bone recognition points and bone recognition regions in the original bone image.
[0019] The original skeletal image can be image data obtained from medical image acquisition of limb bones, including specific skeletal objects to be identified for diagnosis. Skeletal identification points can be image feature points in the original skeletal image that belong to a specific skeletal object. Skeletal identification regions can be image pixel areas in the original skeletal image that belong to a specific skeletal object.
[0020] Specifically, raw bone images can be image data acquired using any medical imaging technology available for clinical diagnosis, and must include images of the specific bone object to be identified for diagnosis. The specific bone object to be identified for diagnosis typically refers to the bone at the affected site, and may also include other bones related to the affected site. Raw bone images usually also include images of other unrelated bones and / or body tissues surrounding the specific bone object. Therefore, in order to analyze the state of a specific bone object based on the raw bone images during subsequent image analysis and disease diagnosis, it is necessary to accurately identify specific portions of the image belonging to the specific bone object within the raw bone images.
[0021] For example, the original skeletal images can be anteroposterior and lateral X-ray images of the patient's spine, and the specific skeletal objects to be identified for diagnosis can be the individual vertebrae of the spine as defined anatomically. Thus, the state of each vertebra of the spine can be determined based on the anteroposterior and lateral X-ray images of the spine, thereby analyzing and diagnosing the patient's spinal condition.
[0022] Furthermore, in the original skeletal image, appropriate image processing techniques can be used to obtain bone recognition points and bone recognition regions respectively. Typically, bone recognition points refer to the anatomical key points of a specific skeletal object in the image, such as the corner points of a vertebral body; there can be one or more of these. Bone recognition regions refer to the pixel set area corresponding to a specific skeletal object in the image; there can be one or more of these. It should be noted that both bone recognition points and bone recognition regions are not required to be accurately identified; their accuracy depends on the image processing techniques used. Optionally, the process of obtaining bone recognition points can also acquire the category and confidence level of each bone recognition point. The category can specifically be its location within the skeleton or its corresponding skeletal part, such as the upper left or lower right corner of a vertebral body.
[0023] In one optional implementation, obtaining bone recognition points and bone recognition regions from the original bone image may include: inputting the original bone image into a keypoint detection model and obtaining the bone recognition points output by the keypoint detection model. The keypoint detection model may be a pre-trained deep learning model, preferably a YOLO (You Only Look Once) series model. Based on pre-training, the keypoint detection model can be used to detect anatomical keypoints of the skeleton and, after processing, generate a result set. This result set may contain one or more sets of keypoints, and each keypoint may further include at least its coordinates in the image, its category, and its confidence score.
[0024] In an optional implementation, obtaining the bone recognition points and bone recognition regions in the original skeletal image may include: inputting the original skeletal image into an instance segmentation model and obtaining the bone recognition regions output by the instance segmentation model. The instance segmentation model may be a pre-trained deep learning model, preferably a SAM model (SegmentAnything Model). Based on pre-training, the instance segmentation model can be used to identify and segment independent skeletal objects in the image, and after processing, generate a result set. This result set may contain one or more sets of segmentation masks, each segmentation mask representing a pixel-level region of an independent skeletal object.
[0025] To initially improve the accuracy of bone recognition points and bone recognition regions, as well as the stability of the acquisition process, in an optional implementation, before acquiring bone recognition points and bone recognition regions in the original bone image, the process may include: standardizing the original bone image, which may specifically include grayscale normalization, contrast enhancement, and noise reduction.
[0026] Step 120: Construct candidate skeleton objects based on the spatial relationship between the skeleton recognition points and the skeleton recognition area.
[0027] The spatial relationship can be the relationship between the positions of each bone recognition point and each bone recognition region in the original bone image. A candidate bone object can be a bone object in the original bone image that is suspected to be a specific bone object to be identified, represented by bone recognition points and bone recognition regions with specific spatial relationships.
[0028] Specifically, based on the spatial relationship between bone recognition points and bone recognition regions in the original bone image, if any bone object region covers a location and has at least one bone keypoint within a certain range nearby, it can be concluded that the bone object region and the bone keypoint can be used to represent the image of the same bone object, and therefore can be constructed as the same candidate bone object. The number of candidate bone objects can be one or more, not exceeding the number of bone recognition regions, and each candidate bone object contains one bone recognition region and at least one bone recognition point.
[0029] Step 130: Based on the candidate skeleton objects, perform candidate optimization processing on the skeleton recognition points and skeleton recognition regions to obtain the skeleton key points and skeleton object regions.
[0030] The candidate optimization process involves cross-validating skeletal recognition points and skeletal recognition regions to determine sufficiently accurate recognition results. Skeletal keypoints can be sufficiently accurate image feature points within the skeletal recognition points that belong to a specific skeletal object image. The skeletal object region can be a sufficiently accurate image pixel region within the skeletal recognition region that belongs to a specific skeletal object image.
[0031] Specifically, based on the spatial relationship between bone recognition points and bone recognition regions within the same candidate bone object, it can be determined whether the candidate bone object is the specific bone object to be identified. This further assesses the accuracy of the bone recognition points and regions, thereby determining sufficiently accurate bone key points and bone object regions. For any candidate bone object, the area covered by its bone object region and within a certain vicinity should contain a specific number and / or type of bone key points. Based on this, the bone object region can be determined from the bone recognition region. Similarly, the locations of its bone key points should be within the coverage area of the bone object region or within a certain vicinity. Based on this, bone key points can be determined from the bone recognition points. Through this process, mutual verification between bone recognition points and bone recognition regions can be achieved, resulting in a sufficiently accurate identification result.
[0032] Step 140: Determine the object image to be identified in the original skeletal image based on the skeletal key points and the skeletal object region.
[0033] The image of the object to be identified can be an image of a specific skeletal object that needs to be identified for diagnosis from the original skeletal image.
[0034] Specifically, once sufficiently accurate skeletal key points and skeletal object regions are determined, the skeletal object regions can be used to determine the region corresponding to the object image in the original skeletal image, and the skeletal key points can be used to determine the pose state of the specific skeletal object presented in the object image, thereby jointly determining the object image.
[0035] The technical solution of this embodiment obtains bone recognition points and bone recognition regions in the original bone image, and constructs candidate bone objects based on the spatial relationship between the bone recognition points and bone recognition regions. Based on the candidate bone objects, the bone recognition points and bone recognition regions are optimized to obtain bone key points and bone object regions. Thus, the recognition object image in the original bone image is determined based on the bone key points and bone object regions. This solves the problem of low reliability and accuracy of existing bone image recognition technologies, realizes mutual verification and fusion of different types of recognition results, and greatly improves the reliability and accuracy of bone image recognition, providing a rich and reliable data foundation for downstream data analysis and disease diagnosis.
[0036] Example 2 Figure 2 This is a flowchart of a skeletal image recognition method provided in Embodiment 2 of the present invention. This embodiment further refines the above technical solution, and may involve performing candidate optimization processing on the skeletal recognition points and the skeletal recognition region based on the candidate skeletal object to obtain skeletal key points and a skeletal object region. This includes: determining the skeletal recognition points constituting the candidate skeletal object as candidate key points; performing region optimization processing on the skeletal recognition region constituting the candidate skeletal object based on the candidate key points to obtain the skeletal object region; and determining the skeletal key points from the candidate key points based on the skeletal object region. Specifically, this method includes: Step 210: Obtain the bone recognition points and bone recognition regions in the original bone image.
[0037] Step 220: Construct candidate skeleton objects based on the spatial relationship between the skeleton recognition points and the skeleton recognition area.
[0038] In an optional implementation, constructing candidate bone objects based on the spatial relationship between bone recognition points and bone recognition regions may include: obtaining the regional spatial range of the bone recognition region; and constructing candidate bone objects by combining bone recognition points within the regional spatial range with the corresponding bone recognition regions.
[0039] The spatial range of the region can be the range of locations covered by the bone recognition region in the original bone image.
[0040] Specifically, the spatial extent of a bone recognition region can be determined based on the position of each pixel within the original bone image. For any bone recognition region, bone recognition points within its spatial extent can be identified as belonging to the same bone object, and thus can collectively construct the same candidate bone object.
[0041] In an optional implementation, the skeleton recognition region can be a skeleton segmentation mask. Accordingly, obtaining the regional spatial range of the skeleton recognition region can include: calculating the target bounding rectangle range of the skeleton segmentation mask as the regional spatial range.
[0042] Each bone segmentation mask represents a pixel-level region of an independent bone object. The bounding rectangle of each target can be the spatial extent of the original bone image covered by the smallest bounding rectangle of the corresponding bone segmentation mask.
[0043] In an optional implementation, constructing candidate bone objects by constructing bone recognition points within a region spatial range and corresponding bone recognition regions within that region spatial range may include: traversing all bone recognition points, determining whether each bone recognition point is within any region spatial range, and, if it is determined that any bone recognition point is within any region spatial range, establishing a preliminary association between the bone recognition point and the corresponding bone recognition region; and constructing each bone recognition region and at least one bone recognition point with a preliminary association between them as a candidate bone object.
[0044] Step 230: Determine the skeletal recognition points that constitute the candidate skeleton object as candidate key points.
[0045] Among them, candidate key points can be skeletal recognition points that are likely to become skeletal key points.
[0046] Specifically, if any skeletal recognition point constitutes any candidate skeletal object, it indicates that the skeletal recognition point may belong to the specific skeletal object to be identified, along with the corresponding skeletal recognition region. Conversely, if any skeletal recognition point does not constitute any candidate skeletal object, it indicates that the skeletal recognition point cannot belong to the specific skeletal object to be identified. Therefore, skeletal recognition points that constitute candidate skeletal objects can be identified as candidate key points to achieve verification and screening of skeletal recognition points based on skeletal recognition regions.
[0047] Step 240: Based on the candidate key points, perform region optimization processing on the bone recognition region that constitutes the candidate bone object to obtain the bone object region.
[0048] Among them, region optimization processing can be an operation that filters and / or adjusts the skeleton recognition region to obtain the skeleton object region.
[0049] Specifically, each candidate skeletal object consists of at least one candidate keypoint and a skeletal recognition region that overlap or are sufficiently close to each other. For any skeletal recognition region, it can be verified and filtered based on the candidate keypoints belonging to the same candidate skeletal object. Typically, based on the characteristics of the specific skeletal object to be identified, the number and categories of candidate keypoints constituting that skeletal object can be determined. For example, each vertebral body candidate skeletal object must have at least four corner points as its corresponding candidate keypoints. Based on this, candidate skeletal objects can be filtered to obtain some or all of the candidate skeletal objects whose corresponding keypoints meet the feature requirements. Furthermore, the skeletal recognition regions of some or all candidate skeletal objects can be fine-tuned to better match the areas they cover with the corresponding candidate keypoint positions, thereby achieving region optimization processing of the skeletal recognition region and obtaining the skeletal object region. At this point, the candidate keypoints corresponding to the skeletal object region can be identified as skeletal keypoints.
[0050] In one optional implementation, the bone recognition region constituting the candidate bone object is optimized based on the candidate key points to obtain the bone object region. This may include: obtaining the number and type of key points of the candidate bone object based on the candidate key points corresponding to the candidate bone object; determining the candidate bone object whose number and type of key points meet the point association condition as the target bone object; and determining the bone recognition region corresponding to the target bone object as the bone object region.
[0051] The number of keypoints in any candidate skeleton object can be the total number of candidate keypoints constituting it. The keypoint category of any candidate skeleton object can be the category to which the candidate keypoints belong, used to describe the feature location represented by each candidate keypoint in the candidate skeleton object. The point association condition can describe the total number and category of skeleton keypoints that should be included in a specific skeleton object to be identified. The target skeleton object can be a specific skeleton object accurately identified in the original skeleton image, represented by skeleton keypoints and skeleton object regions.
[0052] Specifically, the point association conditions can be pre-set based on the characteristics of the specific skeletal objects to be identified in the diagnostic requirements. By obtaining the number and category of keypoints for each candidate skeletal object, it can be determined whether it meets the point association conditions. If it does, it means that the characteristics exhibited by the candidate keypoints contained in the candidate skeletal object are consistent with the specific skeletal object to be identified, and thus it is identified as the target skeletal object, thereby determining the skeletal object region. Conversely, if it does not meet the conditions, it means that the characteristics exhibited by the candidate keypoints contained in the candidate skeletal object are different from the specific skeletal object to be identified, and it cannot be used as the target skeletal object.
[0053] In an optional implementation, after determining the candidate skeleton objects whose number of key points and key point categories meet the point association conditions as the target skeleton objects, the implementation may further include: obtaining the key point positions of each skeleton key point corresponding to the target skeleton object; and optimizing the region boundary of the skeleton object region corresponding to the target skeleton object based on the key point positions.
[0054] Here, the keypoint positions of any target skeletal object can be the locations of the skeletal keypoints that constitute it within the original skeletal image. Region boundary optimization can be an operation that fine-tunes the region boundaries of the skeletal object's region.
[0055] Specifically, after the target skeleton object is determined, since the positional relationship between the key points of the skeleton object and the skeleton object region includes the possibility of overlap and sufficient proximity, if the positions of any key points of the skeleton object and the skeleton object region of the same target skeleton object do not overlap, for example, if any key point of the skeleton object is within the target bounding rectangle of the skeleton object region but not within the actual range of the skeleton object region, then the region boundary of the skeleton object region can be further optimized based on the key point position.
[0056] In one optional implementation, optimizing the region boundary of the skeletal object region corresponding to the target skeletal object based on the key point locations may include: generating a target polygon range with each key point location as a vertex; and optimizing the region boundary of the skeletal object region based on the target polygon range so that the skeletal object region is optimized into the target polygon region corresponding to the target polygon range.
[0057] The target polygon range can be the area covered by polygons formed by keypoints of the same target skeleton object within the original skeleton image. The target polygon region can be a polygon region formed by keypoints of the same target skeleton object.
[0058] Specifically, in order for the optimized skeletal object region to cover all the key points of the same target skeletal object, the positions of each key point of the target skeletal object can be used as vertices to generate the corresponding target polygon range. Then, the target polygon region corresponding to the target polygon range can be used as the optimized skeletal object region.
[0059] In an alternative implementation, the convex hull algorithm can be used to generate the target polygon range with each key point location as a vertex.
[0060] Step 250: Based on the skeletal object region, determine the skeletal key points from the candidate key points.
[0061] Specifically, based on the screening of candidate skeleton objects and corresponding skeleton recognition regions, the candidate key points corresponding to the final obtained skeleton recognition regions can be determined as skeleton key points, while the candidate key points corresponding to the screened skeleton recognition regions can be screened out accordingly.
[0062] Step 260: Determine the object image to be identified in the original skeletal image based on the skeletal key points and the skeletal object region.
[0063] The technical solution of this embodiment optimizes the selection of bone recognition points and bone recognition regions based on the initially constructed candidate bone objects. It provides a highly efficient and reliable means for mutual verification and fusion of bone recognition points and bone recognition regions, which greatly optimizes the accuracy of existing single recognition technologies.
[0064] In a typical example, the skeletal image recognition method provided in Embodiment 2 of the present invention can be applied to identify each individual vertebral body object in anteroposterior and lateral X-ray images of the spine. Specifically, Figure 3 This is a flowchart of a skeletal image recognition method provided in Embodiment 2 of the present invention. Figure 3As shown, the method begins with an input full-spine X-ray image. The input image first enters the image preprocessing step. In this step, one or more operations can be performed, such as resizing the image to a preset size, such as 1024×2048 pixels; performing histogram equalization to enhance the global contrast of the image; and applying Gaussian filtering or median filtering to reduce image noise. Further, independent keypoint detection can be performed. The preprocessed image is fed into the first inference module, which pre-deploys a trained keypoint detection model, such as the YOLOv11 model. This model is trained on a large amount of X-ray image data with vertebral corner annotations and can directly regress the two-dimensional quadrant coordinates and categories of all potential vertebral corners on the entire image, such as the upper left, lower left, upper right, and lower right corners of C7-L5, as well as a confidence score representing the detection's reliability. The output of this step is a preliminary set of keypoints. Simultaneously or sequentially, independent instance segmentation can be performed. Specifically, the preprocessed image is also fed into the second inference module, which deploys an independent instance segmentation model. This model can be a SAM model combined with an appropriate prompt generation strategy, or fine-tuned to focus on segmenting cone-type objects. This model processes the image and outputs a set of pixel-level segmentation masks, theoretically each mask corresponding to an independent cone. The output of this step is a preliminary set of segmentation masks. Further processing can be performed by the result fusion and correction engine. Two independent preliminary result sets are simultaneously fed into the result fusion and correction engine module, which first performs an association matching step. Specifically, the engine first processes the segmentation mask set. For each mask, its minimum bounding rectangle is calculated. Then, the engine iterates through each keypoint in the keypoint set. If the coordinates of a keypoint fall within the bounding rectangle of a mask, the engine establishes a temporary association, assigning the keypoint to that mask. After this process, a batch of candidate "cone objects" is formed, each object potentially containing a mask and zero to multiple keypoints. Subsequently, the engine can perform bidirectional verification and filtering. The engine first identifies keypoints that, after association, do not fall within any mask rectangle. These points are judged as false positives caused by artifacts or noise and are directly eliminated by the system. Next, the engine examines each candidate "vertebral object." Based on a pre-defined rule that "a reliable vertebral object must be associated with at least 3 corner points," for candidate objects with fewer than 3 associated keypoints, the engine determines that their corresponding segmentation mask is likely a false positive, such as incorrectly segmenting ribs or other tissues into vertebral bodies. Therefore, this mask and its few associated keypoints are eliminated. After this step, only highly reliable "reliable objects" remain.Furthermore, in the fusion correction step, for each "reliable object," the engine optimizes the boundaries of its segmentation mask using the key points associated with it. For example, a verified object contains a segmentation mask and four high-confidence corner points. The engine can detect that the position of one of the "top right corner" key points slightly exceeds the pixel boundary of the current mask. At this point, the engine can invoke a convex hull algorithm to generate a minimum convex polygon that can enclose all the key points, using all the key points inside the object as vertices. Then, the interior region of this convex polygon is used as the corrected new mask. This method ensures that the geometry of the segmentation result perfectly matches the precise position of the key points, thereby greatly improving the accuracy of the segmentation. Based on this, the results can be output and passed to downstream task steps. The engine ultimately outputs a series of verified and corrected high-precision cone objects. Each object contains an accurate mask and a set of matching key points. This information is then passed to downstream application modules. This module can use this precise coordinate and regional information to automatically calculate various parameters of clinical interest, such as Cobb angle, vertebral body height, intervertebral disc height, and vertebral body wedge deformation index, and present the results on the user interface or directly generate a structured diagnostic report, completing the entire intelligent analysis process.
[0065] Through the above implementation method, the advantages of two independent optimal models can be successfully combined, and their respective shortcomings can be overcome through a carefully designed fusion and correction mechanism, achieving unprecedented high-precision and robust fully automated analysis of spinal X-ray images.
[0066] Example 3 Figure 4 This is a schematic diagram of the structure of a skeletal image recognition device provided in Embodiment 3 of the present invention, as shown below. Figure 4 As shown, the skeletal image recognition device includes: an image recognition module 410, an object construction module 420, an object optimization module 430, and an object determination module 440, wherein... Image recognition module 410 is used to acquire bone recognition points and bone recognition regions in the original bone image; The object construction module 420 is used to construct candidate bone objects based on the spatial relationship between the bone recognition points and the bone recognition area; The object optimization module 430 is used to perform candidate optimization processing on the bone recognition points and the bone recognition regions based on the candidate bone objects to obtain bone key points and bone object regions. The object determination module 440 is used to determine the recognition object image in the original skeleton image based on the skeleton key points and the skeleton object region.
[0067] The technical solution of this embodiment obtains bone recognition points and bone recognition regions in the original bone image, and constructs candidate bone objects based on the spatial relationship between the bone recognition points and bone recognition regions. Based on the candidate bone objects, the bone recognition points and bone recognition regions are optimized to obtain bone key points and bone object regions. Thus, the recognition object image in the original bone image is determined based on the bone key points and bone object regions. This solves the problem of low reliability and accuracy of existing bone image recognition technologies, realizes mutual verification and fusion of different types of recognition results, and greatly improves the reliability and accuracy of bone image recognition, providing a rich and reliable data foundation for downstream data analysis and disease diagnosis.
[0068] Optionally, the object construction module 420 may include: a range acquisition unit for acquiring the regional spatial range of the skeleton recognition area; and an object determination unit for constructing the skeleton recognition points within the regional spatial range and the skeleton recognition areas corresponding to the regional spatial range as the candidate skeleton objects.
[0069] Optionally, the skeleton recognition region is a skeleton segmentation mask; the range acquisition unit can specifically be used to: calculate the target bounding rectangle range of the skeleton segmentation mask as the spatial range of the region.
[0070] Optionally, the object optimization module 430 may include: a candidate point unit, used to determine the bone recognition points constituting the candidate bone object as candidate key points; a region optimization unit, used to perform region optimization processing on the bone recognition region constituting the candidate bone object based on the candidate key points to obtain the bone object region; and a point optimization unit, used to determine the bone key points among the candidate key points based on the bone object region.
[0071] Optionally, the region optimization unit may include: a point information subunit, used to obtain the number of key points and the key point category of the candidate skeleton object based on the candidate key points corresponding to the candidate skeleton object; a target object subunit, used to determine the candidate skeleton object whose number of key points and key point category meet the point association condition as the target skeleton object; and a region determination subunit, used to determine the skeleton recognition region corresponding to the target skeleton object as the skeleton object region.
[0072] Optionally, the region optimization unit may further include: a boundary optimization subunit, used to obtain the key point positions of each of the bone key points corresponding to the target bone object; and to perform region boundary optimization on the bone object region corresponding to the target bone object based on the key point positions.
[0073] Optionally, the boundary optimization subunit can be used to: generate a target polygon range with each of the key point positions as vertices; and optimize the region boundary of the skeleton object region according to the target polygon range so that the skeleton object region is optimized into a target polygon region corresponding to the target polygon range.
[0074] The skeletal image recognition device provided in the embodiments of the present invention can execute the skeletal image recognition method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.
[0075] Example 4 Figure 5 This is a schematic diagram of the structure of a skeletal image recognition device provided in Embodiment 4 of the present invention, as shown below. Figure 5 As shown, the skeletal image recognition device includes a processor 510, a memory 520, an input device 530, and an output device 540; the number of processors 510 in the skeletal image recognition device can be one or more. Figure 5 Taking a processor 510 as an example; the processor 510, memory 520, input device 530, and output device 540 in the skeletal image recognition device can be connected via a bus or other means. Figure 5 Taking the example of a connection between China and Israel via a bus.
[0076] The memory 520, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the skeletal image recognition method in this embodiment of the invention (e.g., the image recognition module 410, object construction module 420, object optimization module 430, and object determination module 440 in the skeletal image recognition device). The processor 510 executes various functional applications and data processing of the skeletal image recognition device by running the software programs, instructions, and modules stored in the memory 520, thereby realizing the aforementioned skeletal image recognition method.
[0077] The memory 520 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a given function; the data storage area may store data created based on terminal usage. Furthermore, the memory 520 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory, or other non-volatile solid-state storage device. In some instances, the memory 520 may further include memory remotely located relative to the processor 510, which can be connected to the skeletal imaging recognition device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
[0078] The input device 530 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the skeletal image recognition device. The output device 540 may include a display device such as a screen.
[0079] Example 5 Embodiment 5 of the present invention also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform a skeletal image recognition method, including: Obtain bone recognition points and bone recognition regions from the original skeletal image; Candidate bone objects are constructed based on the spatial relationship between the bone recognition points and the bone recognition areas; Based on the candidate skeleton objects, the skeleton recognition points and the skeleton recognition regions are optimized to obtain the skeleton key points and the skeleton object regions. The object image in the original skeletal image is determined based on the skeletal key points and the skeletal object region.
[0080] Of course, the computer-executable instructions provided in the embodiments of the present invention are not limited to the method operations described above, but can also perform related operations in the skeletal image recognition method provided in any embodiment of the present invention.
[0081] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.
[0082] It is worth noting that in the embodiments of the above-mentioned skeletal image recognition device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.
[0083] Although the present invention has been described in detail above with general descriptions, specific embodiments, and experiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.
Claims
1. A method for skeletal image recognition, characterized in that, include: Obtain bone recognition points and bone recognition regions from the original skeletal image; Candidate bone objects are constructed based on the spatial relationship between the bone recognition points and the bone recognition areas; Based on the candidate skeleton objects, the skeleton recognition points and the skeleton recognition regions are optimized to obtain the skeleton key points and the skeleton object regions. The object image in the original skeletal image is determined based on the skeletal key points and the skeletal object region.
2. The method according to claim 1, characterized in that, The step of constructing candidate skeleton objects based on the spatial relationship between the skeleton recognition points and the skeleton recognition region includes: Obtain the spatial range of the skeleton recognition area; The bone recognition points within the spatial range and the corresponding bone recognition regions within the spatial range are constructed as candidate bone objects.
3. The method according to claim 2, characterized in that, The skeleton recognition area is a skeleton segmentation mask; The acquisition of the spatial range of the skeleton recognition region includes: Calculate the target bounding rectangle range of the skeleton segmentation mask, and use it as the spatial range of the region.
4. The method according to claim 1, characterized in that, The step of performing candidate optimization processing on the bone recognition points and the bone recognition regions based on the candidate bone objects to obtain bone key points and bone object regions includes: The bone recognition points constituting the candidate bone object are identified as candidate key points; Based on the candidate key points, the bone recognition region constituting the candidate bone object is subjected to region optimization processing to obtain the bone object region; Based on the skeletal object region, the skeletal key points are determined from the candidate key points.
5. The method according to claim 4, characterized in that, The step of performing region optimization processing on the bone recognition region constituting the candidate bone object based on the candidate key points to obtain the bone object region includes: Based on the candidate key points corresponding to the candidate skeleton object, obtain the number of key points and the key point category of the candidate skeleton object; The candidate skeleton objects whose number of key points and key point categories meet the point association conditions are determined as the target skeleton objects; The bone recognition region corresponding to the target bone object is determined as the bone object region.
6. The method according to claim 5, characterized in that, After determining the candidate skeleton objects whose number of key points and key point categories meet the point association conditions as target skeleton objects, the method further includes: Obtain the key point positions of each bone key point corresponding to the target bone object; Based on the location of the key points, the region boundary of the bone object corresponding to the target bone object is optimized.
7. The method according to claim 6, characterized in that, The step of optimizing the region boundary of the skeleton object corresponding to the target skeleton object based on the key point positions includes: Generate a target polygon range with the positions of each of the aforementioned key points as vertices; The skeletal object region is optimized based on the target polygon range to optimize the skeletal object region into the target polygon region corresponding to the target polygon range.
8. A skeletal image recognition device, characterized in that, include: The image recognition module is used to acquire bone recognition points and bone recognition regions in the original bone image; The object construction module is used to construct candidate bone objects based on the spatial relationship between the bone recognition points and the bone recognition regions; The object optimization module is used to perform candidate optimization processing on the bone recognition points and the bone recognition regions based on the candidate bone objects, so as to obtain the bone key points and the bone object regions. The object determination module is used to determine the object image to be identified in the original skeletal image based on the skeletal key points and the skeletal object region.
9. A skeletal image recognition device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the skeletal image recognition method as described in any one of claims 1-7.
10. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the skeletal image recognition method as described in any one of claims 1-7.