A lighthouse guiding-based minimum cost skeleton detection method and system

Abstract

The application provides a lighthouse-guided minimum cost skeleton detection method and system, and relates to the technical field of computer vision and image processing. The application aims at the problems of structure fracture and topological incompleteness in the existing skeleton detection technology, utilizes a neural network to jointly predict a skeleton probability graph and a key point distribution, defines predicted connection points and breakpoints as lighthouses, and constructs a cost graph reflecting path connectivity cost; and the application performs minimum cost path search under the guidance of the lighthouses and connects fractured skeleton segments. The application can effectively utilize spatial distribution information of key points to guide skeleton repair, and significantly improves the connectivity and structural integrity of skeleton detection under complex background and posture change.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image processing technology, and in particular to a lighthouse-guided minimum-cost skeleton detection method and system. Background Technology

[0002] An object skeleton is an abstract representation of the shape of an object in an image. It is characterized by its simple structure and strong topological properties, and is widely used in fields such as action recognition and road extraction. With the development of deep learning, methods based on convolutional neural networks have become the mainstream for skeleton detection.

[0003] Most existing methods treat skeleton detection as a pixel-level binary classification problem, independently determining whether each pixel belongs to the skeleton. This pixel-level classification method ignores the overall structural continuity of the skeleton. When the object's pose changes significantly or there is occlusion, the skeleton probability map output by the network often responds weakly at connections or complex regions, resulting in broken and discontinuous skeletons. Although some existing techniques attempt to alleviate this problem by decomposing the skeleton into line segments or using magnetic flux fields, it is still difficult to solve the topological breakage problem caused by missing local detections while maintaining the skeleton's accuracy.

[0004] Therefore, designing a method that can utilize the structural features of an object to guide skeleton generation and effectively repair fractured areas is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] Purpose of the invention: To provide a minimum cost skeleton detection method and system based on lighthouse guidance, which solves the problems of discontinuity and easy breakage in skeleton detection in the prior art by introducing key points (endpoints and connection points) as lighthouse guidance and combining them with the minimum cost path search algorithm.

[0006] This invention proposes a minimum-cost skeleton detection method based on lighthouse guidance, comprising the following steps: S1. Obtain the target image to be detected, extract image features through a dual-track collaborative detection network, and output a skeleton probability map and a key point heatmap; the key point heatmap includes an endpoint heatmap and a connection point heatmap. S2. The skeleton probability map is refined to obtain an initial skeleton, and breakpoints are extracted on the initial skeleton. The breakpoints and connection points are defined as guiding lighthouses. S3. Construct a path cost map by combining the skeleton probability map and the key point heatmap; the cost of a pixel in the path cost map is negatively correlated with the probability that the pixel belongs to the skeleton or a key point. S4. Using the breakpoint as the search starting point, and the predicted connection point and other breakpoints as candidate lighthouses, search for the target lighthouse within the preset search area, and calculate the minimum cost path from the search starting point to the target lighthouse on the path cost map. When the minimum cost path meets the preset conditions, connect the search starting point and the target lighthouse to complete the skeleton repair.

[0007] As a preferred embodiment, the dual-track collaborative detection network in step S1 includes a skeleton branch and a point branch. Based on a Transformer encoder and a Transformer decoder, the Transformer encoder encodes the input multi-scale image features into global context features. The Transformer decoder uses a learnable query vector to query the encoded global context features to obtain a unified query feature. This unified query feature is equally divided into skeleton query features and point query features along the channel dimension, and is respectively input into the skeleton branch and the point branch. The skeleton branch outputs a skeleton probability map based on the skeleton query features; the point branch outputs an endpoint heatmap and a connection point heatmap based on the point query features.

[0008] As a preferred embodiment, extracting breakpoints on the initial skeleton specifically includes: The initial skeleton is traversed, and the neighborhood degree of all pixels in the initial skeleton is calculated. Pixels with a neighborhood degree of 1 are marked as candidate endpoints. The Euclidean distance between the candidate endpoints and the endpoints predicted in step S1 is calculated. Candidate endpoints with an Euclidean distance greater than a preset matching threshold are determined as breakpoints caused by non-endpoints and included in the guide beacon set.

[0009] As a preferred embodiment, in step S3, the path cost graph C is constructed using the following formula:

[0010] In the formula, For cost mapping function; This is a skeleton probability map; For endpoint heatmaps; Heatmap of connection points; The coefficients are used to balance the weights.

[0011] As a preferred embodiment, step S4 specifically includes: Determine the skeleton extension direction of the current breakpoint, and set a fan-shaped search area with the breakpoint as the center and along the extension direction; Within the fan-shaped search area, search for other lighthouses in the set of guiding lighthouses as candidate targets; Calculate the path cost from the breakpoint to each of the candidate targets on the path cost graph; Select the path with the minimum cost as the candidate path; The candidate paths are constrained according to preset constraints to obtain the optimal repair path; The optimal repair path is used to locally increase the probability along the path and merge it into the skeleton to obtain an updated skeleton probability map. The updated skeleton probability map is joined with the optimal repair path, and a skeleton refinement algorithm is applied to obtain the final target skeleton.

[0012] As a preferred embodiment, the preset constraints include: The average pixel cost of the optimal repair path is less than a preset first threshold. The maximum single-point pixel cost on the optimal repair path is less than a preset second threshold. The average response value of the heatmap region through which the optimal repair path passes is greater than a preset third threshold.

[0013] As a preferred embodiment, the optimal repair path is used to locally increase the probability along the path and incorporate it into the skeleton, specifically including: For the optimal repair path For each pixel P on the skeleton probability map, update its position in the skeleton probability map according to the following formula. The values ​​in:

[0014] In the formula, Representing the skeleton probability map Quantiles Take a value between 0 and 1; This represents the response value of the keypoint heatmap at pixel P.

[0015] Furthermore, this invention proposes a lighthouse-guided minimum-cost skeleton detection system, which includes: The backbone feature extraction module is used to receive the input target image, extract its multi-scale hierarchical features, and complete the multi-scale feature fusion through the feature pyramid network, outputting fused features with consistent scale. The dual-track collaborative detection module is connected to the output of the backbone feature extraction module. It has built-in parallel skeleton branches and point branches, and synchronously outputs skeleton probability map, key point heat map and corresponding key point coordinates based on the fused features. The lighthouse construction module is connected to the output of the dual-track collaborative detection module. It is used to refine the skeleton probability map to generate an initial skeleton, identify the breakpoints in the initial skeleton, and merge the breakpoints with the detected connection points to construct a set of guiding lighthouses. The path cost construction module is connected to the output of the dual-track collaborative detection module and the lighthouse construction module, respectively, and is used to fuse the skeleton probability map and the key point heat map to generate the path cost map. The minimum cost path repair module is connected to the output of the lighthouse construction module and the path cost construction module, respectively. It is used to perform minimum cost path search starting from the breakpoint and under the constraint of the guiding lighthouse, to perform skeleton fusion and repair on the effective paths, and output the final continuous target skeleton.

[0016] As a preferred solution, in the dual-track collaborative detection module, both the skeleton branch and the point branch use a multilayer perceptron as the prediction head, and the key point heatmap output by the point branch is input to the path cost construction module.

[0017] Furthermore, the present invention also proposes a computer-readable storage medium storing at least one executable instruction, which, when executed on an electronic device, causes the electronic device to perform the aforementioned lighthouse-guided minimum cost skeleton detection method.

[0018] Compared with the prior art, the present invention has the following significant advantages: (1) This invention innovatively proposes a dual-track collaborative detection architecture, which simultaneously realizes skeleton prediction and endpoint and connection point key point detection. It uses the spatial distribution of key points to guide the network to focus on the connection points and endpoint regions where the skeleton is prone to breakage. During the training phase, it strengthens the feature learning of vulnerable regions, thereby improving the accuracy of skeleton detection.

[0019] (2) The present invention proposes a lighthouse-guided skeleton repair mechanism, which uses the break point of the skeleton fracture and the connection point of the key nodes of the structure as a guiding lighthouse. Combined with the minimum cost path search algorithm, it realizes the adaptive global repair of the fractured skeleton, effectively solves the core problem of skeleton topological fracture in existing methods, and generates a finer skeleton with stronger continuity and significantly improved topological integrity.

[0020] (3) While ensuring high-precision detection, the present invention has excellent reasoning efficiency, which can meet the application requirements of real-time vision tasks. Attached Figure Description

[0021] Figure 1 This is a flowchart of the method of the present invention.

[0022] Figure 2 This is an example diagram of the network structure and image of the present invention.

[0023] Figure 3 A schematic diagram of the endpoints and connection points as a lighthouse.

[0024] Figure 4 A schematic diagram of the minimum cost path repair process guided by the lighthouse. Detailed Implementation

[0025] In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be practiced without one or more of these details. In other instances, certain technical features well-known in the art have not been described in order to avoid obscuring the invention.

[0026] Example 1: This embodiment provides a lighthouse-guided minimum-cost skeleton detection method. By introducing key points (endpoints and connectors) as lighthouse guidance and combining them with a minimum-cost path search algorithm, it solves the problems of discontinuity and easy breakage in skeleton detection in existing technologies. See Figure 1 The specific implementation process is as follows: Multi-scale hierarchical features of the input image are extracted by a backbone feature extraction network, and then fused by a feature pyramid network (FPN) to obtain scale-consistent fused features. The fused features are input into the dual-track collaborative detection network, which simultaneously outputs a skeleton probability map and a key point heatmap. The key point heatmap includes an endpoint heatmap and a connection point heatmap. Calculate the joint loss value, perform end-to-end iterative optimization of the dual-track collaborative detection network based on the joint loss value, and use the optimized network to complete the skeleton and key point prediction of the input image; Threshold segmentation and skeleton refinement are performed on the skeleton probability map output by the network to obtain an initial skeleton with a single pixel width. Breakpoints in the initial skeleton are identified, and the breakpoints and detected connection points are defined together as guide beacons to construct a set of guide beacons. A path cost map is constructed by combining the skeleton probability map and the key point heatmap. The cost of a pixel in the path cost map is negatively correlated with the joint probability that the point belongs to the skeleton, endpoint or connection point. Starting from the breakpoint, and considering the remaining lighthouses in the guide lighthouse set as candidate targets, a minimum cost path search is performed within the preset search area. The minimum cost paths that meet the constraints are then fused and repaired to obtain a complete and continuous target skeleton.

[0027] Furthermore, see Figure 2 , Figure 2 In the image, (a) shows the network structure, and (b) shows an example image. The specific steps for extracting multi-scale hierarchical features from the input image using a backbone feature extraction network include: The input RGB image is fed into the Swing Transformer backbone network to obtain the features f2, f3, f4, and f5 of different layers. Features f2, f3, f4, and f5 are input into the Feature Pyramid Network (FPN) for fusion processing, which combines the detailed information of shallow features with the semantic information of deep features to obtain a scale-consistent fused feature F, which is used for subsequent feature scale transformation.

[0028] Furthermore, the steps of inputting the fused feature F into a Deformable DETR network (which consists of only 6 layers in both the decoder and encoder) to extract the target's skeleton feature FS and point feature FP, and then feeding them into a dual-track collaborative detection network to output the skeleton probability map and keypoint heatmap respectively, specifically include: Features f3, f4, and f5 are input into the Deformable DETR module, where they are encoded using a Transformer structure containing an encoder and a decoder. This process uncovers long-distance dependencies among the features and outputs the corresponding query features. The query features are divided into two parts along the channel dimension: skeleton query features and key point query features, which correspond to the skeleton branch and the key point branch, respectively. Multilayer perceptron (MLP) is used to map skeleton query features, and then Einstein summation is performed on the fused features F to align the feature dimensions. After processing by a linear classifier and a sigmoid activation function, a skeleton probability map with values ​​ranging from 0 to 1 is output. At the same time, the coordinates of all skeleton points are predicted through an auxiliary supervision task. The prediction results are only used for loss calculation during the training phase and do not participate in the inference process. The key point query features of the multilayer perceptron (MLP) are used for mapping, and the fusion features F are combined for feature alignment. After processing by a linear classifier, a dual-channel key point heatmap is output. The first channel is the endpoint heatmap with values ​​ranging from 0 to 1, and the second channel is the connection point heatmap with values ​​ranging from 0 to 1. Peak extraction and non-maximum suppression (NMS) are performed on the endpoint heatmap and connection point heatmap respectively to filter redundant detection results, and the final endpoint coordinate set and connection point coordinate set are obtained.

[0029] Furthermore, the step of identifying breakpoints in the initial skeleton and defining the breakpoints and detected connection points together as the guiding beacon specifically includes: The skeleton probability map is subjected to global threshold binarization to filter out low-confidence background noise, and then thinned by the Lee skeletonization algorithm to obtain the initial skeleton S0 with a single pixel width. Traverse all pixels of the initial skeleton S0, calculate the 8-neighborhood degree of each pixel, and mark the pixels with an 8-neighborhood degree of 1 as candidate endpoints P; Remove candidate endpoints whose Euclidean distance to points in the detected endpoint coordinate set is less than the preset matching radius. The remaining candidate endpoints are the breakpoints, and thus the breakpoint set D is constructed. The set of breakpoints D and the set of connection point coordinates J are merged to obtain the set of guiding lighthouses L, where the breakpoints and connection points together serve as lighthouse nodes for guiding skeleton repair.

[0030] Furthermore, the steps for constructing a path cost map by combining the skeleton probability map and the key point heatmap specifically include: fusing the skeleton probability map, endpoint heatmap, and connection point heatmap to construct a joint confidence field and balancing the confidence weights of the skeleton region and the key point region; The joint confidence field is mapped to a path cost map through a cost mapping function. The cost mapping function establishes a negative correlation between joint confidence and path cost value. That is, the higher the joint confidence of a pixel, the lower the corresponding path cost value; the lower the joint confidence, the higher the corresponding path cost value. This ensures that the high-confidence region of the skeleton, the endpoint and connection point regions are the low-cost priority regions for path search.

[0031] Furthermore, the steps of performing minimum cost path search within the preset search area and then fusing and repairing the skeleton of the minimum cost path that meets the constraints specifically include: For each breakpoint in the breakpoint set, the tangent extension direction of the skeleton branch is determined by fitting the tangent of the skeleton branch. A fan-shaped search region is constructed with the breakpoint as the center and the extension direction as the central axis. Within the fan-shaped search area, candidate lighthouses in the set of guiding lighthouses are selected. The minimum path cost from the breakpoint to each candidate lighthouse on the path cost graph is calculated based on the Dijkstra algorithm. The candidate lighthouse with the minimum path cost is selected as the target lighthouse, and the corresponding path is the candidate repair path. To suppress false connections and erroneous paths, a triple constraint is set for candidate repair paths. A path is considered valid only if all of the following conditions are met: the average cost of the path as a whole does not exceed a preset first threshold, constraining the path to be located in a high-confidence region of the skeleton; the maximum cost of a single point on the path does not exceed a preset second threshold, preventing the path from crossing high-cost background regions; and the average response value of the heatmap of the connection points on the path is not lower than a preset third threshold, ensuring consistency between the path and the guidance of key points. For effective repair paths, update the pixel confidence of the skeleton probability map along the path, improve the skeleton confidence of the broken area, and ensure that the confidence of the path area is not lower than the basic confidence of the skeleton as a whole. The updated skeleton probability map is then subjected to Lee skeletonization again to ensure the single-pixel width and connectivity of the skeleton, resulting in a topologically complete continuous target skeleton.

[0032] Furthermore, the formula for calculating the joint loss value is as follows:

[0033] In the formula, For skeleton branch loss, The loss is for the critical branch.

[0034] Skeleton branch loss The calculation formula is:

[0035] In the formula, The pixel-level classification loss for the skeleton probability map is combined with Dice loss and Sigmoid Focal loss to address the extreme imbalance between positive and negative samples in skeleton detection. The formula is as follows:

[0036] In the formula, S is the skeleton probability map prediction value. For the skeleton truth map, , These are the loss weighting coefficients; The regression loss for skeleton point coordinates is L1 loss, used to help improve skeleton localization accuracy. The formula is:

[0037] In the formula, For the predicted skeleton point coordinates, For the true coordinates of the skeleton points, The number of skeleton points. For loss weighting coefficients, Represents the L1 loss function; The Sigmoid Focal loss is used for skeleton point classification.

[0038] Keypoint branch loss The calculation formula is:

[0039] In the formula, For the keypoint heatmap regression loss, a weighted mean squared error (MSE) loss is used to enhance the network's ability to learn sparse keypoints. The formula is as follows:

[0040] In the formula, These are the predicted values ​​from the key point heatmap. For the true value of the key point heatmap, The weights are positive and negative samples, with a positive sample weight of 1 and a negative sample weight of 0.05. The loss for keypoint coordinate regression is L1 loss. The Sigmoid Focal loss is used for keypoint classification.

[0041] Example 2: This embodiment discloses a lighthouse-guided minimum-cost skeleton detection method. It employs a strategy combining dual-track collaborative detection and minimum-cost path repair, dividing the skeleton detection task into two parallel and collaborative sub-tasks: skeleton curve prediction and key point detection. The spatial distribution of key points guides the network to focus on easily broken and difficult-to-detect regions. Simultaneously, a lighthouse-guided minimum-cost path strategy is used, employing detected connection points and skeleton breakpoints as lighthouses to construct a path cost graph and adaptively repair broken skeletons along the minimum-cost path. Specifically, the implementation process of the lighthouse-guided minimum-cost skeleton detection method includes the following steps: Step 1: Obtain the RGB image containing the target object, the corresponding skeleton ground truth map, the endpoint ground truth coordinates, and the connection point ground truth coordinates to construct a skeleton detection dataset.

[0042] To comprehensively cover different detection scenarios and improve the model's generalization ability, this example uses four publicly available datasets commonly used in the field to construct training and testing sets: SK-LARGE, SK-SMALL, WH-SYMMAX, and SYM-PASCAL. The SK-LARGE dataset contains 1491 natural images sampled from the MS COCO dataset; the SK-SMALL dataset is a simplified subset of SK-LARGE, containing 506 images; the WH-SYMMAX dataset contains 328 single-horse target images; and the SYM-PASCAL dataset contains 1435 images covering multiple targets, partial occlusion, and complex backgrounds. Each image in the dataset corresponds to a single-pixel-width skeleton ground truth map, endpoint ground truth coordinates, and connector ground truth coordinates. During training, data augmentation operations such as random cropping, horizontal flipping, multi-scale scaling, and color dithering are performed on the input images.

[0043] Step 2: Extract features at different levels from the RGB target image using the feature extraction network.

[0044] Furthermore, the steps for extracting features at different levels from the RGB target image using a feature extraction network include: The RGB target image is input into the Swin-Base backbone network to obtain features f1, f2, f3, f4, and f5 output from different layers of the Swin-Base network.

[0045] Feature f1 is the side output feature of the first stage in the Swin-Base network, feature f2 is the side output feature of the second stage in the Swin-Base network, feature f3 is the side output feature of the third stage in the Swin-Base network, feature f4 is the side output feature of the fourth stage in the Swin-Base network, and feature f5 is the side output feature of the fifth stage in the Swin-Base network.

[0046] Features f2, f3, f4, and f5 are input into a Feature Pyramid Network (FPN) for fusion processing to obtain scale-consistent fused features. Its tensor dimension is C×H×W, where H and W are the height and width of the input image, respectively.

[0047] Features f3, f4, and f5 are input into the Deformable DETR module to generate query features that include global context information. Where Q is the number of query vectors and C is the channel dimension; the query features are equally divided into skeleton query features along the channel dimension. Features of point queries These are used for feature learning in subsequent skeleton and point branches, respectively.

[0048] Step 3: Input the fused features and query features into the parallel skeleton branch and point branch respectively, and simultaneously predict and output the skeleton probability map and the dual-channel key point heatmap.

[0049] Furthermore, the steps of inputting the fused features and query features into the parallel skeleton branch and point branch respectively, and simultaneously predicting and outputting the skeleton probability map and the dual-channel keypoint heatmap include: Query skeleton features The input is fed into a multilayer perceptron (MLP) for feature transformation, and then combined with the fused features output from the feature pyramid network. Tensor dimension transformation and feature fusion are performed using Einstein's summation convention to obtain a skeleton feature map of dimension Q×H×W. This skeleton feature map is then input into a linear classifier, processed by the Sigmoid activation function, and the output value ranges from [value missing]. Skeleton probability map Its calculation expression is:

[0050] Meanwhile, the skeleton branch predicts the two-dimensional coordinates of all skeleton points through an auxiliary regression task. This prediction result is only used for auxiliary supervision during the training phase and does not participate in the calculation during the inference phase.

[0051] Point query features The input is fed into a multilayer perceptron (MLP) for feature transformation, and then combined with the fused features output from the feature pyramid network. Dimensional reshaping is performed to obtain a dense feature representation of dimension Q×H×W; this dense feature representation is then input into a linear classifier, and the output value ranges from... Dual-channel key point heatmap, where the first channel is the endpoint heatmap. The second channel is a heat map of the connection point. .

[0052] Reverse endpoint heatmap Heatmap of connection points Local peaks are extracted separately, and then non-maximum suppression (NMS) and thresholding are applied to obtain the final set of endpoint coordinates. With the set of coordinates of the connecting points Its calculation expression is:

[0053] In the formula, This is a local maximum extraction function. This is a nonmaximum suppression and threshold filtering function.

[0054] Meanwhile, the point branch predicts the two-dimensional coordinates of the endpoints and connection points through the auxiliary regression task. This prediction result is only used for auxiliary supervision during the training phase and does not participate in the calculation during the inference phase.

[0055] The skeleton branch and the point branch together constitute the dual-track collaborative detection network. The backbone network of the dual-track collaborative detection network preferentially adopts the Swin-Base architecture. The encoder and decoder layers of the Deformable DETR module are both set to 6 layers, and the Gaussian kernel radius σ used to generate key point heatmap labels is set to 10.

[0056] Step 4: Calculate the joint loss value, perform end-to-end iterative optimization of the dual-track collaborative detection network based on the joint loss value, and use the optimized dual-track collaborative detection network to obtain the target skeleton probability map and key point heatmap.

[0057] The end-to-end training process of the dual-track collaborative detection network employs a joint loss function for iterative optimization. The expression for the joint loss function is as follows:

[0058] In the formula, This is the total loss value. The total loss of the skeleton branches, The total loss for the point branch.

[0059] Furthermore, the total loss of skeleton branches It consists of pixel-level classification loss, coordinate-level regression loss, and classification auxiliary loss, and its expression is:

[0060] In the formula, For the pixel-level classification loss of the skeleton probability map, a weighted combination of Dice loss and Sigmoid Focal loss is used to solve the problem of extreme imbalance between positive and negative samples in skeleton detection. The L1 regression loss is used to improve the positioning accuracy of skeleton points. The Sigmoid auxiliary loss is used to classify skeleton points, thereby enhancing the classification accuracy of skeleton points.

[0061] Furthermore, the total loss of the point branch It consists of heatmap regression loss, coordinate-level regression loss, and classification loss, and its expression is:

[0062] In the formula, The weighted mean squared error (MSE) loss of the keypoint heatmap is used to mitigate the imbalance between positive and negative samples caused by keypoint sparsity. The L1 regression loss is used to calculate the coordinates of the endpoints and connection points, which is used to improve the positioning accuracy of key points. The Sigmoid Focal loss is used to classify key points, distinguishing between endpoints and connection points.

[0063] In this embodiment, a cosine learning rate scheduling strategy is adopted, and the initial learning rate is set to... The model is trained for 200 iterations, with the Sym-PASCAL dataset having an extended training period of 300 iterations.

[0064] Step 5: Perform threshold segmentation and refinement on the skeleton probability map to obtain the initial skeleton. Identify breakpoints in the initial skeleton and construct a set of guiding beacons by combining the predicted connection points. (See...) Figure 3 As shown.

[0065] Further, the steps of thresholding and refining the skeleton probability map to obtain the initial skeleton, identifying breakpoints in the initial skeleton, and constructing a set of guiding beacons by combining the predicted connection points include: Probability map of skeleton Global thresholding is performed to obtain a binary skeleton image. Then, the binary skeleton image is thinned using the Lee skeletonization algorithm to obtain an initial skeleton with a single pixel width. The expression is:

[0066] In the formula, τ is the global threshold.

[0067] Traverse the initial skeleton For all pixels, calculate the 8-neighborhood connectivity degree of each pixel, and mark the pixels with an 8-neighborhood degree of 1 as candidate endpoints.

[0068] Calculate the set of candidate endpoints and predicted endpoints for each endpoint. The Euclidean distance between all endpoints is used. If the Euclidean distance between a candidate endpoint and any predicted endpoint is less than the preset matching radius r, then the candidate endpoint is determined to be a true endpoint and discarded. The remaining candidate endpoints are determined to be breakpoints caused by skeleton fracture, forming a breakpoint set B, expressed as:

[0069] In the formula, For the initial skeleton The set of all candidate endpoints, is the predicted endpoint, and r is the matching radius, which takes a value of 10 pixels.

[0070] Compare the breakpoint set B with the predicted connection point set. Merge and build a set of guiding beacons to provide global guidance information for subsequent skeleton repair.

[0071] Step 6: Construct a joint confidence field by fusing the skeleton probability map and the key point heatmap, and generate a path cost map through the cost mapping function.

[0072] Furthermore, the steps of constructing a joint confidence field by fusing the skeleton probability map and the key point heatmap, and generating a path cost map through a cost mapping function, include: A joint confidence field is constructed by combining the skeleton probability map and the dual-channel keypoint heatmap. This joint confidence field is then mapped to a path cost map using a cost mapping function. The expression for this mapping is:

[0073] In the formula, C represents the path cost graph. The cost mapping function is used to map the joint probability value to the path cost value, realizing a negative correlation between the probability value and the cost value. That is, the higher the probability that a pixel belongs to the skeleton or key point, the lower the path cost value. To balance the weighting coefficients, This is a skeleton probability map. For endpoint heatmaps, This is a heatmap of the connection points.

[0074] Step 7: Using the breakpoints in the lighthouse set as the starting point for the search, perform a minimum cost path search guided by the lighthouses, select the optimal repair path, and complete the update of the skeleton probability map.

[0075] Furthermore, the steps of performing a minimum-cost path search guided by lighthouses, using the breakpoints in the lighthouse set as the starting point, selecting the optimal repair path, and completing the skeleton probability graph update include: For each breakpoint in the breakpoint set Fit the local tangent direction of the skeletal branch in which it is located, with the breakpoint With the center of the circle and the tangent extension direction as the central axis, construct a circle with radius R and half-angle R. The fan-shaped search area; Within the fan-shaped search area, filter out other breakpoints and connection points in the lighthouse set besides the current breakpoint as candidate connection targets.

[0076] See Figure 4 As shown, the current breakpoint is calculated on the path cost graph C. The path with the minimum cost to each candidate target is selected as the candidate repair path. Its expression is:

[0077] In the formula, To the initial skeleton Under structural constraints, the path cost graph C starts from the breakpoint To candidate target The minimum cost path computation function.

[0078] Candidate repair paths are evaluated using multi-dimensional constraints. The path that satisfies all constraints is determined to be the optimal repair path. The constraint expressions are as follows:

[0079] In the formula, The average cost of all pixels along the path. This is the average cost threshold, used to ensure that the overall path lies within the high-probability region of the skeleton; The maximum single-point cost of pixels on the path. This is the maximum cost threshold, used to prevent paths from crossing high-cost background regions; This represents the average response value of the heatmap of the connection points along the path. This is the heatmap response threshold, used to ensure consistency between the path and the lighthouse guidance.

[0080] All pixels along the optimal repair path are merged into the skeleton probability map to increase the skeleton probability value of pixels along the path. The update expression is as follows:

[0081] In the formula, p represents the pixel on the optimal repair path. skeleton probability map of Quantiles This represents the response value of the keypoint heatmap at pixel p.

[0082] Step 8: Perform secondary Lee skeleton refinement on the updated skeleton probability map to obtain the final continuous and topologically complete single-pixel target skeleton.

[0083] To further demonstrate the accuracy and topological integrity of the lighthouse-guided minimum cost skeleton detection method described in this invention, the following example is provided.

[0084] The comparison methods used include MIL, HED, RCF, FSDS, SRN, LSN, Hi-Fi, GeoSkeletonNet, DeepFlux, DeepFlux-E, MSB+, AdaLSN, Blumnet, NDASPP, and ProMask. The evaluation metrics include F-measure and single-image inference runtime (seconds). F-measure is the comprehensive accuracy metric for skeleton detection, calculated from precision P and recall. This metric comprehensively reflects the skeleton detection accuracy and completeness of the model. Runtime is the inference processing time for a single image, reflecting the detection efficiency of the model. The comparison results are shown in Table 1. It can be concluded that the F-measure of the present invention reaches 0.822 on the SK-LARGE dataset, which is comparable to the current state-of-the-art BlumNet method. On the SK-SMALL, WH-SYMMAX, and SYM-PASCAL datasets, it improves upon BlumNet by 4.5%, 3.9%, and 13.6%, respectively. At the same time, the inference time per image is controlled at 0.051 seconds, which is on par with mainstream methods such as ProMask. While maintaining high computational efficiency, it significantly improves the accuracy and topological integrity of skeleton detection, thus demonstrating the dual advantages of the present invention in terms of detection accuracy and real-time performance.

[0085] Table 1. Comparison results of the present invention with other methods

[0086] The logical ideas behind the methods disclosed in the above embodiments can be implemented, in whole or in part, through software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions or computer programs.

[0087] When computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions according to the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. Semiconductor media can be solid-state drives (SSDs).

[0088] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0089] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0090] As described above, although the invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the invention itself. Various changes in form and detail may be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A minimum-cost skeleton detection method based on lighthouse guidance, characterized in that, Includes the following steps: S1. Obtain the target image to be detected, extract image features through a dual-track collaborative detection network, and output a skeleton probability map and a key point heatmap; the key point heatmap includes an endpoint heatmap and a connection point heatmap. S2. The skeleton probability map is refined to obtain an initial skeleton, and breakpoints are extracted on the initial skeleton. The breakpoints and connection points are defined as guiding lighthouses. S3. Construct a path cost map by combining the skeleton probability map and the key point heatmap; the cost of a pixel in the path cost map is negatively correlated with the probability that the pixel belongs to the skeleton or a key point. S4. Using the breakpoint as the search starting point, and the predicted connection point and other breakpoints as candidate lighthouses, search for the target lighthouse within the preset search area, and calculate the minimum cost path from the search starting point to the target lighthouse on the path cost map. When the minimum cost path meets the preset conditions, connect the search starting point and the target lighthouse to complete the skeleton repair.

2. The minimum-cost skeleton detection method based on lighthouse guidance according to claim 1, characterized in that, The dual-track collaborative detection network in step S1 includes a skeleton branch and a point branch. Based on a Transformer encoder and a Transformer decoder, the Transformer encoder encodes the input multi-scale image features into global context features. The Transformer decoder uses a learnable query vector to query the encoded global context features to obtain a unified query feature. The unified query feature is equally divided into skeleton query features and point query features along the channel dimension, and then input into the skeleton branch and point branch respectively. The skeleton branch outputs a skeleton probability map based on the skeleton query features; The point branch outputs endpoint heatmaps and connection point heatmaps based on the point query features.

3. The minimum-cost skeleton detection method based on lighthouse guidance according to claim 1, characterized in that, Extracting breakpoints on the initial skeleton specifically includes: The initial skeleton is traversed, and the neighborhood degree of all pixels in the initial skeleton is calculated. Pixels with a neighborhood degree of 1 are marked as candidate endpoints. The Euclidean distance between the candidate endpoints and the endpoints predicted in step S1 is calculated. Candidate endpoints with an Euclidean distance greater than a preset matching threshold are determined as breakpoints caused by non-endpoints and included in the guide beacon set.

4. The minimum-cost skeleton detection method based on lighthouse guidance according to claim 1, characterized in that, In step S3, the path cost graph C is constructed using the following formula: In the formula, For cost mapping function; This is a skeleton probability map; For endpoint heatmaps; Heatmap of connection points; The coefficients are used to balance the weights.

5. The minimum-cost skeleton detection method based on lighthouse guidance according to claim 3, characterized in that, Step S4 specifically includes: Determine the skeleton extension direction of the current breakpoint, and set a fan-shaped search area with the breakpoint as the center and along the extension direction; Within the fan-shaped search area, search for other lighthouses in the set of guiding lighthouses as candidate targets; Calculate the path cost from the breakpoint to each of the candidate targets on the path cost graph; Select the path with the minimum cost as the candidate path; The candidate paths are constrained according to preset constraints to obtain the optimal repair path; The optimal repair path is used to locally increase the probability along the path and merge it into the skeleton to obtain an updated skeleton probability map. The updated skeleton probability map is joined with the optimal repair path, and a skeleton refinement algorithm is applied to obtain the final target skeleton.

6. The minimum-cost skeleton detection method based on lighthouse guidance according to claim 5, characterized in that, The preset constraints include: The average pixel cost of the optimal repair path is less than a preset first threshold. The maximum single-point pixel cost on the optimal repair path is less than a preset second threshold. The average response value of the heatmap region through which the optimal repair path passes is greater than the preset third threshold.

7. The minimum-cost skeleton detection method based on lighthouse guidance according to claim 5, characterized in that, The optimal repair path is used to locally increase the probability along the path and merge it into the skeleton, specifically including: For the optimal repair path For each pixel P on the skeleton probability map, update its position in the skeleton probability map according to the following formula. The values ​​in: In the formula, Representing the skeleton probability map Quantiles Take a value between 0 and 1; This represents the response value of the keypoint heatmap at pixel P.

8. A minimum-cost skeleton detection system based on lighthouse guidance, characterized in that, include: The backbone feature extraction module is used to receive the input target image, extract its multi-scale hierarchical features, and complete the multi-scale feature fusion through the feature pyramid network, outputting fused features with consistent scale. The dual-track collaborative detection module is connected to the output of the backbone feature extraction module. It has built-in parallel skeleton branches and point branches, and synchronously outputs skeleton probability map, key point heat map and corresponding key point coordinates based on the fused features. The lighthouse construction module is connected to the output of the dual-track collaborative detection module. It is used to refine the skeleton probability map to generate an initial skeleton, identify the breakpoints in the initial skeleton, and merge the breakpoints with the detected connection points to construct a set of guiding lighthouses. The path cost construction module is connected to the output of the dual-track collaborative detection module and the lighthouse construction module, respectively, and is used to fuse the skeleton probability map and the key point heat map to generate the path cost map. The minimum cost path repair module is connected to the output of the lighthouse construction module and the path cost construction module, respectively. It is used to perform minimum cost path search starting from the breakpoint and under the constraint of the guiding lighthouse, to perform skeleton fusion and repair on the effective paths, and output the final continuous target skeleton.

9. The lighthouse-guided minimum-cost skeleton detection system according to claim 8, characterized in that, In the dual-track collaborative detection module, both the skeleton branch and the point branch use a multilayer perceptron as the prediction head, and the key point heatmap output by the point branch is input to the path cost construction module.

10. A computer-readable storage medium, characterized in that, The storage medium stores at least one executable instruction, which, when executed on an electronic device, causes the electronic device to perform the lighthouse-guided minimum-cost skeleton detection method as described in any one of claims 1 to 7.

Images