Methods, systems, media, and devices for instance segmentation and computation based on recursive hints

Abstract

This invention provides a method, system, medium, and device for instance segmentation and calculation based on recursive prompts, belonging to the field of industrial artificial intelligence technology. The method includes: performing primary instance segmentation processing on an input image to obtain at least one primary target object mask; generating a candidate prompt point set and a region bounding box prompt set based on the primary target object mask; filtering the candidate prompt point set to obtain a target prompt information set; performing recursive segmentation based on the target prompt information set and the region bounding box prompt set to obtain sub-part masks; and calculating geometric relationship data between each sub-part based on the sub-part masks. This invention effectively solves the problem of poor quality generation of target prompt points for complex contours, significantly improving the robustness of multi-level segmentation and the accuracy of geometric relationship calculation.

Description

Technical Field

[0001] This invention relates to the field of industrial artificial intelligence technology, specifically to a method, system, medium, and device for instance segmentation and calculation based on recursive prompts. Background Technology

[0002] In the fields of computer vision and industrial automation inspection, instance segmentation technology plays a crucial role. Its goal is not only to identify individual object instances in an image, but also to accurately depict the pixel-level boundaries of each instance. In recent years, deep learning-based segmentation models, especially those capable of interactive segmentation using cues such as points and bounding boxes (e.g., SAM, FastSAM), have made significant progress. These models can quickly generate high-quality segmentation masks based on simple user-provided prompts, providing a strong technological foundation for automated and intelligent processing workflows.

[0003] To achieve refined analysis of complex structural targets, the industry has begun exploring multi-level segmentation architectures. A typical existing technical solution is as follows: First, a basic segmentation model is used to perform a first-level segmentation of the input image, obtaining a mask for the primary target object. Then, an intelligent cue generation module analyzes the geometric features (such as contours and bounding boxes) of these primary masks and automatically generates a series of point or box cues. These cues then drive the same or another segmentation model for a second-level segmentation, thereby obtaining masks for the sub-parts that constitute the primary object. Finally, the geometric relationships (such as angles and distances) between the sub-parts are calculated based on these refined masks. The core idea of ​​this method is to decompose the complex overall segmentation task into hierarchical subtasks through recursive cue-driven processing.

[0004] However, through in-depth research and practice, the inventors have discovered that while the aforementioned prior art is ingeniously conceived, its intelligent prompt generation module has inherent defects when processing primary targets with internal spaces (such as ring gears or flanges) or highly irregular, jagged edge contours (such as broken metal parts or corroded surfaces). Specifically, the prompt generation strategy based on contour analysis (such as calculating convex hulls, extreme points, and curvature key points) in the prior art exposes serious problems: it indiscriminately generates a large number of candidate prompt points from the outer contour, inner contour, and every tiny jagged edge. This leads to an explosive increase in the number of prompt messages, which are filled with a large number of redundant, erroneous, and even contradictory noise points. This problem has serious consequences in practical applications: First, it causes a sharp decline in the performance of subsequent recursive segmentation modules. The segmentation model cannot accurately understand the segmentation intent due to receiving a large number of low-quality and contradictory prompts, resulting in the failure of secondary segmentation results or the generation of a large number of fragments. Second, the huge number of prompt points will unnecessarily exhaust computing resources and reduce system efficiency. Finally, the geometric relationship calculation module will output unreliable or even completely wrong geometric data based on the incorrect segmentation results, causing the final result of the entire automated inspection process to lose its reference value and fail to meet the needs of high-precision industrial inspection. Summary of the Invention

[0005] In view of this, it is necessary to provide a method, system, medium, and device for instance segmentation and calculation based on recursive hints, in order to solve the technical problems in the prior art that generate a large number of redundant, erroneous, or contradictory hint points, thereby causing a sharp decline in the performance of subsequent recursive segmentation, waste of computing resources, and unreliable geometric relationship calculation results.

[0006] To address the aforementioned technical problems, in a first aspect, the present invention provides an instance segmentation and calculation method based on recursive hints, comprising:

[0007] Perform primary instance segmentation on the input image to obtain at least one primary target object mask;

[0008] Based on the primary target object mask, generate a candidate cue point set and a region box cue set;

[0009] The target prompt information set is obtained by filtering the candidate prompt point set;

[0010] Based on the target prompt information set and the region box prompt set, recursive segmentation is performed to obtain sub-part masks;

[0011] The geometric relationship data between each sub-part is calculated based on the sub-part mask.

[0012] In one possible implementation, generating the candidate cue point set and the region box cue set based on the primary target object mask includes:

[0013] Extract the contour features of the primary target object mask, calculate convex hull feature points, extreme points or curvature key points based on the contour features, and generate the candidate prompt point set; the candidate prompt point set includes the convex hull feature points, the extreme points and the curvature key points;

[0014] Calculate the minimum bounding rectangle covering the primary target object, and divide the minimum bounding rectangle into m×n sub-regions to generate the region box hint set; m and n are both positive integers greater than 1.

[0015] In one possible implementation, obtaining the target prompt information set based on the candidate prompt point set includes:

[0016] Construct a multidimensional feature vector corresponding to each candidate prompt point. The multidimensional feature vector includes local curvature intensity features, local contour direction change rate features, local region contrast features, and contour hierarchy weight features.

[0017] Calculate the saliency score corresponding to each of the candidate prompt points based on the multidimensional feature vector;

[0018] Calculate the screening threshold based on the contour complexity index;

[0019] Target prompts are selected based on the saliency score; the target prompts are the candidate prompts whose saliency score is greater than or equal to the selection threshold.

[0020] Non-maximum suppression processing is performed on the target prompt point to eliminate redundant prompt points in the local neighborhood.

[0021] In one possible implementation, the contour complexity metric is calculated as follows:

[0022] Extract the set of contour points from the primary target object mask;

[0023] Calculate the perimeter and area of ​​the contour based on the set of contour points;

[0024] The contour complexity index is calculated based on the contour perimeter and the contour area.

[0025] ;

[0026] Where R_complexity is the contour complexity index, L is the contour perimeter, and A is the contour area.

[0027] In one possible implementation, calculating the geometric relationship data between the sub-parts based on the sub-part mask includes:

[0028] The principal component analysis algorithm is used to calculate the principal direction vector of each sub-part, and the angle between the principal direction vectors of any two sub-parts is calculated as the angular relationship data.

[0029] Calculate the centroid coordinates of each sub-part, and calculate the Euclidean distance between the centroids of any two sub-parts as distance relationship data;

[0030] Based on the horizontal and vertical differences in the centroid coordinates of each sub-part, the relative orientation relationship data is determined;

[0031] The geometric relationship data is obtained based on the angle relationship data, the distance relationship data, and the relative orientation relationship data.

[0032] One possible implementation also includes:

[0033] The shape similarity between any two sub-part masks is calculated based on shape invariant moments. The shape invariant moments are used to quantitatively describe the global geometric features of the sub-part masks and have translation, scaling and rotation invariance.

[0034] The relative area ratio between any two sub-parts is calculated based on the number of foreground pixels in each sub-part mask; the geometric relationship data also includes the shape similarity and the relative area ratio.

[0035] In one possible implementation, prior to performing primary instance segmentation on the input image, the process includes:

[0036] The input image is subjected to multi-scale Gaussian filtering noise reduction processing to obtain the filtered image;

[0037] Edge enhancement processing is performed on the filtered images at each scale;

[0038] Select the optimal processing scale based on the total energy of the edge response;

[0039] Morphological closing operations are performed on the contour of the primary target object mask based on the optimal scale image to fill contour breaks.

[0040] Secondly, the present invention also provides an instance segmentation and calculation system based on recursive hints, comprising:

[0041] The instance segmentation module is used to perform primary instance segmentation processing on the input image to obtain at least one primary target object mask;

[0042] The prompt generation module is used to generate a candidate prompt point set and a region box prompt set based on the primary target object mask;

[0043] The prompt filtering module is used to filter the candidate prompt point set to obtain the target prompt information set;

[0044] The recursive segmentation module is used to recursively segment the target prompt information set and the region box prompt set to obtain sub-part masks;

[0045] The relationship calculation module is used to calculate the geometric relationship data between each sub-part based on the sub-part mask.

[0046] Thirdly, the present invention also provides an electronic device, including a memory and a processor, wherein,

[0047] The memory is used to store programs;

[0048] The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the instance segmentation and calculation method based on recursive hints described in any of the above implementations.

[0049] Fourthly, the present invention also provides a computer-readable storage medium for storing a computer-readable program or instruction, wherein when the program or instruction is executed by a processor, it is able to implement the steps in the instance segmentation and calculation method based on recursive hints described in any of the above implementations.

[0050] The beneficial effects of this invention are as follows: The instance segmentation and calculation method based on recursive prompts provided by this invention firstly effectively filters out a large number of redundant and erroneous prompts generated by internal holes and noisy edges by introducing a prompt point filtering mechanism based on multi-dimensional feature fusion and dynamic thresholds. This ensures the high quality and high relevance of the prompt information on which recursive segmentation depends, fundamentally solving the problem of the sharp decline in the performance of secondary segmentation when processing complex contours in existing technologies. Furthermore, through multi-angle geometric relationship calculations, it not only outputs basic relative position (distance, orientation) and direction (angle) information, but also provides advanced geometric metrics such as shape similarity and relative area ratio. This provides comprehensive data support from perception to cognition for industrial inspection, greatly improving the depth and reliability of automated analysis. Furthermore, by accurately filtering prompt points, it avoids the waste of computational resources caused by the segmentation model processing a large number of invalid prompts, significantly shortening the processing time. This makes the method suitable for industrial application scenarios with real-time requirements while ensuring high accuracy. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a schematic flowchart of an embodiment of the instance segmentation and calculation method based on recursive hints provided by the present invention;

[0053] Figure 2 For the present invention Figure 1 A schematic diagram of an embodiment of S102;

[0054] Figure 3 For the present invention Figure 1 A schematic diagram of an embodiment of S103;

[0055] Figure 4 For the present invention Figure 1 A schematic diagram of an embodiment of S105;

[0056] Figure 5 For the present invention Figure 1 Another embodiment of the process diagram of S105;

[0057] Figure 6 A schematic diagram of an embodiment of the instance segmentation and calculation system based on recursive hints provided by the present invention;

[0058] Figure 7 A schematic diagram of an embodiment of the electronic device provided by the present invention. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0060] In the description of the embodiments of the present invention, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0061] The terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0062] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0063] This invention provides a method, system, medium, and device for instance segmentation and calculation based on recursive hints, which are described below.

[0064] Figure 1 This is a schematic flowchart of an embodiment of the instance segmentation and calculation method based on recursive hints provided by the present invention, as shown below. Figure 1 As shown, the instance segmentation and calculation method based on recursive hints includes:

[0065] S101. Perform primary instance segmentation on the input image to obtain at least one primary target object mask.

[0066] It should be noted that the primary target object refers to the overall target object identified and segmented from the input image through first-level instance segmentation. A pre-configured, cue-based segmentation model (such as SAM or FastSAM) is used to perform primary instance segmentation on the input image to be analyzed, obtaining one or more primary target object masks. First, the training parameters of the instance segmentation model are configured. Using an instance segmentation dataset containing a large number of pixel-level annotations, point cues, bounding box cues, and their corresponding ground truth masks are used to form training samples. A combination loss function of cross-entropy loss and Dice loss is used, and the model parameters are optimized through backpropagation to train the cue-based instance segmentation model, such as SAM or FastSAM. In practical applications, the input image (e.g., an industrial inspection image) is input to the trained segmentation model, and the model outputs one or more binary masks of foreground objects, i.e., the primary target object masks.

[0067] S102. Generate a set of candidate cue points and a set of region box cue points based on the primary target object mask.

[0068] It should be noted that: The contour point sequence of the primary target object mask is extracted. Based on this contour, its convex hull vertex set is calculated using the GrahamScan algorithm; the leftmost, rightmost, topmost, and bottommost extreme points are obtained by traversing the contour point coordinates; the discrete curvature is obtained by calculating the rate of change of the angle between the line segment vectors of the preceding and following contour points, and non-maximum suppression is used to filter curvature extreme points. The above convex hull vertices, extreme points, and curvature key points are merged to form a candidate cue point set. The minimum bounding rectangle covering the primary target object mask is calculated. This rectangle is uniformly divided into m parts horizontally and n parts vertically, generating m×n grid-like sub-region boxes. These region boxes constitute the region box cue set, providing prior spatial guidance for subsequent segmentation.

[0069] S103. Filter the candidate prompt point set to obtain the target prompt information set.

[0070] S104. Based on the target prompt information set and the region box prompt set, perform recursive segmentation to obtain sub-part masks.

[0071] S105. Calculate the geometric relationship data between each sub-part based on the sub-part mask.

[0072] It should be noted that the filtered and optimized target cue information (points) and the set of region bounding box cue are simultaneously input into the segmentation model used in step S101. At this point, the model uses the image region corresponding to the primary target object mask as the processing object and performs second-level segmentation using these high-quality cue. The model accurately locates the key feature points of the sub-parts based on the point cue and uses the bounding box cue to constrain the local search range of the segmentation, finally outputting one or more more refined sub-part masks that constitute the primary target. Based on all sub-part masks, the geometric relationship data between each sub-part is calculated.

[0073] In summary, the instance segmentation and calculation method based on recursive prompts provided in this embodiment of the invention firstly, by introducing a prompt point filtering mechanism based on multi-dimensional feature fusion and dynamic thresholds (S103), effectively filters out a large number of redundant and erroneous prompts generated by internal holes and noisy edges, ensuring the high quality and high relevance of the prompt information on which recursive segmentation (S104) depends, fundamentally solving the problem of the sharp decline in the performance of secondary segmentation when processing complex contours in existing technologies. Furthermore, through multi-angle geometric relationship calculation (S105), not only is basic relative position (distance, orientation) and direction (angle) information output, but also advanced geometric metrics such as shape similarity and relative area ratio are provided. This provides comprehensive data support from perception to cognition for industrial inspection, greatly improving the depth and reliability of automated analysis. Furthermore, by accurately filtering prompt points (S103), the waste of computational resources caused by the segmentation model processing a large number of invalid prompts is avoided, significantly shortening the processing time, making this method suitable for industrial application scenarios with real-time requirements while ensuring high accuracy.

[0074] In some embodiments of the present invention, such as Figure 2 As shown, step S102 includes:

[0075] S201. Extract the contour features of the primary target object mask, calculate convex hull feature points, extreme points or curvature key points based on the contour features, and generate the candidate prompt point set; the candidate prompt point set includes the convex hull feature points, the extreme points and the curvature key points.

[0076] It should be noted that highly representative geometric feature points are automatically extracted from the contour of the primary target object mask as candidate cue points to drive secondary segmentation. A boundary tracing algorithm (such as the Suzuki85 algorithm) is used to process the primary target object mask, obtaining one or more ordered closed contour point sets Q={Q1, Q2, ..., QM} composed of pixel coordinates. For targets with internal holes, both the outer and inner contours are extracted simultaneously. The GrahamScan algorithm is applied to the obtained contour point set Q to calculate its convex hull. This algorithm first finds the point with the smallest y-coordinate as the base point, then sorts the remaining points according to their polar angles relative to the base point, and finally determines the concavity / convexity of the points through stack operations and vector cross products, filtering out the vertex set Pconvex={Pc1, Pc2, ..., PcK} that constitutes the smallest convex polygon. These convex hull feature points describe the overall structural shape of the outermost layer of the target object. The entire contour point set Q is traversed, and points with the largest or smallest coordinate values ​​in the four principal axes are found through linear comparison. Specifically, the leftmost point Pleft (the point with the smallest x-coordinate), the rightmost point Pright (the point with the largest x-coordinate), the topmost point Ptop (the point with the smallest y-coordinate), and the bottommost point Pbottom (the point with the largest y-coordinate) are calculated and recorded. The extreme point set Pextreme = {Pleft, Pright, Ptop, Pbottom} defines the bounding box of the target object, providing information on the object's extreme positions in space. A discrete curvature estimation algorithm is used to capture the local detailed features of the contour. For each point on the contour... Take its forward point and backward point (s is the preset step size, for example, s=5), construct two vectors and Calculate the approximate curvature at this point. k k :

[0077] .

[0078] After calculating the curvature of all points, one-dimensional non-maximum suppression is applied: for each point, it is only identified as a curvature keypoint and added to the set Pcurvature if its curvature value is the maximum value in its local neighborhood (e.g., s points before and after it) and is greater than a preset curvature threshold Tcurvature. Curvature keypoints accurately identify local salient features such as corners, inflection points, and depressions on the contour. The union of the convex hull feature point set Pconvex, the extreme point set Pextreme, and the curvature keypoint set Pcurvature obtained above is used to generate the final candidate cue point set.

[0079] S202. Calculate the minimum bounding rectangle covering the primary target object, and divide the minimum bounding rectangle into m×n sub-regions to generate the region box prompt set; m and n are both positive integers greater than 1.

[0080] It should be noted that: for each primary target object mask, its geometric and structural features are automatically analyzed, and based on this, one or more refined prompts are generated to drive secondary segmentation. That is, the contour of the primary target object mask is extracted, and the convex hull, extreme points or curvature key points of the contour are calculated as candidate prompts.

[0081] First, the geometric boundary of the primary target object mask is calculated to obtain the minimum bounding rectangle covering the target. ,in These represent the coordinates of the bottom left vertex of the rectangle. This represents the coordinates of the top-right corner vertex. Based on the rectangle's boundary, its width and height are further calculated as follows:

[0082] ;

[0083] Where W represents the span along the horizontal direction and H represents the span along the vertical direction.

[0084] After obtaining the parameters of the circumscribed rectangle, the rectangle is divided evenly into... Each subregion generates a set of subregions. Each sub-region Defined as:

[0085] ;

[0086] in, Indicates the horizontal partition index. This represents the vertical segmentation index. This formula allows us to obtain the coordinate range of each sub-region, ensuring that the set of sub-regions completely covers the entire circumscribed rectangle in space, and that there is no overlap or omission between different sub-regions.

[0087] Ultimately, all sub-regions The output is used as bounding box cues to provide region-level guidance for subsequent recursive segmentation. These bounding box cues can provide uniform spatial partitioning in complex targets, enabling subsequent segmentation models to extract more refined sub-parts within local areas, thereby improving the completeness and accuracy of instance segmentation.

[0088] In this embodiment, the generated candidate cue point set comprehensively describes the target contour from three different dimensions: geometric topology (convex hull points), spatial range (extreme points), and local details (curvature key points). This multi-feature fusion strategy ensures that the cue information covers all levels from macroscopic structure to microscopic features, providing a rich and complementary driving force for subsequent segmentation and avoiding the problem of incomplete cueing caused by relying on a single feature. Preliminary screening using mature computer vision algorithms (Graham Scan, discrete curvature estimation, etc.) generates a set of higher-quality and more representative candidate cue points. This lays a good foundation for the subsequent core "cue saliency screening" step, reducing its computational burden and improving the overall quality of the final optimized cue points. Furthermore, the region bounding box cue set divides the target object into uniform grid regions. This division provides clear spatial guidance for the segmentation model, enabling it to systematically scan various local regions within the target, effectively avoiding the problem of missed sub-parts that may occur when relying solely on point cueing. It is particularly beneficial for discovering sub-parts with indistinct contour features but fixed spatial locations, thereby significantly improving the completeness and recall of instance segmentation. This method combines point cues (precise positioning and strong directionality) with bounding box cues (clear scope and broad coverage). In subsequent recursive segmentation, point cues are responsible for "activating" sub-parts at key locations, while bounding box cues define the "stage" for the search. This point-bounding box collaboration mechanism enables the system to generate high-quality segmentation results stably and reliably even when faced with complex structures, noise interference, or internal voids, greatly enhancing the robustness and practicality of the entire method in real-world industrial scenarios.

[0089] In some embodiments of the present invention, such as Figure 3 As shown, step S103 includes:

[0090] S301. Construct a multi-dimensional feature vector corresponding to each candidate prompt point. The multi-dimensional feature vector includes local curvature intensity features, local contour direction change rate features, local region contrast features, and contour hierarchy weight features.

[0091] It should be noted that in the calculation of local curvature intensity, candidate cue points are first obtained from the contour curve. and its two adjacent points and These points all originate from the pixel sequences extracted via the sub-part mask boundaries in the aforementioned steps. During calculation, adjacent vectors are constructed respectively. and The curvature intensity at that point is obtained using the three-point curvature formula:

[0092] ;

[0093] in, Indicates the coordinates of the candidate suggestion point. , All are nearby points on the contour curve. The symbol × represents the cross product operation of two-dimensional vectors. Indicates its quantity value, This represents the magnitude of the vector. Specifically, The directed area representing the triangular region formed by three points. , and These represent the Euclidean distances between adjacent points. This formula measures the degree of curvature of a local curve using the ratio of area to side length. When the numerator is relatively large and the denominator is finite, the local curvature intensity... A higher value indicates that the geometric changes at that location are more drastic, effectively reflecting the salience of local feature points.

[0094] In the calculation of the rate of change of local contour direction, candidate prompt points are first selected on the contour curve. and obtain its neighboring points. and .

[0095] Then construct the forward line segment vector. and backward line segment vector Next, calculate the direction angles of the two vectors respectively: .

[0096] in Given the arctangent function of a two-dimensional vector, it accurately gives the polar angle of the vector in the plane. Then, the rate of change of the local contour direction is obtained: If the value is greater than Then by calculation Normalize it to Range. In the formula and The differences in direction angles between the forward and backward line segments are described separately. This indicates the degree of inflection of the local contour at the candidate cue point. A larger value indicates a more abrupt change in direction at that location, effectively capturing the significant turning points of the curve.

[0097] In the process of calculating the contrast of local regions, candidate cue points are first identified in the original image. And construct two regions with that point as the center, one of which has a radius of... circular inner area Secondly, the outer annular region surrounding the circle. Extract the grayscale values ​​of all pixels within each region, denoted as I(p). Then calculate the average grayscale value between the two regions. and Furthermore, the gray-level variance of the region is calculated. For the internal region, the variance calculation formula is:

[0098] ;

[0099] in, Indicates the region The number of pixels in the inner area This represents the average gray level of the region. For the outer annular region, the formula for calculating the gray level variance is:

[0100] ;

[0101] in, Indicates the region The number of pixels in the inner area This represents the average gray level of the region. Finally, the local region contrast is calculated. .

[0102] in, and These two values ​​respectively reflect the grayscale fluctuations in the central and outer regions of the candidate suggestion point. The larger the difference between the two values, the stronger the grayscale fluctuation of the point. It is more likely to be located at a significant image boundary, thus highlighting the salience of local features.

[0103] S302. Calculate the saliency score corresponding to each of the candidate prompt points based on the multidimensional feature vector.

[0104] S303. Calculate the screening threshold based on the contour complexity index;

[0105] S304. Target prompts are selected based on the saliency score; the target prompts are the candidate prompts whose saliency score is greater than or equal to the selection threshold.

[0106] It should be noted that: for the candidate prompt point set, the salience score of each candidate prompt point is calculated, and adaptive filtering is performed based on the salience score to filter out redundant and erroneous prompt points, and an optimized refined prompt information set (including the target prompt point) is output; the salience score is calculated based on at least one feature of the candidate prompt point, including local curvature intensity, local contour direction change rate, local region contrast, and contour level weight.

[0107] The set of input candidate suggestions is denoted as . Each candidate cue point Corresponding to its outline after repair Two-dimensional coordinates on During the processing, the first step is to address each candidate suggestion point. Construct a four-dimensional feature vector Among them, local curvature intensity It is achieved by calculating the radius of curvature of the contour segment in the neighborhood of the candidate cue point. And take its reciprocal to obtain, that is This is used to measure the degree of curvature of the profile at that point.

[0108] Contour Hierarchy Weight Used to distinguish the importance of "outer contour hint points" and "inner hole / jawline edge hint points" (outer contours better reflect the structure of sub-parts, while inner holes are prone to redundancy). Defines the contour nesting depth. Outer contour d i =0, first layer internal void d i =1, second layer internal cavity d i =2, and so on. Based on the contour inclusion relationship, then:

[0109] ;

[0110] Where λ=0.5 is the nesting depth attenuation coefficient, and the weight decreases by approximately 60% for every 1 increase in depth; γ=0.1 is the contour length adjustment coefficient. The outer perimeter is for The perimeter of the contour. This formula ensures that cue points for outer contours and long contours have higher weights, while cue points for inner holes and short jagged contours have lower weights.

[0111] Local contour direction change rate It is achieved by extracting the direction angles of two adjacent contour lines before and after the candidate cue point. And calculate the absolute value of their difference to obtain, that is This is used to characterize the intensity of the local directional abrupt change at that point. Local region contrast. It is obtained by the difference in gray-level variance of pixels inside and outside the local neighborhood of the candidate prompt point, denoted as ,in This represents the pixel grayscale variance within the mask containing the point. This represents the variance of pixel grayscale values ​​in the adjacent regions outside the mask, reflecting the structural saliency of that point at the image level. Then, a weighted scoring function is defined based on these features. ,in The preset weighting coefficients control the contribution of three features—curvature, direction change, and contrast—to the scoring.

[0112] After obtaining the saliency scores of all candidate cue points, to avoid the failure problem caused by a uniform fixed threshold under contours of different complexity, a dynamic threshold mechanism based on the overall contour complexity is introduced. The contour complexity index is defined as follows: Where L is the perimeter of the current contour, reflecting the boundary length, and A is the area of ​​the contour, reflecting the size of the closed region. This ratio characterizes the regularity of the contour; a larger value indicates a more irregular contour. The filtering threshold is set to... Where α is a scaling factor that controls the amplification effect of complexity on the threshold. For each candidate cue point, when its significance score... If the value is less than the screening threshold T, the point is considered redundant or invalid and is filtered out, retaining only those points that meet the criteria. point.

[0113] After initial screening based on scores and thresholds, further removal of duplicate points within local neighborhoods is necessary. Specifically, a neighborhood of radius r is defined on the contour curve of each candidate suggestion point, and non-maximum suppression is performed within this neighborhood. Only the point with the highest significance score is retained, while other points with lower scores are discarded. This ensures that the output suggestion point set is spatially uniformly distributed and avoids dense redundancy. Finally, the optimized, refined suggestion information set is obtained. This provides a reliable and concise driving input for subsequent recursive suggestion segmentation. Ultimately, all sub-regions... The output is used as bounding box cues to provide region-level guidance for subsequent recursive segmentation. These bounding box cues can provide uniform spatial partitioning in complex targets, enabling subsequent segmentation models to extract more refined sub-parts within local areas, thereby improving the completeness and accuracy of instance segmentation.

[0114] S305. Perform non-maximum suppression processing on the target prompt point to eliminate redundant prompt points in the local neighborhood.

[0115] It should be noted that in the non-maximum suppression process, the input is the set of candidate cue points filtered by dynamic thresholding. and its corresponding significance score set Each candidate cue point Represents the two-dimensional coordinate position on the contour, saliency score This is derived from the weighted scoring function in the previous step. First, the neighborhood radius is determined based on the total contour length L, defined as... This ratio ensures that the neighborhood range adapts to the contour size, avoiding excessive local point density on long contours while maintaining sufficient differentiation on short contours.

[0116] After the neighborhood range is determined, for each point in the candidate suggestion point set Find all points that satisfy the Euclidean distance. :

[0117] ;

[0118] These points are considered The local neighbors. Among these neighbors, if any one of them exists... Significance score Strictly greater than rating ,Right now Then it is believed If a point is not the optimal point within its neighborhood, it is marked as a suppressed point (i.e., a redundant cue point) and removed. This mechanism ensures that only the highest-scoring point is retained within each local neighborhood, effectively avoiding multiple redundant cue points in the same area.

[0119] After performing the above checks on all points in the set in sequence, the final output is an optimized set consisting of the unsuppressed points:

[0120] ;

[0121] This means that all points in the candidate suggestion point set after dynamic filtering threshold are evaluated sequentially. If there is no candidate suggestion point pj with a score sj greater than its score si in the neighborhood of a candidate suggestion point pi, then pi is retained. Finally, these unsuppressed pi points form the optimized and refined suggestion information set. This result set guarantees the sparsity of the local spatial distribution and the uniqueness of the score significance, providing a stable and effective point input for recursive cue segmentation.

[0122] In this embodiment, a multi-dimensional feature vector (S301) integrating geometric features (curvature, orientation variation), image features (regional contrast), and topological features (hierarchical weights) is constructed to provide a comprehensive and quantitative description of each candidate cue point. This multi-feature fusion evaluation system replaces traditional single or empirical judgment criteria, making the saliency assessment of cue points more scientific, reliable, and interpretable. By introducing a dynamic threshold mechanism linearly related to the contour complexity index (S303), the screening strategy can intelligently adapt to target instances of different complexity. For simple, smooth contours, a lower threshold is used to avoid over-filtering effective points; for highly irregular, multi-hole complex contours, the threshold is automatically increased for more aggressive noise suppression. This adaptability greatly enhances the robustness and universality of the method in diverse industrial scenarios. Through a two-level screening architecture of "dynamic threshold initial screening" (S304) and "non-maximum suppression fine screening" (S305), redundant and erroneous points are effectively eliminated from the massive and noisy initial candidate cue points. The final output set of target cue information significantly improves the representativeness and reliability of points while ensuring uniform spatial distribution. This fundamentally solves the problem of decreased performance in secondary segmentation caused by poor cue point quality under complex contours. Through meticulous screening, this application ensures that the cue information driving secondary segmentation is of high quality and high relevance, laying a solid foundation for obtaining accurate refined sub-part masks and indirectly guaranteeing the accuracy and reliability of the final geometric relationship calculation results, meeting the stringent requirements of high-precision industrial inspection.

[0123] In some embodiments of the present invention, the contour complexity index is calculated in the following manner:

[0124] Extract the set of contour points from the primary target object mask;

[0125] Calculate the perimeter and area of ​​the contour based on the set of contour points;

[0126] The contour complexity index is calculated based on the contour perimeter and the contour area.

[0127] ;

[0128] Where R_complexity is the contour complexity index, L is the contour perimeter, and A is the contour area.

[0129] It should be noted that in the calculation of the overall complexity index, the contour of the primary target object mask is first extracted to obtain a set of contour points arranged in order. , where each point This represents a pixel coordinate on the contour. Based on this sequence of points, the contour perimeter is calculated by accumulating the Euclidean distances between adjacent points segment by segment, using the following formula:

[0130] ;

[0131] in , is used to measure the spatial straight-line distance between adjacent points, and the total boundary length L is obtained by summing the cumulative distances.

[0132] After obtaining the perimeter, Green's theorem is used to calculate the area enclosed by the contour. The formula is: When k=M, let This ensures that the outline is closed at both ends. The formula calculates the directed area of ​​the point sequence and takes the absolute value to obtain the actual area A of the closed region, thus accurately reflecting the spatial occupancy of the primary target object.

[0133] After obtaining the perimeter L and area A, calculate the overall complexity index. The numerator of this indicator The denominator represents the square of the boundary length. This represents the ideal proportional relationship between the area and circumference of a circular shape. For a regular circle, The theoretical value is 1, but when the contour has jagged edges, depressions, or irregular heights, the perimeter relatively increases while the area relatively decreases, causing the value of this index to increase. Therefore... It can quantitatively characterize the complexity of the contour shape. The larger the value, the more complex and irregular the boundary. This result is used in step 3 to construct a dynamic threshold, thereby achieving adaptive screening of the saliency score.

[0134] In this embodiment, the classic geometric metric of isoperimeter ratio is introduced into the filtering mechanism of recursive cue segmentation. This metric, Rcomplexity, provides a stable, dimensionless numerical description of contour complexity with clear geometric meaning, replacing traditional methods that rely on subjective experience or single features (such as the number of concave and convex points), making the assessment of shape complexity more scientific and objective. Furthermore, the calculation of the contour complexity metric relies only on basic geometric properties (perimeter and area), and the calculation process is simple and efficient, requiring no complex iterations or transformations.

[0135] In some embodiments of the present invention, such as Figure 4 As shown, step S105 includes:

[0136] S401. Calculate the principal direction vector of each sub-part based on the principal component analysis algorithm, and calculate the angle between the principal direction vectors of any two sub-parts as angular relationship data.

[0137] It should be noted that during the calculation of the angular relationship data, the input is the refined sub-part mask set obtained from step 4. For each sub-part mask First, based on the corresponding primary target object mask or refined sub-part mask, pixels identified as foreground are selected to extract all pixels within the foreground region (see the paper "Geometric Feature Prompting of Image Segmentation Models"), forming a point set. ,in This represents the coordinates of a pixel in a two-dimensional plane. Next, the centroid coordinates of this point set are calculated. ,in This represents the number of elements in the point set. and These represent the average positions of all points in the horizontal and vertical directions, respectively.

[0138] After obtaining the centroid, construct the covariance matrix of the point set. :

[0139] ;

[0140] The matrix This is used to measure the dispersion of a point set in different directions. By performing eigenvalue decomposition on the covariance matrix, two eigenvalues ​​and their corresponding eigenvectors are obtained, where the eigenvector corresponding to the largest eigenvalue is represented as:

[0141] ;

[0142] As the principal direction vector of the sub-part mask, it is used to describe the main extension direction of the region in space.

[0143] After obtaining the principal direction vectors of all sub-parts, the directional angles between different sub-parts are further calculated. For any two sub-parts... and The included angle is defined as:

[0144] ;

[0145] in Represents the vector dot product. and These represent the magnitudes of the vectors, and the sign of the absolute value ensures that the angle between the directions is between zero and ninety degrees. The final result includes... As angular relationship data output, it is used to characterize the relative orientation features between each refined sub-part, thereby providing a reliable quantitative indicator for subsequent geometric relationship analysis.

[0146] S402. Calculate the centroid coordinates of each sub-part, and calculate the Euclidean distance between the centroids of any two sub-parts as distance relationship data.

[0147] It should be noted that the input in the calculation of the distance relationship data is the refined sub-part mask set obtained in step 4. For each sub-part mask Extract the coordinates of all pixels within its foreground region to form a point set. Calculate the centroid coordinates based on this point set. ;

[0148] in, The numerator represents the number of foreground pixels in this sub-part, and the numerator is the sum of the values ​​of all pixels on the x and y axes, ultimately yielding the centroid. It is used to depict the center position of a sub-part in space.

[0149] After obtaining the centroids of all sub-parts, the Euclidean distance between the centroids of any two sub-parts is further calculated. For each sub-part... and Its centroid distance is defined as ;

[0150] in, and The centroids and The coordinates. This formula quantifies the straight-line distance between two centroids by using the sum of squares and the square root, accurately reflecting the relative distances between sub-parts.

[0151] After calculating the distance matrix for all pairings in the subset, the complete distance matrix is ​​obtained, consisting of all... The distance relationship data is constructed and output. This data complements the angle relationship data in step 5, representing both the relative orientation of different sub-parts in space and their relative position and distribution, thus providing a complete quantitative basis for the comprehensive analysis of geometric relationships.

[0152] S403. Determine the relative orientation relationship data based on the horizontal and vertical differences in the centroid coordinates of each sub-part.

[0153] It should be noted that in the calculation of the relative orientation relationship data, the input is the refined sub-part mask set obtained from step 4. For any two sub-part masks and First, based on their respective refined sub-part masks, pixels identified as foreground are selected to extract all pixels within the foreground region, forming point sets. and in and These represent the coordinates of pixels in the two sub-parts in the two-dimensional plane. Next, the centroid coordinates of the two point sets are calculated:

[0154] For Ui, the centroid coordinates are ( )= ∑ ∈Ui ;

[0155] For Uj, the centroid coordinates are ( , )= ∑ ∈Uj ;

[0156] in and Let represent the number of elements in the two point sets respectively. , , , These represent the average positions of the corresponding point sets in the horizontal and vertical directions, respectively. After obtaining the centroid coordinates of the two sub-parts, the coordinate difference is calculated to characterize the spatial position offset: Horizontal difference: Vertical difference: Based on the sign characteristic of the coordinate difference, the relative orientation of the two sub-parts is determined: a sign function sign(·) is defined, which returns 1 when the parameter > 0 and returns 0 when the parameter < 0. 1. When the parameter is 0, it returns 0. The relative orientation can then be represented as: All that was ultimately obtained As a relative orientation data output, it is used to characterize the spatial arrangement features between each refined sub-part, thereby providing intuitive orientation quantification indicators for subsequent scenarios.

[0157] S404. Obtain the geometric relationship data based on the angle relationship data, the distance relationship data, and the relative orientation relationship data.

[0158] In this embodiment, a complete description of the spatial relationships of sub-parts is achieved for the first time in post-processing of instance segmentation through three complementary geometric dimensions: angle, distance, and orientation. Angular relationships reveal the directional alignment characteristics of components, distance relationships quantify the spatial distribution density of components, and orientation relationships provide intuitive positional arrangement information. This multi-dimensional fusion description system overcomes the limitations of single geometric relationships, providing unprecedentedly rich information for industrial visual analysis. Furthermore, the principal direction extraction method based on principal component analysis (S401) can accurately capture the actual extension direction of sub-parts, unaffected by local contour noise. Stable positional relationship measurement is provided through centroid distance calculation (S402), and statistically based geometric parameter estimation ensures the robustness and accuracy of the measurement results, fully meeting the stringent measurement requirements for dimensional and positional tolerances in industrial inspection.

[0159] In some embodiments of the present invention, such as Figure 5 As shown, step S105 further includes:

[0160] S501. Calculate the shape similarity between any two sub-part masks based on shape-invariant moments. The shape-invariant moments are used to quantitatively describe the global geometric features of the sub-part masks and have translation, scaling and rotation invariance.

[0161] It should be noted that: sub-part shape similarity Used to quantify the shape consistency of two sub-parts, calculated based on the shape invariant moment Hu:

[0162] For each sub-part mask Extract 7 Hu moments Calculated using the moment function: ,in (for normalized central moments)

[0163] Construct the weighted Hu moment vector ,in To suppress the excessive influence of large Hu moments;

[0164] Shape similarity formula:

[0165]

[0166] in, For vector dot product, Let be the vector magnitude. The exponent term is used to penalize the difference in Hu moments. The smaller the difference, the closer the similarity is to 1.

[0167] S502. Calculate the relative area ratio between any two sub-parts based on the number of foreground pixels in each sub-part mask; the geometric relationship data also includes the shape similarity and the relative area ratio.

[0168] It should be noted that: relative area ratio Used to quantify the size relationship of sub-parts:

[0169] ;

[0170] in, The number of foreground pixels, ε=0.001 is the area difference adjustment coefficient, the smaller the area difference, the closer the ratio is to The greater the difference, the closer the ratio approaches 0, highlighting the sub-parts with abnormally large sizes. The final result, including sub-part masks and geometric relationship data, is generated and output. The geometric relationship data includes angles, distances, relative orientations, sub-part shape similarity, and relative area ratios.

[0171] In this embodiment, the use of shape invariant moments enables translational, scaling, and rotational invariance descriptions of the global geometric features of sub-parts. This ensures that shape comparisons are unaffected by object posture and size, laying a solid foundation for shape classification and anomaly detection. The relative area ratio calculation not only considers basic area proportions, but also allows this indicator to sensitively capture dimensional anomalies. In industrial inspection, it can effectively identify defective parts with non-compliant dimensions, significantly improving the accuracy and reliability of quality control. Furthermore, through the quantitative output of shape similarity and relative area ratio, the system can automatically identify process problems such as misassembly and omissions in product assembly, while also detecting shape degradation caused by wear and deformation. This intelligent analytical capability significantly reduces the subjectivity and labor intensity of manual inspection, improving the automation level of the production process and the efficiency of quality control.

[0172] This application constructs a multi-dimensional feature vector that integrates local curvature intensity, local contour direction change rate, and local region contrast. A weighted scoring function (not the main invention, but part of an improved candidate cue point saliency scoring and adaptive screening algorithm, which is one of the main inventions) is used to comprehensively quantify and evaluate each candidate cue point generated by contour analysis. A dynamic threshold T, positively correlated with contour complexity, is then introduced for initial filtering, and non-maximum suppression technology is used to remove redundant points within the contour neighborhood. The improved algorithm achieves the technical effect of accurately identifying and retaining feature points that best represent the sub-structure, effectively filtering out a large number of redundant and erroneous cue points generated by internal holes and noisy edges, fundamentally solving the problem of a sharp decline in secondary segmentation performance under complex contours. Furthermore, a three-point curvature estimation algorithm is used to calculate local curvature intensity to capture sharp features; the difference in direction angles between preceding and following line segments is used to obtain the local contour direction change rate to characterize abrupt changes in direction; and the difference in gray-level variance between the inner and outer regions around a point is used to obtain local region contrast to confirm the existence of the true boundary. The technical effect of this method is to provide a robust and interpretable quantitative scoring criterion for each candidate cue point, replacing traditional single or experience-based judgment standards and significantly improving the quality of cue point generation. Using the contour compactness ratio, a macro-geometric metric, as an input variable for threshold adjustment, the threshold increases linearly with the complexity index. The technical effect of this strategy is that the stringency of the screening mechanism can adapt to target instances of varying complexity: for smooth and simple contours, the threshold is lower to avoid over-filtering valid points; for highly irregular and porous complex contours, the threshold is automatically increased for more aggressive noise suppression, thus achieving intelligent and universal processing strategies.

[0173] In some embodiments of the present invention, step S100 includes, before the initial instance segmentation processing of the input image, the following:

[0174] S010. Perform multi-scale Gaussian filtering noise reduction processing on the input image to obtain the filtered image;

[0175] S020. Perform edge enhancement processing on the filtered images at each scale;

[0176] S030. Select the optimal processing scale based on the total energy of the edge response;

[0177] S040. Perform morphological closing operation on the contour of the primary target object mask based on the optimal scale image to fill contour breaks.

[0178] It should be noted that, due to the potential noise in the input image, typical of industrial scenes, directly extracting the contour can easily introduce false edges, leading to redundancy in subsequent candidate cue points. Therefore, adaptive multi-scale preprocessing is performed on the initial target object mask and the original input image. This adaptive multi-scale preprocessing specifically includes the following steps:

[0179] S010, or multi-scale Gaussian filter noise reduction, is used for noise reduction of the input image. Constructing a multi-scale Gaussian kernel set ,in σ is a scaling parameter, ranging from 0.5 to 3.0. The Gaussian kernel formula is:

[0180] ;

[0181] Filter the original image: , This represents the convolution operation. Multi-scale Gaussian filtering denoising can suppress noise in images to varying degrees while preserving different levels of contour information.

[0182] S020 stands for Multi-Scale Edge Enhancement: This involves filtering the wavefront image at each scale. The edge response is calculated using the Laplace operator:

[0183] The Laplacian operator can highlight areas of abrupt grayscale changes in an image, enhance edge response, and provide clearer boundary information for subsequent contour extraction.

[0184] S030, or Optimal Scale Selection, involves calculating the total energy of the edge response at each scale and selecting the scale with the highest energy as the optimal scale. And obtain the optimal scale image. This selection mechanism ensures that noise interference is suppressed to the greatest extent while preserving significant edges.

[0185] ;

[0186] S040, or contour restoration: based on optimal scale images. The contour of the primary target object mask output in step S101 is analyzed. A morphological closing operation is performed, using a 3×3 rectangular structuring element, first dilating and then eroding to fill in minor breaks and holes in the contour while maintaining the overall shape of the contour, resulting in a repaired complete contour. , as input for S201 contour analysis.

[0187] In this embodiment, the collaborative processing of multi-scale Gaussian filtering and edge enhancement effectively overcomes common problems in industrial scene images, such as noise interference and uneven illumination. An adaptive optimal scale selection mechanism ensures that the system can automatically select the most suitable processing granularity for different image content, achieving the best balance between preserving realistic edges and suppressing noise, providing high-quality contour information for subsequent processing. Furthermore, through morphological closing operations, it can intelligently fill in minute breaks and gaps in the contour, forming a complete and continuous closed contour. This processing fundamentally avoids errors in prompt point generation caused by contour breaks, such as generating invalid prompt points pointing to the background area, ensuring the reliability of the input data for the subsequent intelligent prompt generation module. Furthermore, adaptive multi-scale preprocessing does not rely on fixed parameter settings but rather addresses various complex industrial scenarios through multi-scale analysis and adaptive selection. Whether processing parts with complex surface textures or corroded components with blurred edges, the system can automatically adjust to the optimal processing state through this preprocessing stage, greatly improving the engineering practical value of the method and its adaptability to different industrial environments. Furthermore, adaptive multi-scale preprocessing organically combines multi-scale analysis with morphological processing, solving key issues affecting subsequent analysis results at the forefront of image processing. This avoids complex error correction in the post-processing stage, improves the overall system processing efficiency, and meets the dual requirements of real-time performance and accuracy in industrial inspection.

[0188] To better implement the instance segmentation and calculation method based on recursive hints in the embodiments of the present invention, based on the instance segmentation and calculation method based on recursive hints, the corresponding method is as follows: Figure 6 As shown, this embodiment of the invention also provides an instance segmentation and calculation system 600 based on recursive hints. The instance segmentation and calculation system 600 based on recursive hints includes:

[0189] The instance segmentation module 601 is used to perform primary instance segmentation processing on the input image to obtain at least one primary target object mask.

[0190] The prompt generation module 602 is used to generate a candidate prompt point set and a region box prompt set based on the primary target object mask;

[0191] The prompt filtering module 603 is used to filter the candidate prompt point set to obtain a target prompt information set;

[0192] The recursive segmentation module 604 is used to perform recursive segmentation based on the target prompt information set and the region box prompt set to obtain sub-part masks;

[0193] The relation calculation module 605 is used to calculate the geometric relationship data between each sub-part based on the sub-part mask.

[0194] The instance segmentation and calculation system 600 based on recursive hints provided in the above embodiments can implement the technical solutions described in the above embodiments of the instance segmentation and calculation method based on recursive hints. The specific implementation principles of each module or unit can be found in the corresponding content in the above embodiments of the instance segmentation and calculation method based on recursive hints, and will not be repeated here.

[0195] like Figure 7 As shown, the present invention also provides an electronic device 700. The electronic device 700 includes a processor 701, a memory 702, and a display 703. Figure 7 Only some components of the electronic device 700 are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0196] In some embodiments, processor 701 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 702 or process data, such as the instance segmentation and calculation method based on recursive hints in this invention.

[0197] In some embodiments, processor 701 may be a single server or a group of servers. The server group may be centralized or distributed. In some embodiments, processor 701 may be local or remote. In some embodiments, processor 701 may be implemented on a cloud platform. In one embodiment, the cloud platform may include a private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, intranet, multi-cloud, etc., or any combination thereof.

[0198] In some embodiments, memory 702 may be an internal storage unit of electronic device 700, such as a hard disk or memory of electronic device 700. In other embodiments, memory 702 may also be an external storage device of electronic device 700, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on electronic device 700.

[0199] Furthermore, the memory 702 may include both internal storage units of the electronic device 700 and external storage devices. The memory 702 is used to store application software and various types of data installed on the electronic device 700.

[0200] In some embodiments, display 703 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 703 is used to display information from electronic device 700 and to display a visual user interface. Components 701-703 of electronic device 700 communicate with each other via a system bus.

[0201] In one embodiment, when the processor 701 executes the multi-level instance segmentation and geometric relationship processing program in the memory 702, the following steps can be implemented:

[0202] Perform primary instance segmentation on the input image to obtain at least one primary target object mask;

[0203] Based on the primary target object mask, generate a candidate cue point set and a region box cue set;

[0204] A set of target prompt information is obtained based on the candidate selection;

[0205] Based on the target prompt information set and the region box prompt set, recursive segmentation is performed to obtain sub-part masks;

[0206] The geometric relationship data between each sub-part is calculated based on the sub-part mask.

[0207] It should be understood that when the processor 701 executes the multi-level instance segmentation and geometric relationship processing program in the memory 702, in addition to the functions mentioned above, it can also perform other functions, as can be found in the description of the corresponding method embodiments above.

[0208] Furthermore, this embodiment of the invention does not specifically limit the type of electronic device 700 mentioned. Electronic device 700 can be a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, laptop computer, or other portable electronic device. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable electronic device can also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the invention, electronic device 700 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0209] Accordingly, embodiments of this application also provide a computer-readable storage medium for storing computer-readable programs or instructions. When the programs or instructions are executed by a processor, they can implement the steps or functions of the instance segmentation and calculation methods based on recursive hints provided in the above-described method embodiments.

[0210] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.), and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0211] The above provides a detailed description of the instance segmentation and calculation method, system, medium, and device based on recursive hints provided by this invention. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A method for instance segmentation and computation based on recursive cues, characterized in that, The method comprises the following steps: performing a primary instance segmentation process on an input image to obtain at least one primary target object mask; generating a candidate prompt point set and a region frame prompt set based on the primary target object mask; the step of generating a candidate prompt point set and a region frame prompt set based on the primary target object mask comprises: extracting contour features of the primary target object mask, calculating convex hull feature points, extreme points or curvature key points based on the contour features, and generating the candidate prompt point set; the candidate prompt point set comprises the convex hull feature points, the extreme points and the curvature key points; calculating a minimum circumscribed rectangle covering the primary target object, uniformly dividing the minimum circumscribed rectangle into m*n sub-regions, and generating the region frame prompt set; m and n are both positive integers greater than 1; screening a target prompt information set based on the candidate prompt point set; the step of screening a target prompt information set based on the candidate prompt point set comprises: constructing a multi-dimensional feature vector corresponding to each candidate prompt point, the multi-dimensional feature vector comprising local curvature intensity features, local contour direction change rate features, local region contrast features and contour level weight features; calculating a respective saliency score of each candidate prompt point based on the multi-dimensional feature vector; calculating a screening threshold based on a contour complexity index; screening a target prompt point based on the saliency score; the target prompt point is the candidate prompt point whose saliency score is greater than or equal to the screening threshold; performing a non-maximum suppression process on the target prompt point to eliminate redundant prompt points in the local neighborhood; the contour complexity index is calculated by the following method: extracting a contour point set of the primary target object mask; calculating a contour perimeter and a contour area based on the contour point set; calculating the contour complexity index based on the contour perimeter and the contour area: ； wherein R_complexity is the contour complexity index, L is the contour perimeter, and A is the contour area; performing recursive segmentation based on the target prompt information set and the region frame prompt set to obtain sub-part masks; calculating geometric relationship data between sub-parts based on the sub-part masks.

2. The instance segmentation and computation method based on recursive hints according to claim 1, characterized in that, the step of calculating geometric relationship data between sub-parts based on the sub-part masks comprises: calculating a principal direction vector of each sub-part based on a principal component analysis algorithm, and calculating an included angle between the principal direction vectors of any two sub-parts as angle relationship data; calculating the centroid coordinates of each sub-part, and calculating the Euclidean distance between the centroids of any two sub-parts as distance relationship data; determining relative orientation relationship data based on the horizontal difference and the vertical difference of the centroid coordinates of each sub-part; obtaining the geometric relationship data according to the angle relationship data, the distance relationship data and the relative orientation relationship data. 3.The instance segmentation and computation method based on recursive hints according to claim 2, characterized in that, Further comprising: calculating the shape similarity between any two sub-part masks based on shape invariant moments, wherein the shape invariant moments are used to quantitatively describe the global geometric features of the sub-part masks and have translation, scaling and rotation invariance; Calculate a relative area ratio between any two sub-parts based on a number of foreground pixels of each of the sub-part masks; the geometric relationship data further comprises the shape similarity and the relative area ratio. 4.The instance segmentation and computation method based on recursive hints according to claim 1, wherein, Before performing the primary instance segmentation processing on the input image, comprising: Performing multi-scale Gaussian filter denoising processing on the input image to obtain a filtered image; Performing edge enhancement processing on the filtered image under each scale; Selecting an optimal processing scale based on total edge response energy; Performing a morphological closing operation on the outline of the primary target object mask based on the optimal scale image to fill in outline breaks.

5. A system for instance segmentation and computation based on recursive cues, the system comprising: Comprising: An instance segmentation module, configured to perform primary instance segmentation processing on an input image to obtain at least one primary target object mask; A prompt generation module, configured to generate a candidate prompt point set and a region box prompt set based on the primary target object mask; the generation of the candidate prompt point set and the region box prompt set based on the primary target object mask comprises: Extracting contour features of the primary target object mask, calculating convex hull feature points, extreme points or curvature key points based on the contour features, and generating the candidate prompt point set; the candidate prompt point set comprises the convex hull feature points, the extreme points and the curvature key points; Calculating a minimum circumscribed rectangle covering the primary target object, uniformly dividing the minimum circumscribed rectangle into m×n sub-regions to generate the region box prompt set; m and n are both positive integers greater than 1; A prompt screening module, configured to screen a target prompt information set based on the candidate prompt point set; the screening of the target prompt information set based on the candidate prompt point set comprises: Constructing a multi-dimensional feature vector corresponding to each candidate prompt point, the multi-dimensional feature vector comprising local curvature intensity features, local contour direction change rate features, local region contrast features and contour level weight features; Calculating a saliency score corresponding to each of the candidate prompt points based on the multi-dimensional feature vector; Calculating a screening threshold based on a contour complexity index; Screening a target prompt point based on the saliency score; the target prompt point is the candidate prompt point whose saliency score is greater than or equal to the screening threshold; Performing non-maximum suppression processing on the target prompt point to eliminate redundant prompt points in a local neighborhood; The contour complexity index is calculated by: Extracting a contour point set of the primary target object mask; Calculating a contour perimeter and a contour area based on the contour point set; Calculating the contour complexity index based on the contour perimeter and the contour area: ； Wherein, R_complexity is the contour complexity index, L is the contour perimeter, and A is the contour area; A recursive segmentation module, configured to perform recursive segmentation based on the target prompt information set and the region box prompt set to obtain sub-part masks; A relationship calculation module, configured to calculate geometric relationship data between each sub-part based on the sub-part masks.

6. An electronic device, comprising: Comprise a memory and a processor, wherein, The memory is used to store programs; The processor is coupled with the memory and used to execute the program stored in the memory to realize the steps in the instance segmentation and calculation method based on recursive hints in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, A computer readable program or instruction is stored, and the program or instruction is executed by the processor to realize the steps in the instance segmentation and calculation method based on recursive hints in any one of claims 1 to 4.

Images